Machine Learning Techniques (MCA556)

From your syllabus.

Supervised Learning

In supervised learning:

Input data is given
Correct output (label) is known
Model learns relationship between input and output

Examples:

Spam detection
Disease prediction
Student result prediction

Logistic Regression

Used for classification problems.

Unlike Linear Regression, Logistic Regression predicts categories.

Examples:

Pass / Fail
Spam / Not Spam
Yes / No

Sigmoid Function

Logistic Regression uses the Sigmoid Function.

Output range:

0 to 1

Interpretation:

Close to 1 → Positive Class
Close to 0 → Negative Class

Applications of Logistic Regression

Email spam detection
Disease diagnosis
Loan approval
Fraud detection

Support Vector Machine (SVM)

SVM is a powerful classification algorithm.

Goal:

Find the best boundary that separates classes.

Example:

Students
Pass  ● ● ● ●

-----------
Boundary

○ ○ ○ ○
Fail

The boundary is called a:

Hyperplane

Advantages of SVM

High accuracy
Effective in high dimensions
Works well with small datasets

Kernel Function

Sometimes data cannot be separated by a straight line.

Kernel functions transform data into higher dimensions.

Types:

Linear Kernel

Used for linearly separable data.

Polynomial Kernel

Creates curved boundaries.

Radial Basis Function (RBF)

Most commonly used kernel.

Sigmoid Kernel

Similar to neural networks.

Neural Network

Inspired by the human brain.

Consists of:

Input Layer
      ↓
Hidden Layer
      ↓
Output Layer

Used for:

Classification
Prediction
Pattern recognition

Artificial Neuron

Basic building block of neural networks.

Components:

Inputs
Weights
Summation
Activation Function
Output

Perceptron

Simplest neural network model.

Developed by:

Structure:

Inputs
  ↓
Weights
  ↓
Activation
  ↓
Output

Used for binary classification.

Limitations of Perceptron

Cannot solve complex non-linear problems.

Example:

XOR problem

Multilayer Neural Network

Contains multiple hidden layers.

Input
 ↓
Hidden Layer 1
 ↓
Hidden Layer 2
 ↓
Output

Advantages:

Handles complex patterns
Better prediction

Backpropagation

Most important neural network learning algorithm.

Purpose:

Update weights
Reduce prediction error

Steps:

Step 1

Forward Pass

Prediction is generated.

Step 2

Calculate Error

Difference between actual and predicted values.

Step 3

Backward Pass

Error travels backward.

Step 4

Update Weights

Model learns and improves.

Activation Functions

Used to introduce non-linearity.

Sigmoid

Output between 0 and 1.

Tanh

Output between -1 and 1.

ReLU

Deep Neural Network (DNN)

Neural network with many hidden layers.

Input
 ↓
Hidden Layer
 ↓
Hidden Layer
 ↓
Hidden Layer
 ↓
Output

Deep Learning

Branch of Machine Learning using Deep Neural Networks.

Applications:

Image Recognition

Face detection

Speech Recognition

Voice assistants

Natural Language Processing

ChatGPT, translation systems

Self Driving Cars

Object detection

Difference Between ML and Deep Learning

Machine Learning	Deep Learning
Less data needed	Large data needed
Faster training	Slower training
Manual feature extraction	Automatic feature extraction
Simpler models	Complex neural networks

Important Exam Questions

Short Questions

Define Logistic Regression.
What is SVM?
Define Hyperplane.
What is a Kernel Function?
Define Perceptron.
What is Backpropagation?
What is Deep Learning?
What is ReLU?

Long Questions

Explain Logistic Regression with Sigmoid Function.
Explain SVM and Kernel Functions.
Discuss Neural Networks and their architecture.
Explain Perceptron and its limitations.
Explain Backpropagation Algorithm.
Differentiate Machine Learning and Deep Learning.

Quick Revision

Logistic Regression = Classification algorithm.
Sigmoid Function = Converts output to probability.
SVM = Finds best separating boundary.
Hyperplane = Decision boundary.
Kernel = Converts data to higher dimensions.
Perceptron = Basic neural network.
Backpropagation = Weight update algorithm.
ReLU = Most popular activation function.
Deep Learning = Neural networks with many layers.

Next Unit 4:

Decision Trees, CART, Ensemble Learning, Bagging, Boosting, Probability & Learning, Gaussian Mixture Models, Nearest Neighbour Methods. This unit is frequently asked in university exams.