Machine Learning Techniques (MCA556)
From your syllabus.
Dimensionality Reduction
In Machine Learning, datasets may contain many features (columns).
Example:
Student Data
------------
Name
Age
Gender
Address
Attendance
Marks
Projects
Activities
...
Too many features can:
- Increase training time
- Increase memory usage
- Cause overfitting
Dimensionality Reduction reduces the number of features while preserving important information.
Benefits
- Faster computation
- Less storage
- Better visualization
- Reduced overfitting
Principal Component Analysis (PCA)
Most important dimensionality reduction technique.
Purpose:
- Convert many features into fewer important features.
Idea:
- Preserve maximum variance.
- Reduce dimensions.
Example:
100 Features
↓
PCA
↓
10 Important Features
Applications:
- Face Recognition
- Image Compression
- Data Visualization
Linear Discriminant Analysis (LDA)
Used for:
- Dimensionality Reduction
- Classification
Difference:
| PCA | LDA |
|---|---|
| Unsupervised | Supervised |
| Maximizes variance | Maximizes class separation |
Applications:
- Face recognition
- Pattern classification
Factor Analysis
Statistical method used to identify hidden factors affecting data.
Example:
Student Performance depends on:
- Intelligence
- Study Hours
- Motivation
These hidden variables are called factors.
Applications:
- Psychology
- Market Research
- Social Sciences
Independent Component Analysis (ICA)
Separates mixed signals into independent components.
Example:
Two people speaking simultaneously.
ICA can separate:
Mixed Audio
↓
ICA
↓
Speaker 1
Speaker 2
Applications:
- Signal Processing
- Medical Data Analysis
- Audio Separation
Locally Linear Embedding (LLE)
Non-linear dimensionality reduction technique.
Assumption: Nearby data points remain nearby after transformation.
Used when data lies on a curved surface.
Applications:
- Pattern recognition
- Data visualization
Isomap
Isomap = Isometric Mapping
Advanced dimensionality reduction technique.
Purpose:
- Preserve geometric structure of data.
Applications:
- Image analysis
- Visualization
- Pattern recognition
Least Squares Optimization
Used to minimize prediction error.
Idea:
Find the best line that minimizes squared errors.
Linear Regression is based on Least Squares Optimization.
Evolutionary Learning
Inspired by biological evolution.
Key concepts:
- Selection
- Mutation
- Crossover
- Survival of the fittest
Used to solve optimization problems.
Genetic Algorithms (GA)
One of the most important evolutionary algorithms.
Inspired by natural selection.
Basic Terminology
Chromosome
A possible solution.
Population
Collection of chromosomes.
Fitness Function
Measures solution quality.
Higher fitness = Better solution.
Genetic Algorithm Steps
Step 1
Initialize Population
Generate random solutions.
Step 2
Evaluate Fitness
Check quality of each solution.
Step 3
Selection
Choose best solutions.
Step 4
Crossover
Combine parents to create offspring.
Parent A + Parent B
↓
Child
Step 5
Mutation
Randomly modify genes.
Purpose:
- Maintain diversity
Step 6
Replacement
Create next generation.
Repeat until optimal solution found.
Applications of Genetic Algorithms
- Scheduling
- Route Optimization
- Machine Learning
- Robotics
- Engineering Design
Reinforcement Learning (RL)
Learning through rewards and punishments.
Agent learns by interacting with environment.
Components of RL
Agent
Learner.
Example: Robot
Environment
World around the agent.
Example: Road
Action
Decision taken by agent.
Example: Move Left
Reward
Feedback received.
Example:
Correct Action → +10
Wrong Action → -5
Reinforcement Learning Process
Agent
↓
Action
↓
Environment
↓
Reward
↓
Learning
Applications of Reinforcement Learning
- Self-driving cars
- Robotics
- Game playing AI
- Resource management
Markov Decision Process (MDP)
Mathematical framework for Reinforcement Learning.
An MDP contains:
- State (S)
- Action (A)
- Reward (R)
- Transition Probability (P)
Example of MDP
Robot Navigation:
State:
Current Position
Action:
Move Left / Right
Reward:
Reach Destination
Next State:
New Position
Markov Property
Future state depends only on the current state.
Not on previous history.
Important Exam Questions
Short Questions
- What is PCA?
- Define LDA.
- What is ICA?
- Define Isomap.
- What is Genetic Algorithm?
- What is Fitness Function?
- Define Reinforcement Learning.
- What is MDP?
Long Questions
- Explain PCA with advantages.
- Differentiate PCA and LDA.
- Explain Genetic Algorithm with steps.
- Discuss Evolutionary Learning.
- Explain Reinforcement Learning architecture.
- Explain Markov Decision Process.
Quick Revision
- PCA = Reduce dimensions while preserving variance.
- LDA = Reduce dimensions while separating classes.
- ICA = Separate mixed signals.
- Isomap = Preserve geometric structure.
- GA = Optimization inspired by evolution.
- Population = Collection of solutions.
- Fitness Function = Quality measure.
- RL = Learning through rewards.
- MDP = Mathematical model for RL.
- Markov Property = Future depends only on current state.
✅ Machine Learning Techniques (MCA556) is now complete.
Next Subject Options
- .NET Framework with C# (MCA552)
- Compiler Design (MCA554)
- Optimization Techniques (MCA555)
- Advanced JavaScript (MCA557 B/C)
For exams, I would suggest Compiler Design next because it is usually considered the toughest paper and benefits from early preparation.
