UNIT 4: IMAGE RESTORATION

 

Introduction to Image Restoration

Image Restoration is the process of recovering an original image that has been degraded by known or unknown factors (such as blur, noise, or motion).
It focuses on model-based correction, not just enhancing the image visually.

• Goal: Retrieve the most accurate version of the original image.

---

2. Image Formation Models

The mathematical relationship between the original image, the degradation, and the observed image.

General Model:

$$g(x, y) = h(x, y) * f(x, y) + \eta(x, y)$$

Where:

• $g(x,y)$ = degraded image

• $h(x,y)$ = degradation function (e.g., blur)

• $f(x,y)$ = original image

• $\eta(x,y)$ = noise

• $*$ = convolution operation

---

3. Noise Models

Describes how random variations corrupt an image.

Common Noise Types:

• Gaussian Noise: Random values from a normal distribution.

• Salt-and-Pepper Noise: Random white and black pixels (high contrast noise).

• Poisson Noise: Related to photon counting in sensors.

• Speckle Noise: Common in radar and ultrasound images (multiplicative noise).

Noise Model Example:

$$g(x, y) = f(x, y) + \eta(x, y)$$

4. Restoration Techniques

---

(a) Inverse Filtering

• Idea: Directly undo degradation by applying the inverse of the degradation function.

Formula:

$$F(u,v) = \frac{G(u,v)}{H(u,v)}$$

• Sensitive to noise → not ideal when noise is significant.

---

(b) Wiener Filtering

• Idea: Reduces overall mean square error between estimated and original image.

Formula:

$$F(u,v) = \left( \frac{H^*(u,v)}{|H(u,v)|^2 + S_\eta(u,v)/S_f(u,v)} \right) G(u,v)$$

Where:

• $H^*(u,v)$ = complex conjugate

• $S_\eta$ = noise power spectrum

• $S_f$ = original signal power spectrum

• Balances both deblurring and denoising.

---

(c) Least Squares Filtering

• Idea: Minimizes the mean squared difference between restored and original image.

• Often uses regularization to prevent noise amplification.

---

(d) Recursive Filters

• Idea: Filter output depends on both current and past inputs/outputs.

• Efficient for large images.

• Useful for real-time restoration (low computational load).

---

(e) Maximum Entropy Method (MEM)

• Idea: Chooses the solution with the maximum entropy (least amount of prior information) that fits the observed data.

• Especially useful when multiple possible restorations exist.

---

(f) Blind Deconvolution

• Idea: Restores an image without prior knowledge of the degradation function.

• Requires iterative techniques to estimate both the image and the blur simultaneously.

Common Techniques:

• Iterative blind deconvolution

• Regularized blind deconvolution

---

(g) Bayesian Method of Noise Removal

• Idea: Uses Bayesian probability to model prior knowledge about noise and image.

• Restoration by maximizing the posterior probability.

Bayes Rule:

$$P(f|g) = \frac{P(g|f)P(f)}{P(g)}$$

Where:

• $P(f|g)$ = probability of original image given the observed

• $P(g|f)$ = likelihood

• $P(f)$ = prior

• Especially effective when statistical models of noise and the image are available.

---

5. Image Reconstruction

Process of recovering an image from incomplete, noisy, or indirect measurements.

Examples:

• CT scans (from projections)

• MRI images

• Astronomical imaging

Common Techniques:

• Radon Transform

• Iterative Reconstruction Algorithms (e.g., ART - Algebraic Reconstruction Technique)

---

Summary Table

Topic	Key Point
Image Formation	Convolution + Noise
Noise Models	Gaussian, Poisson, Salt-Pepper, Speckle
Inverse Filtering	Direct deblurring (very sensitive to noise)
Wiener Filtering	Balances deblurring and noise reduction
Least Squares	Error minimization
Recursive Filters	Fast, history-dependent filtering
Maximum Entropy	Picks least-biased solution
Blind Deconvolution	No knowledge of blur needed
Bayesian Methods	Probabilistic noise removal
Image Reconstruction	Recover from incomplete data

---

In Short:

Image Restoration focuses on undoing known distortions (unlike enhancement which just improves appearance).
It uses mathematical models like inverse filtering, Bayesian methods, and more, to faithfully recover the original image from the corrupted one.

unit 3 IMAGE ENHANCEMENT

 

Introduction to Image Enhancement

Image Enhancement refers to improving the visual appearance of an image or to convert the image to a form better suited for analysis by a human or machine.

• Goal: Highlight important features and suppress irrelevant details.

• Applied in areas like medical imaging, satellite imaging, robot vision, etc.

---

2. Point Operations

Operations that modify each pixel independently without considering neighboring pixels.

Types:

• Image Negative: Inverts the intensities to highlight hidden details.

• Log Transformation: Expands dark pixel values and compresses bright values.

• Power-Law (Gamma) Transformation: Controls overall brightness.

• Contrast Stretching: Increases dynamic range of pixel intensity.

• Thresholding: Converts image into binary by setting a threshold.

Formula Example for Negative:

$$s = L - 1 - r$$Where:

• $r$ = input pixel

• $s$ = output pixel

• $L$ = maximum intensity level (256 for 8-bit)

---

3. Histogram Modeling

A histogram represents the frequency distribution of intensity levels in an image.

Key Techniques:

• Histogram Equalization:

  • Improves contrast by redistributing pixel intensities.

  • Makes the histogram uniform.

• Histogram Specification (Matching):

  • Adjusts the histogram to match a specified distribution.

Use:

• Brightening dark images

• Enhancing contrast without prior information.

---

4. Filtering and Spatial Operations

Operations that modify a pixel based on its neighboring pixels.

Spatial Filters Types:

Type	Purpose
Low-Pass Filter (Smoothing)	Reduces noise, blurs edges.
High-Pass Filter (Sharpening)	Enhances edges and fine details.
Median Filter	Removes "salt-and-pepper" noise.
Mean Filter	Smooths by averaging neighborhood pixels.

Spatial Operations:

• Convolution: Applying a mask/kernel across the image.

• Gradient Operators (Sobel, Prewitt): Detect edges.

---

5. Transform Operations

Enhancement using transformations in a different domain (frequency domain).

Key Techniques:

• Fourier Transform: Enhances based on frequency content.

• Hadamard Transform: Enhances using square wave decomposition.

• Discrete Cosine Transform (DCT): Highlights important low-frequency components.

• Wavelet Transform: Provides multi-resolution analysis for local and global features.

Use:

• Enhancing textures, denoising, image restoration.

---

6. Multi-Spectral Image Enhancement

Enhancing images that have multiple bands (e.g., satellite images with visible, infrared, etc.).

Key Techniques:

• Band Combination: Merging different spectral bands to create a new image.

• Principal Component Analysis (PCA): Reduces redundancy and enhances major variations.

• Color Composite Techniques: Assigns different spectral bands to RGB channels to highlight features.

Applications:

• Environmental monitoring

• Agriculture (crop health)

• Urban planning (land cover classification)

---

Summary Chart

Category	Techniques
Point Operations	Negative, log, gamma, contrast stretch
Histogram Modeling	Equalization, specification
Spatial Filtering	Mean, Median, Sobel, Prewitt
Transform Domain	Fourier, Hadamard, DCT, Wavelet
Multi-Spectral	PCA, band combination, color composites

---

In Short:

• Image enhancement is about making an image more useful or attractive.

• It can be done in spatial domain (point, neighborhood operations) or transform domain (frequency based).

• Multi-spectral images allow new dimensions of enhancement beyond human vision.

Unit 2 - Image Quantization and Image Transform Theory based answer

 

1. Sampling Theorem

▶ Theory:

The Sampling Theorem, also known as the Nyquist-Shannon Sampling Theorem, is the foundation of digital signal and image processing. It states that a band-limited analog signal can be perfectly reconstructed from its samples if it is sampled at a frequency greater than or equal to twice the maximum frequency present in the signal.

▶ In image processing:

Images are sampled in both horizontal and vertical directions. Insufficient sampling leads to loss of detail and aliasing.

---

 2. Anti-Aliasing

▶ Theory:

Aliasing is the effect of different signals becoming indistinguishable when sampled, leading to visual artifacts like moiré patterns. It occurs when the sampling rate is too low.

Anti-aliasing techniques involve pre-filtering the image using a low-pass filter to remove high-frequency components before sampling. This ensures the sampled image retains important features without distortion.

---

 3. Image Quantization

▶ Theory:

Quantization is the process of mapping continuous values into a finite set of discrete values. In images, this usually refers to reducing the number of gray levels or color values.

• Spatial Quantization → Reduces resolution.

• Intensity Quantization → Reduces the number of brightness levels.

▶ Example:

For an 8-bit image, intensity values range from 0–255. Reducing it to 4 bits maps all values into 16 levels.

Quantization introduces errors (quantization noise), but with intelligent algorithms, quality can be preserved.

---

4. Orthogonal and Unitary Transforms

▶ Orthogonal Transforms:

• Orthogonal transforms use basis vectors that are mutually perpendicular.

• They preserve energy and allow lossless transformations.

• Examples: DCT, DFT, Haar, Hadamard.

▶ Unitary Transforms:

• A unitary matrix is the complex counterpart of an orthogonal matrix.

• It satisfies: $U^H U = I$ (conjugate transpose of U times U is identity).

• Useful in transforms involving complex values, e.g., DFT.

---

 5. Discrete Fourier Transform (DFT)

▶ Theory:

The DFT transforms a signal or image from the spatial domain to the frequency domain. It represents the image in terms of its frequency components, where low frequencies describe smooth areas, and high frequencies describe edges and noise.

▶ Applications:

• Image filtering

• Image compression

• Frequency analysis

---

 6. Discrete Cosine Transform (DCT)

▶ Theory:

The DCT expresses an image as a sum of cosine functions oscillating at different frequencies. It’s similar to the DFT but uses only cosine components, making it real-valued and more efficient.

▶ Advantage:

• DCT is highly efficient in energy compaction, making it ideal for image compression, e.g., JPEG.

---

 7. Hadamard Transform

▶ Theory:

The Hadamard transform uses a matrix with only +1 and -1 values and operates on image data using simple addition and subtraction. It is orthogonal and fast to compute.

▶ Use:

• Image compression

• Pattern recognition

---

 8. Haar Transform

▶ Theory:

The Haar transform is the earliest wavelet transform. It represents data as a set of averages and differences, making it ideal for multi-resolution analysis (processing the image at multiple scales).

▶ Properties:

• Simple and fast

• Good for edge detection

• Used in image compression and analysis

---

9. Karhunen-Loeve Transform (KLT) / PCA

▶ Theory:

KLT is a statistical method that transforms data into a set of uncorrelated variables using eigenvalue decomposition. It is data-dependent and optimal for decorrelation and energy compaction.

▶ Steps:

1. Compute the covariance matrix.

2. Calculate eigenvectors and eigenvalues.

3. Project the image onto these eigenvectors.

▶ Applications:

• Face recognition (Eigenfaces)

• Compression

• Dimensionality reduction

unit 2 - Image quantization and Image Transforms

1. Sampling Theorem

 Definition:

The Sampling Theorem (Shannon-Nyquist) states that a continuous signal can be perfectly reconstructed from its samples if it is sampled at twice the maximum frequency present in the signal.

 Formula:

$$f_s \geq 2f_{max}$$

where:

• $f_s$ = sampling frequency

• $f_{max}$ = highest frequency component in the signal

 Application:

Used in digitizing analog images to ensure no information is lost during sampling.

---

2. Anti-Aliasing

Aliasing:

Occurs when sampling is done below the Nyquist rate, leading to overlapping frequency components and distortion.

Anti-Aliasing:

A process to suppress high frequencies before sampling using low-pass filters to prevent aliasing.

---

3. Image Quantization

 Definition:

Process of mapping a range of continuous pixel values to a finite number of levels.

 Types:

• Scalar Quantization: Each pixel is quantized independently.

• Vector Quantization: Blocks of pixels are quantized together.

Quantization Error:

$$\text{Error} = \text{Original Pixel} - \text{Quantized Pixel}$$

Too few levels → loss of detail, visible banding.

---

4. Orthogonal and Unitary Transforms

 Orthogonal Transform:

A linear transformation using an orthogonal matrix $T$ where:

$$T^T T = I$$

• Preserves energy.

• Examples: DFT, DCT, Haar, Hadamard.

 Unitary Transform:

A generalization using complex numbers:

$$T^H T = I$$

where $T^H$ is the conjugate transpose.

---

5. Discrete Fourier Transform (DFT)

Formula:

$$F(u,v) = \sum_x \sum_y f(x,y) \cdot e^{-j2\pi \left(\frac{ux}{M} + \frac{vy}{N}\right)}$$

• Converts spatial image to frequency domain.

• Captures periodic patterns.

• Used in filtering, compression.

---

6. Discrete Cosine Transform (DCT)

Formula (1D):

$$X_k = \sum_{n=0}^{N-1} x_n \cdot \cos\left[\frac{\pi}{N}(n + 0.5)k\right]$$

• Like DFT, but uses only real cosine terms.

• Energy compaction is high → used in JPEG compression.

---

7. Hadamard Transform

 Properties:

• Uses only +1 and −1 (binary values).

• Fast to compute (no multiplications).

• Not based on sinusoidal functions.

 Matrix:

Hadamard matrix is recursively defined:

$$H_2 = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}, \quad H_{2^n} = \begin{bmatrix} H_{2^{n-1}} & H_{2^{n-1}} \\ H_{2^{n-1}} & -H_{2^{n-1}} \end{bmatrix}$$

---

8. Haar Transform

 Properties:

• Simplest wavelet transform.

• Breaks signal into approximation and detail parts.

• Useful for multi-resolution analysis.

 Steps:

• Divide signal into pairs.

• Calculate average and difference.

• Recurse on averages.

---

9. Karhunen-Loeve Transform (KLT / PCA)

 Definition:

A statistical transform that decorrelates data. Also known as Principal Component Analysis (PCA).

 Steps:

1. Calculate covariance matrix.

2. Compute eigenvalues and eigenvectors.

3. Transform data using eigenvectors.

 Advantage:

• Optimal energy compaction.

• Basis vectors are data-dependent.

• Used in face recognition, compression.

unit -1 Introduction of Image processing

 

What is Image Processing?

Image processing is a method to perform operations on an image to enhance it or extract useful information. It is a type of signal processing where the input is an image, and the output may be either an image or characteristics/features associated with that image.

Goals of Image Processing

• Image Enhancement: Improving visual appearance (e.g., contrast, sharpness)

• Image Restoration: Removing noise or distortion

• Image Compression: Reducing the amount of data required to represent an image

• Feature Extraction: Identifying objects, edges, or patterns

• Image Analysis: Understanding and interpreting image content

• Object Recognition: Detecting and identifying objects in an image

What is an Image?

An image is a two-dimensional function f(x, y), where x and y are spatial coordinates, and f is the intensity (brightness or color) at that point. For digital images, both x, y, and f are finite and discrete.

Types of Image Representation

1. Spatial Domain Representation: Direct representation using pixel intensity values in a grid.

2. Frequency Domain Representation: Using transforms like Fourier to represent the image in terms of its frequency components.

Types of Images

• Binary Image: Only black and white (pixel values: 0 or 1)

• Grayscale Image: Shades of gray (pixel values: 0 to 255)

• Color Image: Consists of multiple channels, commonly RGB (Red, Green, Blue)

• Indexed Image: Uses a colormap or palette to store color information

Image Models

1. Geometric Model: Describes the shape and position of image elements.

2. Photometric Model: Describes the brightness/intensity or color of each point.

3. Color Models:

• RGB: Red, Green, Blue components

• HSV: Hue, Saturation, Value

• YCbCr: Used in video compression

• CMYK: Used in printing

Resolution

• Spatial Resolution: Amount of detail in an image (measured in pixels)

• Gray-level Resolution: Number of distinct gray levels available (e.g., 8-bit = 256 levels)

Image Size

• Described in terms of width × height × number of channels (e.g., 512 × 512 × 3 for RGB)

2D Linear System

• A 2D linear system in image processing refers to a system where the output image is a linear transformation of the input image, usually involving operations like convolution.

• Linearity implies two properties:

  1. Additivity: T[f1 + f2] = T[f1] + T[f2]

  2. Homogeneity (Scaling): T[a·f] = a·T[f]

• Spatial Invariance: The system's response doesn’t change when the input is shifted.

• Example: Applying a kernel (filter) over an image using convolution is a classic example of a 2D linear system:

  $$g(x, y) = \sum_m \sum_n h(m, n) \cdot f(x - m, y - n)$$

Luminance

• The measured intensity of light emitted or reflected from a surface in a given direction.

• Closely related to the perceived brightness, but it's a physical quantity.

• Important in grayscale and color image processing.

Contrast

• The difference in luminance or color that makes an object distinguishable from others or the background.

• High contrast makes features pop; low contrast makes the image appear flat.

• Often enhanced using techniques like contrast stretching or histogram equalization.

Brightness

• A subjective visual perception of how much light an image appears to emit or reflect.

• Can be increased by adding a constant to all pixel intensities.

Color Representation

Images can be represented using various color models, each suitable for different applications:

RGB (Red, Green, Blue)

• Additive color model (used in screens).

• Each color is a mix of Red, Green, and Blue components.

CMY/CMYK (Cyan, Magenta, Yellow, Key/Black)

• Subtractive color model (used in printing).

HSV (Hue, Saturation, Value)

• Hue: Color type (0° to 360°)

• Saturation: Color purity

• Value: Brightness of the color

YUV / YCbCr

• Used in video processing.

• Separates brightness (Y) from color information (U and V or Cb and Cr).

Visibility Functions

• Visibility functions describe how sensitive the human eye is to different spatial frequencies.

• The Contrast Sensitivity Function (CSF) is a common example. It shows that humans are:

  • Most sensitive to mid-range spatial frequencies

  • Less sensitive to very low or very high frequencies

• Important in compression algorithms and display optimization.

---

Monochrome and Color Vision Models

Monochrome Vision Model

• Uses only intensity (luminance) values.

• No color, only grayscale from black to white.

• Basis of early vision systems and useful in medical/scientific imaging.

Color Vision Model

• Based on how the human eye perceives color using three types of cones:

  • L (long wavelengths) → Red

  • M (medium) → Green

  • S (short) → Blue

• Color models (like RGB, HSV) are built around this biological model.

• Opponent Process Theory: Human vision processes color differences (Red-Green, Blue-Yellow) rather than absolute colors.

Shannon Nyquist Theorem

 What should be the ideal size of the pixel? should it be big or small?

The answer is given by the shannon nyquist theorem. As per this theorem, the sampling frequency should be greater than or equal to 2 ✕ fmax, where fmax is the highest frequency present in the image.

SubSampling 

The key idea in image sub-sampling is to throw away every other row and column to create a half-size image. When the sampling rate gets too low, we are not able to capture the details in the image anymore.

Instead, we should have a minimum signal/image rate, called the Nyquist rate.

Using Shannons Sampling Theorem, the minimum sampling should be such that :

$$f_s\ge 2f_{\max }$$

Simple Image Model

 Simple Image model 

I(x, y, l) = σ(x, y, l) × L(l)

Mach Bands

Mach band effect is a phenomenon of lateral inhibition of rods and cones, where the sharp intensity changes are attenuated by the visual system.

Components of Digital Camera

The essential components of a digital camera are as follows

A subsystem of sensors to capture the image. The subsystem uses photodiodes to convert light energy into electrical signals.

A subsystem that converts analog signals to digital data.

A storage subsystem for storing the captured images. A digital camera also has an immediate feedback system to see the captured image. Digital cameras can be connected to computers through a cable, to transfer images to the computer system.