New learnings:

1. Pre-processing of images by subtracting mean activity over the training set from each pixel:

Intuition:

• Purpose: The goal of mean subtraction is to normalize the data by centering it around zero. This helps in stabilizing the optimization process during training, as it ensures that the input data has a mean of zero.
• Why it helps: Neural networks and other machine learning models often perform better when the input data is centered around zero. This is because:
  • It reduces the impact of biases in the data.
  • It helps the optimization algorithm (e.g., gradient descent) converge faster by ensuring that gradients are more balanced across dimensions.
  • It can improve the numerical stability of the model.

Maths:

Compute the Mean Image

• Suppose you have a training set of N images, each of size H×W (height × width). Let’s denote the pixel at location (i ,j) in the kth image as

  $$  x_{i,j,k} $$

• The mean image is computed by taking the average value of each pixel across all images in the training set. Mathematically, the mean value for pixel (i,j) is:

  $$ μ_{i,j} = \frac{1}{N} \sum_{k=1}^{N} x_{i,j,k} $$

  Here, $μ_{i,j }$  is the mean value of the pixel at location (i,j) across all images.

Subtract the Mean Image

• Once the mean image μ is computed, it is subtracted from each image in the training set. For the kth image, the normalized pixel value at location (i,j) is

  $$ x'{i,j,k}=x{i,j,k}−μ_{i,j} $$

  Here, $x'_{i,j,k}$ is the normalized pixel value.

Result:

• After this process, the mean value of each pixel across the training set will be zero. This means the data is now centered around zero, which is the desired outcome.

Example

Suppose you have a training set of 3 grayscale images, each of size 2×22×2: