Investigation into the effect of batch size on batch normalisation during inference for image colourisation

The Pix2Pix architecture is widely used for image colourisation. This is the problem of transforming a greyscale image into a realistic colour image. However, the canonical Pix2Pix colourisation model uses batch normalisation during inference, which makes the model output dependent on the other images in the inference batch, and leads to excessive colourfulness in its output. In this work, we analyse the effect of small batch sizes on the colourfulness of the Pix2Pix model output. We propose a method for measuring image colourfulness, allowing us to study the colourisation problem quantitatively. We then propose a method for correcting the output of the batch normalisation layers of the Pix2Pix colourisation model. This reduces its dependence on batch size and enables inference of realistic colour images at small batch sizes.