The issue

When we draw image in canvas, we may do scaling up or scaling down the image size. For example, if the image has size 100*100, we may want it to cover the whole canvas. If the canvas size is 200*200(bigger than image size), then this is scaling up. If the canvas size is 50*50(smaller then image size), then this is scaling down.

When we do scaling up, then we need to decide how to fill the extra spaces(image smaller then canvas). When we do scaling down, then we need to decide how to do with the extra pixels(image bigger then canvas). This process is called resampling.

The api in canvas

In canvas, we have 2 apis to control this feature: imageSmoothingEnabled and imageSmoothingQuality. imageSmoothingEnabled is used to control if we want to enable the image smoothing feature, and imageSmoothingQuality is used to control the quality of the image smoothing effect.

We can use them like below:

function showScaled(img, scale) {
  const width = img.width * scale;
  const height = img.height * scale;

  const originalCanvas = document.createElement('canvas');
  originalCanvas.width = width;
  originalCanvas.height = height;
  document.body.appendChild(originalCanvas);
  const originalCtx = originalCanvas.getContext('2d');
  // originalCtx.imageSmoothingEnabled = false;
  originalCtx.imageSmoothingQuality = "high";
  originalCtx.drawImage(img, 0, 0, width, height);
}

How it works

Take scaling up as an example. Since we are scaling up, so we will have some extra pixels need to fill in values. In below diagram, p is the extra pixel and q is the pixel in original image.

 q1             q2


        p



 q3             q4

One of the most simple method is the nearest neighbor. We can calculate the distance of position p and its surrounding position, find the nearest neighbor and use its value.

In code, it will looks like:

function nearestNeighbor(
  width,
  height,
  scaledWidth,
  scaledHeigt,
  scale,
  pixels,
  scaledPixels
) {
  for (let y = 0; y < scaledHeigt; y++) {
    for (let x = 0; x < scaledWidth; x++) {
      const nearestX = Math.round(x / scale);
      const nearestY = Math.round(y / scale);

      const scaledPosition = x * 4 + y * scaledWidth * 4;
      const originalPosition = nearestX * 4 + nearestY * width * 4;

      scaledPixels.data[scaledPosition] = pixels.data[originalPosition];
      scaledPixels.data[scaledPosition + 1] =
        pixels.data[originalPosition + 1];
      scaledPixels.data[scaledPosition + 2] =
        pixels.data[originalPosition + 2];
      scaledPixels.data[scaledPosition + 3] =
        pixels.data[originalPosition + 3];
    }
  }
}

Nearest neighbor is efficient and simple, but may not provides good effect. Another commonly used method is bilinear.

In nearest neighbor, we only choose the nearest neighbor's pixel value, in bilinear, we will consider the 4 surrounding values.

 q1    r1       q2


        p



 q3     r2      q4

Specifically, if we want to calculate the value of p, we will calculate r1 and r2 first. To calculate r1, we will consider pixels q1 and q2. We also take into the distance between them. Let's see the code for the calculation details.

function bilinearInterpolation(
  width,
  height,
  scaledWidth,
  scaledHeight,
  scale,
  pixels,
  scaledPixels
) {
  for (let y = 0; y < scaledHeight; y++) {
    for (let x = 0; x < scaledWidth; x++) {
      const xOrigin = x / scale;
      const yOrigin = y / scale;

      const x1 = Math.floor(xOrigin);
      const y1 = Math.floor(yOrigin);
      const x2 = Math.ceil(xOrigin);
      const y2 = Math.ceil(yOrigin);

      const q1Pos = (y1 * width + x1) * 4;
      const q2Pos = (y1 * width + x2) * 4;
      const q3Pos = (y2 * width + x1) * 4;
      const q4Pos = (y2 * width + x2) * 4;

      for (let channel = 0; channel < 4; channel++) {
        const q1 = pixels.data[q1Pos + channel];
        const q2 = pixels.data[q2Pos + channel];
        const q3 = pixels.data[q3Pos + channel];
        const q4 = pixels.data[q4Pos + channel];

        const r1 = q1 + (xOrigin - x1) * (q2 - q1);
        const r2 = q3 + (xOrigin - x1) * (q4 - q3);
        const pixelVal = r1 + (yOrigin - y1) * (r2 - r1);

        const scaledPos = x * 4 + y * scaledWidth * 4;
        scaledPixels.data[scaledPos + channel] = pixelVal;
      }
    }
  }
}

There are also some more sophisticated methods. But the idea is the same.

Lastly, let's see the effect in the example image.

const img = new Image();
img.onload = () => {
  showOriginal(img);
  showScaled(img, 5);
  show(img, 5, nearestNeighbor);
  show(img, 5, bilinearInterpolation);
};
img.src = './star.png';