Making sure a number is between 0 and 255

Recently while reworking the code for the Ascgen dotNET I needed to make sure that a variable is between 0 and 255 before converting it to a byte, and I needed it to be as fast as possible since it can get called several million times a second.

After realising that there really wasn’t a built in function like:

Result[x, y] = Math.Between(0, 255, pixel);

I initially used the minimum and maximum functions:

Result[x, y] = Math.Max(0, Math.Min(255, pixel));

This works, and is a pretty neat little piece of code. However, after profiling the application it quickly became obvious that it was slowing things down. A look at the compiled code shows that even when built for release, Min and Max are left as function calls instead of being inlined.

Looking into bitwise operations, we see that using a bitwise AND with 255 will trim the value to be between 0 and 255. Unfortunately it loops around, so that isn’t what we need:

  100101100   300
& 011111111   255
= 000101100   044

The next option was to just check it manually:

if (pixel > 255) {
	Result[x, y] = 255;
}
else if (pixel < 0) {
	Result[x, y] = 0;
}
else {
	Result[x, y] = (byte)pixel;
}

Which is better, or:

Result[x, y] = (byte)(pixel > 255 ? 255 : (pixel < 0 ? 0 : pixel));

Which is silly and harder to maintain.

Either way a new problem is that most of the pixel values are between 0 and 255, therefore two checks will usually be required. Thanks to a great suggestion by biznatchio, we can instead do it in one comparison.

We saw earlier that “value &= 255″ will trim the value to be between 0 and 255. So, if we just compare the value to the trimmed value we will know if it is in range:

if ((pixel & 255) == pixel) {
	Result[x, y] = (byte)pixel;
}
else if (pixel > 255) {
	Result[x, y] = 255;
}
else {
	Result[x, y] = 0;
}

This will be slower then before if the value is <0 or >255. As our values will be in range most of the time, this method is the best.

This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.