Image Filtering

(kernel-based)


Image filtering is a process by which we can enhance (or otherwise modify, warp, and mutilate) images. I've seen enough posts asking about this that I've made this info file to answer most people's questions. This file has information (so far) only about "compare" filters, which compare a pixel somehow with the pixels around it to filter the image.

Another term that may be important is that we're doing "weighted" filtering. (as opposed to "equal" filtering) That means that different pixels have different importance in calculating the image. When I find out how to do other types of filters, I'll add information about it to this document.


Filtering "modes"

In this document I'll refer to two different types of filtering:

  • Color-based, and
  • Height-based.
Height-based filtering only works on indexed-color images. The most common example of this is a 256-color image. Each color has an index number 0-255 and we can treat that number as the "height", as if on a map.

Color-based filtering ignores the index number of a color and uses its actual red/green/blue components for calculations. This works on 256-color images as well as truecolor images. Note that most good paint programs have rather nice support for color-based filters, but completely lack indexed-color tools.


Performance -- speed

Well, filters will work differently speedwise depending on how they are implemented, and how they are optimized. If you do color-based filters on a 256-color image, it'll probably go rather slow. However, the same thing on a truecolor image would go quite fast. Also, height-based filters on 256-color images are usually fast. Height- based filters on truecolor images are the same as color-based filters.


How it works

The process is fairly simple:

  • Make a copy of the image in memory. We'll use this as the destination for our filter. (makes the filter work much better)
  • Set aside a "final" pixel value variable. For height-based filters, this will probably be an integer; and color-based will use an rgb triplet.
  • For each pixel in the image,
      For the pixel and those around it,
        Get the pixel's value and multiply it by the corresponding value in the filter grid.
        Add this result to the total.
      Now divide this number by the "division factor". And put the "final" pixel value back in the image.
In C, the outermost layer would look something like this:
        for(y=top; y<=bottom; y++)
          for(x=left; x<=right; x++)
          {
            ...
          }
This just sets up a two-dimensional loop to process each pixel in the image.

And now, a side-track: The "filter grid" The filter grid is how you define what the filter does. Here's a 5x5 example:


0 0 0 0 0
0 1 3 1 0
0 3 5 3 0
0 1 3 1 0
0 0 0 0 0
Divide: 21

This filter will "soften" the image. What the numbers mean is this: Imagine that the "hot pixel" (the one we're calculating now) is at the center of the grid, where it has a 5. The pixels around that hot pixel correspond to the numbers around the 5.

This leads back to what we're doing to apply the filter.

We'll want to have another loop to process each of the 25 pixels in that 5x5 grid. For each pixel in the loop, we'll multiply that pixel's value by the number in the same position on the grid. In this example, the pixels that are 2 units away from the "hot" pixel aren't even used (the 0's cancel any effect those pixels might have). Then we add this value to the running total for this pixel.

Notice that the pixel in the center has the highest number. That makes it the most important pixel in the filter, and its color will carry the most "weight" in the resulting calculated color.

Finally, we divide the total by the division factor, and put this value where the original "hot" pixel was.

The loop would now look something like this:

 for(y=top; y<=bottom; y++)      // for each pixel in the image
   for(x=left; x<=right; x++)
   {
     gridCounter=0;      // reset some values
     final = 0;

     for(y2=-2; y2<=2; y2++)     // and for each pixel around our
       for(x2=-2; x2<=2; x2++)   //  "hot pixel"...
       {
         // Add to our running total
         final += image[x+x2][y+y2] * filter[gridCounter];
         // Go to the next value on the filter grid
         gridCounter++;
       }
     // and put it back into the right range
     final /= divisionFactor;

     destination[x][y] = final;

   }
And when it's all done, copy the destination to the source, and get rid of the destination image.

This loop should do a height-based filter on a 256-color image. It will do the filter rather slowly, but it's up to you to optimize.

To do a color-based filter, you'll need to do the same thing, but use three "final" values; for red, green, and blue. If you're doing a color-based filter on a 256-color image, you'll need to do even more work. You'll need to find the closest color in the palette to the "final" color. (this will probably cut the code speed down by a factor of 3 or more)

IMPORTANT NOTE: You really should use two images when doing a filter. If you don't, the results will come out quite strange.


Sample Filters (most are in Digital Artist)

Soften (medium)
0 0 0 0 0
0 1 3 1 0
0 3 9 3 0
0 1 3 1 0
0 0 0 0 0
Divide: 25

Soften (a lot)
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
Divide: 25

Soften (a little)
0 1 2 1 0
1 3 10 3 1
2 10 90 10 2
1 3 10 3 1
0 1 2 1 0
Divide: 154

Sharpen (low) (negative values are useful for creating contrast)
0 0 0 0 0
0 -1 -3 -1 0
0 -3 41 -3 0
0 -1 -3 -1 0
0 0 0 0 0
Divide: 25

Sharpen (medium)
-1 -1 -1 -1 -1
-1 -1 -1 -1 -1
-1 -1 49 -1 -1
-1 -1 -1 -1 -1
-1 -1 -1 -1 -1
Divide: 25

"Soft" Sharpen (weird... try it repeatedly on an image)
-1 -1 -1 -1 -1
-1 3 4 3 -1
-1 4 13 4 -1
-1 3 4 3 -1
-1 -1 -1 -1 -1
Divide: 25

Diagonal "shatter"
1 0 0 0 1
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
1 0 0 0 1
Divide: 4

Horizontal Blur
0 0 0 0 0
0 0 0 0 0
1 2 3 2 1
0 0 0 0 0
0 0 0 0 0
Divide: 9

The "fire" filter (creates the "fire" effect, when used repeatedly)
0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 1 1 1 0
0 0 0 0 0
Divide: 4


Other ideas...

It's possible to do quite a bit with filters like this. The fire effect is done this way, and the water effect is done in a very similar way too. Depending on the filter you use, you could create some very interesting effects.

However, another type of filter is also useful: The "contrast" filter. It works by contrasting each pixel with those around it, instead of comparing the values. I don't really know how to get this to work correctly, though; but it can do things such as embossing and edge-finding.

You may also want to try filters that have independent red, green, and blue filter grids (and do something different with each) to get some really strange (and neat) effects. I plan on implementing this in Digital Artist sometime, though I don't know when.


Testing filters without having to write a program for it

This is a plug for my paint program.

To test filters, you can use Digital Artist (it's on my web page). It allows the user to create custom filters (and has several defaults ones) to apply to images. You can apply these filters both in color-based mode and height-based mode. (so you could treat an image as a 3D landscape)

I've found this program to be very useful, and I hope others do too.


(written in the mid 1990's)
Last modified: November 15, 2003 @ 2:55 MST
Copyright (C) 1996-2017 Selene Scriven