SIMD Image Processing
Posted: April 2007
This page is an example of how to use SSE (actually more correctly SSE2) that exists in Pentium4 and AMD64 CPU's to improve performance of image processing functions that increase brightness and YUV422 to RGB conversion. I also have examples here of how to use Altivec on the PowerPC CPU and WMMX on Xscale.
SSE (basically a 128 bit version of the 64 bit MMX instruction set) is Intel's SIMD (Single Instruction, Multiple Data) instruction set included on Pentium III and extended to SSE2 on Pentium 4's. SSE2 adds integer math to SSE's floating point processor.
All assembly code posted on this page is written by me and anyone is free to cut and past it and use it in their own programs. The asm code here assembles with the NASM Assembler.
I have another possibly interesting SSE project here: Mandelbrots with SIMD Assembly.
Update December 12, 2018: I started cleaning up the code, changing it to 64 bit and adding some AVX2 versions of the code (still working on it). I put everything in a git repository linked to at the bottom of this page.
Related Projects @mikekohn.net
Explanation Of Brightness
The brighter test program here reads in a BMP file (color or not color) and converts it to black and white. The image data is stored in a buffer of width*height bytes where each byte represents the brightness of each pixel (0 is black, 255 is white, and all numbers inbetween are shades of gray).
To increase the brightness of an image, the value of every byte in the image buffer is increased by some value. To make the image darker, every byte in the buffer is decreased in value. For a color image, to be technically correct, the image needs to be in YUV format and the Y portion can be treated like a black and white image using this function. If this function was used on a standard RGB buffer, I don't think it would work properly, especially at the saturation points of the buffer, but it's worth a try :). Converting to and from YUV from RGB is pretty computationally expensive.
Explanation Of YUV422 to RGB
YUV is another colorspace that can be used to represent image. An explanation of YUV can be found on Wikipedia here: http://en.wikipedia.org/wiki/YUV. YUV422 planer represents Y as single bytes in the top part of the buffer, while U is represented next at 1/2 the resolution of Y, and V is represented last at 1/2 the resolution of Y. For every 2 Y (brightness) bytes, there is 1 U (color) and 1 V (color).
I wrote 3 different versions of YUV422 to RGB. The first one follows the exact formula of YUV as described on Wikipedia, the second uses a integer/shifting trick to get rid of some of the multiplication and floating points, and the third is based on the floating point version but written in total assembly language using SSE. I was actually able to almost double the speed of the original integer/shift version by using some simple lookup tables to get rid of all the multiplication and saturation, but I haven't posted that version :). Maybe one of these days I'll try an SSE integer version too.
How SIMD helps Brightness
SSE adds eight 128 bit registers to the x86 instruction set. These registers can do load/store operations to and from memory 128 bit at a time (well, in one instruction at least), but when doing math operations the register gets divided up into either sixteen single bytes, eight 16 bit words, four 32 bit double-words, four single precision floats, or 2 double precision floats.
In the brightness example, every single byte of the xmm1 register is loaded with the same single byte value. In the brightness loop, xmm0 is loaded with the next 16 bytes in the buffer. Using paddusb (parallel add unsigned bytes with saturation), xmm1 is added to xmm0. Every byte of the xmm1 register is added to every byte of the xmm0 register. Because paddusb uses "saturation" if the resulting byte would overflow it's simply set to 255. The xmm0 is then written back to memory.
If the value passed to the function was 3, and the memory at the start of the image was 00 01 02 03 04 05 06 07 248 249 250 251 252 253 254 255 (a 16 pixel gradient from black to white):
Things to remember:
How SIMD helps YUV422 to RGB
For the YUV422, I use SSE process 4 pixels at one time. I set up
"vectors" of 4 floating points. In my example I have the following
C Version Of Brightness
SSE2 Version Of Brightness
I got one of these Atmel ATNGW100 embedded computer boards with an AT32AP7000 cpu on it. It's an AVR32 with some SIMD and DSP instructions. I ran the non-SIMD version of brightness on it and I'm trying to figure out how to get ahold of the compiler intrinsics so I can make an SIMD version to test.
Altivec on PowerPC
I've started translating the SSE/x86 code to Altivec/PowerPC for MacOSX and the Cell CPU found in the Playstation 3. After benchmarking this the C code on Playstation 3 Linux, I was kinda disappointed with the results, so I translated it to straight PPC assembly and PPC+Altivec. Unfortunately, I did all the development on MacOSX using the "as" assembler which doesn't appear to be compatible with the "as" assembler on Playstation 3 Linux, so I have to rewrite it. The benchmark on the Mac G4 looks pretty good tho, I'll post the results soon.
In the future i'm hoping to translate the code to one of the Cell's SPU's. I also plan on adding Altivec YUV422 to RGB.
Source Code: image_proc_altivec.asm
The following table shows the difference between the C and SSE2 version of the brighter() function. The time represents how long it took to read in the bmp, call the brighter routine 100,000 times, and then write out a modified bmp. Note: Performance differences could be due to memory bus speed and not to processor speed. I can't remember what speed of memory are in these two boxes, but the AMD64 box is a laptop which typically have slower memory.
Brightness Adjust (100,000 iterations on a 352x240 image)
YUV422 to RGB (10,000 iterations on a 704x480 image)
Note: -m32 tells gcc to compile for a 32 bit cpu while -m64 says to compile for 64 bit
I made a multithreaded version of the yuv2rgb.c. It breaks up the 10,000 interations over multiple threads.YUV422 to RGB (10,000 iterations on a 704x480 image) 64 bit compiled C code only
I've recently started cleaning up this code and moving it to a git repository. I added some AVX2 examples and plan on possibly adding AVX-512.
Copyright 1997-2019 - Michael Kohn