To distinguish colors I relied entirely on RGB space. Simply if R>G && R>B and either G or B was 0 then I considered it red, etc. To find the shape, I first found if it was an oval by looking at the column that first contained a colored pixel -- this column has many colored pixels if it is an oval and few otherwise. I distinguished the others by looking at how many columns were colored -- the squigle had fewer. I then counted objects by looking at white space transitions. Finally I determined the shading by looking at a strip in the middle of the object and counting how many pixels were colored.
Here are some images of the program playing set. It prints which cards make up sets in the window on the right. You can't really see it in the picture, but if you zoom in on the windo you can.