I think you could do something like this with Imagemagick. It has image quantization and histogram analysis features that you'll probably need to give this a real treatment.
The simplest thing to do is count the number of unique colors in each picture - cartoons should generally have fewer than photos. This may work as is if your search space is fairly simple. i.e. differentiating simple cartoons form color photos. If you have 'fancy' cartoons, you may have to add additional checks. I added an extra echo for RGB vs. Grey color space before checking each image.
A more sophisticated test might involve checking the histogram, either total or in RGB space of each image.
#!/bin/bash
for i in `ls *.jpg`
do
echo "$i is `convert $i -format \"%[colorspace]\" info:`"
x=`convert $i -unique-colors txt:- | wc -l`
if [ $x -le 512 ]; then
echo "$i is cartoon like ($x)"
elif [ $x -le 1024 ]; then
echo "$i is a bw photo ($x)"
else
echo "$i is real life-like ($x)"
fi
done
The main complication is separating complex computer animations from b&w photos. A B&W photo may have relatively few unique colors in it, while a sophisticated cartoom may have thousands due to computer aided shading. You'll probably need to experiment with the thresholds for 'X' depending on what your images look like.