The background color is generally frame-specific.
You can set the background color of the
default face to be frame specific using
set-face-background, like so:
(set-face-background 'default "#CCCCCC" (window-frame (frame-selected-window))
Frame customizations are generally controlled through frame parameters. Now, if you look closely at the
background-color for frame parameters, you'll see that the default background color is taken from the
default face - which is why the above elisp has an effect.
Past that, it seems the closes you can come to a buffer background color is by using the minor-mode
buffer-face-mode (added in 23.1) - however that only changes the background for the text of a buffer, and not the entire background.