The official minimum recommended RAM for a desktop system is 512Mb though I have run 9.04 on an old physical machine with only 384Mb and it was slow but just about acceptable (my main machine was playing up so I used the old machine as an emergency measure to get some mail read+sent and a document thrown together).
Below 512Mb the OS will be using swap space, but in a VM environment reading back from swap may be an operation that reads the data from the hosts cache rather then the physical disk (and if you have the "optimise for speed" rather than "optimise for safety" option on writing to swap will be buffered too) so you can get away with giving the VM less RAM such as the 256Mb you quote as long as you are fine with it possible being slow for a short while when you go to use it for the first time after some hours away (as any info from the VM will in that case most likely have been replaced in the cache by more recently used bits of your filesystems. I would not give the VM less than 256Mb though.
To reduce the RAM use you might try lubuntu instead of Ubuntu: it is the same base system but with a less demanding desktop environment and less installed/running by default. You might be able to push the allocated RAM for the VM down to 192Mb or perhaps even 128, or if you stick to 256Mb it should be more responsive than Ubuntu's standard setup.