I have a 4096 MB and a 1024 MB DDR3 memory stick. They both have the same clock. I know they won't work in dual channel speeds because they have different sizes. But will they work better alone than together?
There is a performance hit, but in most real-world scenarios it should be negligible - you'd probably have to run RAM benchmarks to see the difference.
Also note that all recent Intel-based systems1 support asynchronous dual-channel, which in your case would mean that 1+1 GB would work in dual-channel mode and only the remaining 3 GB (of the 4 GB stick) would be in single-channel mode. This makes the performance difference even less noticeable.
Provided you actually need 5 GB of RAM, I'd say go for it.
1 Most systems from 2004 or later, and definitely all that support DDR3 RAM.