I recently had a similar issue and found that it was low quality audio cables that I was using.
I replaced my 3.5mm audio jack cable for a higher quality branded one.
I also replaced generic RCA extension cable for a high quality Philips branded one.
All noise is now gone (at least to my human ear), at all volumes with all devices. Despite extensive testing I have been unable to induce any noise.
The cables contain an earth to ground out background noise, similar to static on analog televisions.
Cables also contain shielding to protect from noise induction caused by electromagnetic interface from other electrical devices and cables.
Cheap cables do not often seem capable of properly providing these functions, possibly because the cable isn't properly manufactured to iEEE standards. This is a low cost manufacturing issue.
Good quality cables made a big difference in ensuring that the noise is filtered out.
The less amplified the original signal the less risk there is of amplifying unwanted noise.
If you have a the following setup
Computer -> Amplifier -> Speakers
You should ensure that the volume on the computer is set to 100% then use the amplifier to control the volume. This is the same for most analog audio, it should always be amplified closest to the source.
This does not include digital audio, such as optical and HDMI.