Sometimes a website will load and immediately start playing audio, whether it be music or just general voice messages. Short of just hitting the mute button on my keyboard, is there a setting in Firefox that will prevent this from happening?
You could block flash explicitly with FlashBlock.
In Windows Vista and later versions, you can mute or reduce the volume for specific software, so you can turn off the audio from your browser; it works the same as the FlashMute option, but without having to install a new plugin, without any compatibility issue, works for any browser and even for non-flash ads.
There are a variety of ways that audio can be played via your browser, so there is no single setting to disable audio. You could conceivably use something like NoScript to block all Java, Flash, and embeds by default, but that may have a greater impact than you want.