A lot of the reasons are historical. That doesn't mean that they don't make sense today.
Issues in Portability
When naming a file, you may also have to consider how other (file) systems will treat that file name. A character in a file name may be fine for your system, but it may be an issue for another system.
So, as long as there was the slightest possibility that you may want to be able to access the file easily from an older system, you'd pick only safe character. This may include booting into an old recovery system you kept around or the fear that recent Windows versions are still somehow based on MS-DOS.
A file system may limit the length a file can have. This was even more serious during the days when MS-DOS was limited to 8.3 filenames. So, leaving out spaces enabled you to put more meaningful characters into the name.
Several other file systems also defined strict limits on their file name length. Wikipedia has a table in the article about file system comparison for those that want the details.
MS-DOS also defined the space character as a reserved character. This is due to the fact that the space character was used for padding in the FAT. Additionally, MS-DOS did not provide for an escaping system in the shell.
Most command-lines I am aware of use the space character as a parameter delimiter. When neglecting to properly escape a filename, it can have dire consequences as parts of the filename can be interpreted as parameters to the application you wanted to call.
Consider the difference between
rm foo bar
rm "foo bar"
The WikiPedia article linked above even points out ambiguity introduced by missing to properly escape a command:
Ambiguity can be prevented either by prohibiting embedded spaces in file- and directory names in the first place (for example, by substituting them with underscores '_'), or, if supported by the command-line interpreter and the programs taking these parameters as arguments, by enclosing a name with embedded spaces between quote characters or using a escape character before the space, usually a backslash ('\'). For example
Long path/Long program name Parameter one Parameter two ...
is ambiguous (is "program name" part of the program name, or two parameters?); however
Long_path/Long_program_name Parameter_one Parameter_two ...,
LongPath/LongProgramName ParameterOne ParameterTwo ...,
"Long path/Long program name" "Parameter one" "Parameter two" ...
Long\ path/Long\ program\ name Parameter\ one Parameter\ two ...
are not ambiguous.
Uniform Resource Locators (URL)
When trying to describe the location of a file, using a URL, spaces need to be escaped.
Characters can be unsafe for a number of reasons. The space
character is unsafe because significant spaces may disappear and
insignificant spaces may be introduced when URLs are transcribed or typeset or subjected to the treatment of word-processing programs.
Thus, a space has to be replaced with a
%20 instead. This makes the filename part of the URL less readable and, thus, makes people avoid it in the first place.