I think they'd still get English by default with whatever keyboard they use(if all they did is plug the keyboard in). Some use a standard keyboard with labels stuck on it for their language. A keyboard with a special script written on the letters, would function the same. In order for their language to come up when they hit ABCDE, they go in windows xp, to start..ctrl panel..regional and language options.. and they add for their language. The Win XP CD would be inserted, for certain languages e.g. one that uses a complex script.. that can have dots.. like hebrew.
Then from the taskbar they can choose which language they want, from the ones they installed.
So you could type a foreign language on your computer with your keyboard. (ignore the yellow star in the pic.. somebody just added that in for effect!)

If you look in regional and language options, you see you can use a shortcut to switch to a particular language. And once you install support for right to left languages, you find that CTRL+LEFT SHIFT, causes all text to be aligned left. CTRL+RIGHT SHIFT, causes all text to be aligned right. To type dots e.g. in hebrew, one would enter the letters, then hit CAPS, to turn caps lock on, then shift+num
And most don't do this, but if one doesn't like that mapping.. one can create their own and install it as if installing a new language! MSKLC is the program for that.
People may have folders and filenames, with names with non-english characters. and open them..
I think getting around windows e.g. control panel, device manager, menus in standard programs e.g. windows calculator e.t.c. will still be in english. (at least in windows xp anyway. when I tried. but maybe in future more standard programs will have their menus in a foreign language). I think windows technical settings including error messages might remain in english otherwise technical support could be trickier because people google error messages.
I did find a picture of ms word with arabic menus.. so maybe microsoft office 2003 specifically has that capability. This article shows pictures of that.
http://www.microsoft.com/middleeast/arabicdev/office/office2003/word.aspx#_Toc50211798
But certainly long before then.. just like even in notepad, you can type a foreign language in.