It has nothing to do with the viewing angle. The problem are the sub pixels of the display and how Windows thinks they should work.
There is something in Windows called ClearType which, based on the assumption that you use your monitor horizontally uses sub-pixel rendering to smooth fonts. When you turn it vertically this smoothing actually does more harm than good. You can actually 'tune' it in the Control Panel, just search for 'ClearType'.
Also manufacturers position the sub pixels physically under the same assumption (there is a certain geometry which works best for red green blue), and your display will look a little off especially if you use one vertically and one horizontally at the same time. It will seem that one or the other has 'something weird' about it, in the way you perceive the same image on each display. This happens because you can actually sense the difference in the way the sub pixels are arranged, forming a pixel. Sub pixels (red, green, blue) are almost invisible but one pixel which has geometrically different sub pixels will definitely feel different than one that doesn't an entire screen of them more so. Even if they are small, pixels are quite big to the eye.
This effect is less noticeable, by a big margin, on HiDPI displays. The density of actual pixels is very small anything above 200dpi is fine. Sub pixels will definitely be invisible now and their respective pixels will blend much better.
Retina displays come to mind, they are offer no noticeable difference when used horizontally or vertically. Also smaller 4K displays like 23' or 24' like the Dell P2415q pack a lot of pixels per inch and will probably be very nice to use vertically. Larger ones won't as they will probably just be better for the inexperienced user, the effect less noticeable but still there.
PS. You should trust your inner sense, especially after using a monitor for a couple of hours. Everyone will feel the difference, but some will ignore it or blame the headache on something else because they just don't know how sensitive the body is. Once someone points out the link between the two for you you will start developing your own sense of the phenomenon.
PS1. This effect was studied, and probably still is, in dedicated print shops especially Offset printing to achieve the best result possible given the hardware at your disposal.
This is just for pixels, we can go on to say that manufacturers only mention the vertical refresh rate. That's right, a monitor can have and probably always will different refresh rates for vertical and horizontal. This however doesn't apply to LCDs, because
Refresh rate or the temporal resolution of an LCD is the number of times per second in which the display draws the data it is being given. Since activated LCD pixels do not flash on/off between frames, LCD monitors exhibit no refresh-induced flicker, no matter how low the refresh rate. However, high refresh rates may result in visual artifacts that distort the image in unpleasant ways. High-end LCD televisions now feature up to 600 Hz refresh rate, which requires advanced digital processing to insert additional interpolated frames between the real images to smooth the image motion. Such high refresh rates may not be supported by pixel response times, resulting in distorted images.
For a refresh rate of 60 Hz to be displayed correctly, a LCD display would require a response time of approximately 1.667 (5/3) milliseconds GtG (grey-to-grey). In addition to the technical aspects of achieving such a high refresh rate, there are limits to the capability of the human eye. However, improving the response time of LCD pixels would improve the image quality for refresh rates that are on the fringe of what the human eye is capable of processing." http://en.wikipedia.org/wiki/Refresh_rate
This means, for example, that people using 4K x 2K displays @30hz, because they don't have the latest graphics card to drive it at 60hz, won't actually see the flicker associated with a classic monitor @30hz, but will only notice the difference professional games, where no matter how high the frames per second they will always 'see' only 30 of them IF they play at 4K resolution, playing at a lower 1080p will use @60hz for probably @120hz if the monitor supports that actual rate. Some TVs advertise 900hz but they only have 60z given to them by the computer and cable so what they actually do is show the same frame twice or three times.