If I have a 1080p video played by VLC-player and a graphic card that allows highest resolution 1360x768 on a monitor of native resolution 1366x768 (connected by VGA), how do all these components work together to render the final resolution?
They're all different 1920x1080 (video), 1360x768(graphic card), and 1366x768 (monitor).
VLC itself will interpolate the video to the window size or in full screen to the computers resolution, generally whatever size that is, even much larger or a few pixels different.
The graphics card will output the computers resolution to the Analog VGA output.
To an analog Monitor:
The analog VGA input will then be displayed to the best of its ability, analogs horizontal streams can be easily interpolated and blend well, analog monitors often had areas that were not visualised (overscan) so it is possible that the 6 pixels extra would not change its pix=pix representation.
To a Digital Monitor:
In a digital LCD monitor with VGA input , the analog already has to be converted to digital prior to display, it is also likely to use an interpolation method, even if it was pixel for pixel it is still done as a stream, this 6pix differance would have the horizontal line end in 6pix black again, or interpolated with the rest of the stream. Any negative effects from this small of differance would either not be seen, or would so minimally effect the output as to not be an issue.
In this scenario the analog would be both the biggest blending, and the best interpolation (both good and bad) The lower computer resolution intital, and the analog (also if it needed conversion to digital) would be the biggest negatives. The minor issues here with the pixels would be the least of the deficiencies.
Short of replacing everything :-) I would think that you would still have a very nice picture, that will look fine , and these differances your seeing in numbers are usually worked out using the best method, and should not distract, or cause you to see it badly.
If your seeing a really bad picture it could be:
a) the settings in VLC for the way the video is interpoated and the render type.
b) software settings for the display both the OS, and the GPU software
c) if any interferances exists that seems unrelated to the picture, the VGA cable itself.
d) Settings in the monitor itself
Are you having any real problems with the setup?
Your graphics card detects the display and, finding that the display's max resolution (1366x768) is lower than the graphics cards' own resolution, starts outputting at 1366x768 to the display. VLC decodes individual frames and presents them to the video card for display. Since the frames are larger (1080p) than the display's max size they get downscaled to 1366x768. Whether the video is downscaled by VLC using the CPU, or by the graphics card in the GPU, depends on whether the video is played using software or hardware decoding. This all assumes fullscreen mode. If VLC is playing in a window then the video gets scaled to the size of the video window; that's the only big difference.. the mechanics of the scaling process itself don't change, just the target output size.
