If you take a screenshot of a 1980x1020 monitor and have 16px text at "100%", a single glyph of the text itself is going to be a max of 16px high and maybe 12-18px wide.
If Libre Office is like Word (very likely) then the internal rendering size of a page is going to be 250-300ppi at standard paper size (i.e. 8.5 x 11). They then scale it for presentation to you at the screen resolution/zoom level that you have set.
The full screen shot at 100% pixel size is going to be 3.6 inches (1080/300), but that 16px glyph of type rendered at 100% pixel size is going to be (16/300= .05 inches).
So usually people scale up the image and this reduces quality severely, the larger you go. Bear in mind the 16px glyph is mostly white space and the lines and curves are only made up of a handful of individual pixels. The general rule of thumb for professional printing is to not scale up more than 2x or down more than .5x
So the problem is not scaling down a high-res image, but scaling up a low res one, and again, what matters here is not the total size or resolution of the image, but the number of pixels used to resolve the type that you are interested in.
If you want to get the type more clear, zoom in on the type as far as you can to fill the screen, then take the capture. You now have 1080px of type instead of 16px. The zooming difference here is that you are using math-based vector images (fonts) which scale "infinitely" and then fixing the resolution in a screen capture.
If you cannot zoom or need the whole screen at once, then use the highest resolution your monitor will support, presuming you have high-dpi aware scaling enabled in the OS.