This is SuperUser, so I assume this is a desktop system. For a server system, the answer might be entirely different.
For desktop systems, a high swappiness (such as 60) can be very bad for interactivity, namely when you often run over 50% of RAM usage (e.g. during editing large or many files (photos, videos ...), doing 3D work, gaming, ...). The key point here is, the single task working with that much data will work fast and flawlessly, even on high swappiness, because Linux will see that the memory is in active use and won’t swap it out.
However, as soon as other tasks come into play, this doesn’t work anymore. If you for example work on your 3D scene and then switch over to a file manager or your mail client, it is very likely that these have been swapped out already and it will take a few seconds for them to be swapped in (and possibly something else swapped out before). During that, the application is not responsive at all. This can make for a frustrating and slowing down user experience during every-day desktop work.
So when often using a lot of memory for a single task and still needing to use other tasks, it is sensible to reduce the swappiness. But what if, as common on desktop systems, the other use case is having several tasks only using a few memory, with a low total memory use (say, below 50%)? In that case, having a low swappiness also doesn’t do any harm, because there is lot’s of free RAM and swapping wouldn’t kick in even with a high swappiness.
When having lots of tasks consuming much memory, you’re in a bad position anyways, because swapping will occur. In that case you have to take a trade off: do you need file system responsiveness (high swappiness, because there will be more space allocated for file system caches, even during memory pressure) or task responsive ness (low swappiness, take as much memory for tasks as possible).