I was trying a new player called MPlayerX on OSX. It is showing subtitles as squares as shown below. It seems to be a problem with the settings. I tried different fonts, but it doesn't solve the problem. Any idea how to fix this?
I do not have a Mac, but this should be a universal advice. Use some advanced text editor like TextWrangler to open the subtitle file (could also try something like Aegisub) to find out whether the subtitle file is corrupted or not.
If the file is OK, then find out which encoding it uses – you did not tell us, which language is it?
In some cases the font or the system could not have support for certain languages or the player just has set incorrect encoding for the subtitles. UTF-8 should be universal solution, but sometimes the subtitles also use other encoding.
It seems like eighter the subtitles file is corrupted, it is encoded with a wrong charset, or your player just doesn't work properly. You should try VLC Media Player, the best and simplest, yet powerful player I've every tried.
Hope it solves your problem!