Other answers here speak about some of the differences. I will get at your following question,
I know both approaches will work, just wondering if there is a reason to prefer one or the other
So let's say you only use your router for access to inet (no VPN or other routing needs)
If I have Router R_A and switches S_A and S_B. Suppose we connect both switches directly to the router. Then suppose I have 2 hosts on S_A (S_A_h1, S_A_h2) and 2 hosts on S_B (S_B_h1, S_B_h2).
If S_A_h1 wants to talk to S_B_h2 how many hops must be made? 3, one through each switch and the router.
If S_A_h1 wants to reach the internet, how many hops must be made in the LAN? 2, one through the switch and one through the router.
Now suppose we connect only S_A to the router and then connect S_B to S_A.
Now if S_A_h1 wants to talk to S_B_h2 how many hops must be made? 2, one through each of the switches.
If S_B_h2 wants to access the internet how many hops must be made? 3, both switches and the router.
So if you want to reduce network latency you will want to set up your network in such a way that reduces hops. e.g. you would probably want to keep your local media server on the same switch as your TV's and your internet access computers on a switch that connects to the router.