The router directs the packet to a switch.
No it does not. The router send the packet to an IP address.
And for that it needs to know the next hops ARP adrress. So it will use its own cache for that or delay until after it has send an 'ARP who has XXXX?'
But the switch does not change the destination and source MACs.
That means that the router has to know beforehand at which MAC address
belongs to the host with that IP.
Correct. There is nothing special about this case. With our without a switch. A datagram arrives, gets parsed. The routes now knows the destination (and that this destination is not the router itself). Thus the routing tables is checked to determine which network (and which NIC) needs to output to the next hop. And if the next hop's ARP is needed for that.
And the ARP follow the fully normal method: 1a check check. 1b) if not known retrieve and store in cache. 2) send.