On my computer or on the router? Or both? I know how set them on each, I just need to know which one, or if I really need to bother.
Well, it depends.
You should be using your ISP DNS servers, and that's automatically set if you use a dynamic ip address. If you can't or don't want to use them for some reason I'd set it on the router as then DNS server settings are network wide, but setting them per computer will overrule that. I'd then set them on any system you didn't want the DNS settings on the router to be in effect for