Google and third parties know your location from a number of sources:
- Based on your IP address it is possible to reliably, though not perfectly so, know where you are located (nothing to do with WiFi) as an example go here and see what country and city it thinks you're in, I bet it's right http://www.ipaddresslocation.org/
- If you ever used google maps to get directions in your house that would be a separate data point (or cross referenced with the first).
- If you ever found a restaurant or hardware store near you by searching
- If you've ever clicked on an ad that is local (say a nearby store)
- Perhaps you use facebook and it knows your address or perhaps only one or two of your friends' addresses...
- Ever googled your name?
- More....
Since your wireless router doesn't move (I'm assuming it's not your travel hotspot) then it will always be at your house, but that doesn't say to whom it belongs. So long as you don't use an SSID that is your social security number or name then your wifi box is not what gives away your location. The phone book OTOH may very well map your name to an address - if not the public phone book there are plenty of public records that would (own a house or condo?) or semi-public records that would (have a credit card or bank account, how about a social security number).
If you want to remain anonymous online it's virtually impossible (bad pun, sorry). Even using a decent technology like TOR won't get you as far as you might hope - you should really not use the public network from home if you don't want to be identifiable.