General troubleshooting tips...
Make sure you have the latest Linksys firmware on your WRT54G (beware that there are many different hardware revisions of the WRT54G, and many devices with similar model numbers, so be sure to get your model number and hardware revision exactly right when you go to the Linksys website looking for the latest firmware version). Update your Question with your hardware revision and firmware version and exact model number of your Linksys device.
Also make sure you've applied all software updates to your MacBook Air. Update your Question with your exact MacBook Air revision (from System Profiler; like "MacBookAir 3,1") and your exact OS version.
In case the channel your WRT54G is picking isn't the cleanest, try manually setting the channel. Try channels 1, 6, and 11, and see which one works best.
Try to be specific with your description of symptoms and troubleshoot different symptoms as different problems. If Wi-Fi stays connected but the Internet doesn't work, that's probably a whole separate problem that you need to investigate/troubleshoot/diagnose/fix separately from case where you fully lose your Wi-Fi association.
What signal strength (RSSI) do you usually see from your AP in the spots in your home where you're using your MacBook Air? Hold down Option while clicking on the Wi-Fi Menu Extra in Mac OS X to see the RSSI and various other stats for your current network connection.
What do your system log and kernel logs say regarding Wi-Fi or other network problems? You can crank up the Wi-Fi logging in Mac OS X using the /usr/libexec/airportd command, but the syntax varies between OS version, and since you didn't provide your OS version, I can't tell you exactly what to type. There's even a Wi-Fi Diagnostics app in Lion in
/System/Library/CoreServices/, but I don't know if you're running Lion or not. Please always try to be as specific as possible and include as much information as possible when you ask technical questions to a technical audience like this. It's frustrating to get questions where the asker left out easy-to-include details that could have made a difference. Please see How to Ask Questions the Smart Way.
Also note that the Linksys WRT54G was introduced in late 2002, so even if you bought your unit more recently than that, it's still basically a decade-old design. Maybe it's time to consider buying a modern AP. Your MacBook Air is capable of 300mbps operation, so it's a pity to keep it shackled to a decade-old AP that can only do one-sixth the speed.