Although Im sure this is technically possible, I wouldnt recommend it.
First off, it wouldnt reduce latency. Latency is the time it takes for information to travel from one location to another, not the amount of data. Just because you have a bigger pipe, does not mean the data is flowing faster, its just that there is more of it.
Just because you have two connections to the internet, does not mean you use them both concurrently. When you download something from the internet, it will pick one path for traffic, say from yours. If you start downloading something else and it takes your neighbors path, you will benefit from a speed increase. But if your neighbor starts downloading something, then any benefits would be lost.
On top of that, 802.11g has a max speed of 54 Mbps, however its rare to obtain that speed. If you both have 20 Mbps download from your ISP, you have saturated your wireless networks capacity. Not to mention if he decides to transfer something wireless within his own network, then your capacity is lost.
If you really wanted to do something like this, it would be much easier to purchase a Small Office Home Office (SOHO) load balancing router and connect the wired internet connections into it. At that point the aggregation would be at the fastest point, not at the wireless level. Then you could each plug your wireless routers into it and still have separate networks.
some new thoughts:
I think you would be better off just paying a few dollars more a month for higher bandwidth for yourself. For all you know, your neighbors will start torrenting movies and eating all the bandwidth. Not to mention if they get caught, you might be culpable, as it might be happening over your network.