Your biggest limitation is the number of concurrent network connections that can be made to the client OS (Windows XP). Windows XP will only support a maximum of 10 connections while server will support an unlimited amount.
The 10 concurrent connections are included as part of the CAL you receive when purchasing WinXP.
Server (2003/2008) will come standard with 5 user/connection CALS meaning from a legal standpoint, if you will have more than 5 users or connections accessing services served up by the server you will have to buy additional CALS (legal standpoint only, server will still work).
On Remote Desktop
Will only support a single session. If someone is logged in on the console, and then you log in using Remote Desktop, you will take over the desktop session.
Will accept up to 2 Remote Desktop connections, and one console session for a total of 3 sessions. You can buy additional Terminal Services licenses to support more users (licenses are tracked and you cannot expand the number of Remote Desktop/Terminal Server users without buy additional licenses).