I spent at least a dozen years playing with various hardware configurations trying to build a perfect home server. I started with the cheapest PC under my desk running FreeBSD, then moved to an industrial strength server running Fedora Core OS, then a recycled old laptop with Ubuntu Server, then a hacked Linksys NAS with custom firmware and a bunch of packages for a web server, ftp, etc. In the end, I settled on a vanilla consumer-grade NAS (Network-Attached Storage) and I supplement it with various services "from the Cloud". I use Google Apps to store and share documents, calendars and to host email, for example. I use Google AppEngine to host my many websites (it requires programming skills, check out Google Sites or any number of blog hosting services if you do not want to learn programming). I use http://rsync.net and MobileMe for offsite backup (there's also Mozy and many other services that are easier to use than rsync, for example). I use del.icio.us for bookmarks, flickr.com to store and share images, and so on and so forth.
All this is to say that unless you have a copious amounts of free time to invest into building and maintaining your own infrastructure, you'd be better off my going with the hosted services. Yes, it often costs money (albeit usually not much) and privacy is always a concern, but you won't have to spend your weekends in the "server closet" in your basement trying to patch your crashed media server just so your wife could watch the latest episode of "Project Runway".
The NAS that is my "home server" now: Western Digital MyBook World Edition NAS.