If you can find Windows 3.1 and some software for it, I suppose it'll be fast enough. Windows 98 should also be quite fast on a modern PC.
On the Linux side, you're probably looking at Damn Small Linux, which you can even boot from a usb pen drive.
This is reproduced from their site:
DSL has a nearly complete desktop, and
a tiny core of command line tools. All
applications are chosen with the best
balance of functionality, size and
speed. Damn Small also has the ability
to act as an SSH/FTP/HTTPD server
right off of a live CD. In our quest
to save space and have a fully
functional desktop we've made many GUI
administration tools which are fast
yet still easy to use. What does DSL
have?
XMMS (MP3, CD Music, and MPEG), FTP
client, Dillo web browser, Netrik web
browser, FireFox, spreadsheet,
Sylpheed email, spellcheck (US
English), a word-processor (Ted),
three editors (Beaver, Vim, and Nano
[Pico clone]), graphics editing and
viewing (Xpaint, and xzgv), Xpdf (PDF
Viewer), emelFM (file manager), Naim
(AIM, ICQ, IRC), VNCviwer, Rdesktop,
SSH/SCP server and client, DHCP
client, PPP, PPPoE (ADSL), a web
server, calculator, generic and
GhostScript printer support, NFS,
Fluxbox and JWM window managers,
games, system monitoring apps, a host
of command line tools, USB support,
and pcmcia support, some wireless
support.
You can still extend DSL easily using the Damn Small Linux myDSL Application Extensions.
Although I've never used DSL, I'm a great admirer of this project.