What you could do to get a small install with aptitude, is get a debian netinstall cd, and install it using "Expert install". If there are steps where you're unsure of what to do, just use the pre-set values. When you reach the step where you choose which "modules" (at least that's what I recall it's named, I can't double-check right now) to install, unmark all options ("default install" and "desktop system" should be marked as default). This will exclude a lot of convenient little tools that you won't need anyway.
When everything is set up, it's time to install a graphical user interface (GUI). Login with your chosen credentials, and execute
sudo apt-get install xserver-xorg-core <your favourite wm>. As hinted, this will install a minimal Xorg system and a window manager of your choice. If unsure which wm to choose, I can tip you of openbox, one that I often use and which pretty lightweight. I'm not very knowledgeable about many other options, so if anyone has other suggestions, please post them in a reply.
When the installation is completed, you can start the x-server with
startx. Observe that since you installed X manually, it hasn't been made to autostart on boot. I'm actually not aware of how to make it, so someone will have to fill in on that if necessary.
Next, to making your chosen app autostart with X. In the users home directory, edit the file named .xinitrc (create it if it doesn't exist), and append:
sleep 5; your-app-here to the end.
That should do it, please return here if there are any problems :).