One way to install Ubuntu would be to use network installation and TFTP. You'll need to set up a TFTP server on another computer on the network and set the old PC to boot from network. There is a guide which describes needed steps here.
If the computer doesn't support booting from network, then you could try booting from floppy. The idea is to install GRUB on the floppy and use it to run the installation.
If you can connect the hard disk to another computer, then there is this guide which may help. It's idea is to make partitions and copy installation files to the hard disk on another computer and then use the floppy to boot the installation files.
If you can't to that, there are guides to install Ubuntu using floppies here, this guide explains how to install Ubuntu using floppies by making minimal boot floppy which will download data from the Internet.
Some of the guides are a bit old, so you may have to install older version of Ubuntu and later upgrade, or try your luck by experimenting with newer versions.