To boot from your second HDD, you have to tell your BIOS to boot from said HDD. Telling it to boot from a CD drive (that no longer exists) won't work.
Double check that there isn't an option in your BIOS' boot order for the second HDD itself (if both are the same model, you might see two identical entries - try choosing the second one if the first boots from your old hard disk). If there is no such option, there are two plausible causes for this:
- You didn't actually connect the second HDD properly. To test this you can simply check whether you can access it when booting from the other one.
- Your BIOS doesn't actually support a second HDD. While this is rather unlikely, it certainly is a possibility if the manufacturer never meant for the CD drive to be replaced. (Though the very fact that this was a CD drive tells volumes about the age of the computer and thus its BIOS, which slightly increases this likelyhood.)
However, I think the actual solution to your problem doesn't even include changing the boot order inside the BIOS at all:
It sounds like you want to keep a different OS installed on either hard drive and be able to change between those two in a somewhat convenient manner.
If that is the case, installing a boot loader like GRUB to the first HDD and using that to choose your OS is probably the way to go.