There are two separate questions: latency and throughput.
Seek time
Optical drives have random seek times around 100 ms, whereas hard drives are around 10 ms - why is this?
An optical read/write head consists of a laser, detector, mirrors, lens, and multiple voice coils to position the head/lens for tracking/focusing. This entire assembly has a number of parts and is relatively heavy. The entire head assembly moves on a worm gear.

A hard disk head consists of a tiny GMR sensor for reading and coil for writing. The rest of it is the plastic slider and metal arm; the whole assembly is relatively light. The head is moved by a powerful voice coil. This is why seeking is fast.

Transfer rate
CD drives top out at about 10 MB/s. DVD drives top out at about 30 MB/s. Blu-ray drives top out at about 70 MB/s. Hard drives routinely hit 100~200 MB/s.
The two main factors that determine transfer rate are linear speed (metres per second) and linear density (bits per meter).
Desktop hard drives in the 3.5-inch form factor have a platter diameter of about 90 mm. Optical discs have a diameter of about 120 mm. I would rate these as "close enough" for comparison purposes.
Hard drives usually spin at 5400 RPM or 7200 RPM, with old enterprise models going up to 15000 RPM. Optical discs are spun at various speeds, depending on how quickly the host wants to read/write data (e.g. bulk read vs. streaming audio/video), how long the drive has been actively used, how much noise is desired, etc. But optical discs can spin up to about 10000 RPM in real drives without problems. So this is also in the same ballpark as HDD RPM.

Areal density is a huge factor to consider. Pretend for a moment that a CD is 1 GB and an HDD is 1000 GB and they have the same physical dimensions. Clearly, the HDD has 1000× the data density per area. As for linear density, the HDD is √1000 = 32 times denser than the CD. So if you position a head over the discs and make one full revolution, the HDD should read 32× more data than the CD, simply because more data is packed on each track. As we can see, this is why DVDs and BDs have higher transfer rates than CDs. But a 25-GB single-layer Blu-ray disc absolutely pales in comparison to even a cheap, basic 1 TB HDD. Though however, multi-layer optical discs and multi-side multi-platter hard disks make this calculation more complicated.
Bonus
To add insult to injury, my experience shows that after inserting an optical disc into a drive, it takes about 20 seconds to begin reading any user data on it. This start-up time is far worse than flash drives and even hard drives, and is especially painful when sifting through many discs.