Suppose we go to a cyber cafe that uses chromebooks. Since the OS is designed to make it difficult to install software keyloggers (by tampering the OS), and it is also difficult to install hardware keyloggers, should I fear that my keystrokes could be being logged?
Yes. Always assume that your activity -- including your keystrokes -- could be monitored/recorded any time you are using a computer that you do not physically control.
Sure, Chrome OS may make it nigh impossible to install a software keylogger, but "difficult" or even "extremely difficult" is not the same as "impossible". Further, you have no way of knowing that the OS at that cyber cafe is in fact Chrome OS, as opposed to something else that's been carefully designed to look like Chrome OS. Or, alternatively, it might be a completely legit Chrome OS, but running under a hypervisor of some kind that is itself logging your keystrokes.
As for hardware keyloggers: They aren't difficult to install at all. If the keyboard isn't physically built into the computer (i.e. not a laptop keyboard), it's as simple as plugging in a potentially very tiny device (apparently these things are now wi-fi enabled even!) between the keyboard's cord and the port on the computer. And you know those little round thingies you often see on e.g. monitor cords? In theory (though in practice I know of no such device) a keylogger could be designed to be attached to the keyboard cable in much the same way, reading the keyboard input via the magnetic field the electrical signals inherently generate without actually making physical contact with any wires. (I do know of "vampire" attachments for hardware network sniffers that function in this manner.) Further, keyboards can have built-in keyloggers, requiring someone to merely swap out a legit keyboard with an identical-looking on with a built-in keylogger.
Upping the complexity a little, a legit keyboard almost always has a single controller chip on it; this chip could relatively easily be replaced, or another chip added to the same connectors, to turn a legit keyboard into one with a built-in keylogger.
Laptop keyboards are much harder to get hardware keyloggers into, but, again, "much harder" is not the same thing as "impossible".
Currently a lot of Chrome OS is being used based on one person (hexxeh) compilation (Flow, Lime, Vanilla etc.), it doesn't seem to be anyone involved or co-operate with him. So as far as you know he could be putting a keylogger in his build and laughing all the way to the bank - how can you tell !? (Has anyone actually investigated/monitored his build !?)
If we're just talking about the ChromeOS you get when you purchase a new Chromebook then I'd say you have less to fear than with another flavor of computing device.
As @Kromey says, anything can be compromised and standard PCs have a lot of potential vulnerabilities. A Chromebook has a built-in keyboard and display so the peripherals are more secure. On top of that, the OS is built with a number of security features like process namespacing, storage block hashing, and a kernel signature to reduce the threat of tampering to the underlying OS (see https://www.youtube.com/watch?v=1u3mTNj3TJ0).
That leaves your Chrome web pages, extensions and web apps as the biggest risk. If the keylogger is injected into a web page via the web server then there's not much you can do about it. Both extensions and web apps require users to choose to install them; if you choose not to then you're safer. The content of these extensions can be reviewed by any interested party, it's all open source if you know where to look. And lastly, any extension or web app from the Chrome web store has to name the permissions it wants in it's manifest file so users have a chance to know what the thing can do to the web pages they're viewing.
Overall, I believe that Chromebooks run the most secure consumer OS out there. In part due to the custom hardware it sits on and in part due to the reduced functionality (and attack surface) of the system.