Let's say, for example, that you're using Google as your OpenID provider and logging into SE with this OpenID (so in OpenID vocabulary, SE is a relying party).
In this scenario, Google is responsible for authenticating you. In practical terms, that means you don't need to set a password on SE. You only have a password with Google, and only Google knows it. In particular, you don't need to trust SE to keep your password safe.
The same goes for any personal information that you may have filled in your account. You only have to trust Google with it, not SE, because SE will only see the personal information that Google allows it to see. (Different providers offer you more or less control over what relying parties can see)
Another property of OpenID (which can be seen as a benefit or a problem) is that you only need to store your personal information once. For example, if your email address changes, you only need to tell Google, and not the many different relying parties you may have set up to handle your OpenID. (The reason this can be a problem is if the OpenID provider doesn't give you enough control over which relying parties can see what information, so that you end up e.g. telling everyone your age when you only wanted to convey it to a particular website that has age restrictions.)
Yet another different property of OpenID (which can also be good or bad) is that you can have the same persona on multiple sites. This is done on Stack Exchange: you can link your account at Stack Overflow with your account at Super User and your account at Server Fault and so on. The sites know that you're the same person, and not just someone who's happened to choose the same user name, because you use the same OpenID on all sites.
