I am looking for a way to draw E/R Diagrams with Visio. From what I've seen, they seem to look like this
when instead I want something like this:
How is that possible?
What you want to do is look for standard flow chart shapes, not Entity Relationship shapes - this is easily achieved in Visio.
Either create a new document and choose Flowchart, or load the Flow Chart shapes.
The diagram shown is not a flowchart. It is an entity-relationship diagram using Chen's notation. As far as I know, recent versions of Visio do not include Chen ERD stencils, only crowsfoot. You can download a basic Chen ERD stencil set from the Visio Cafe.