Ok, apparently the key term is "glue points".
There is a button on the Drawing toolbar which allows them to be added to a shape.
However, the actual documentation on this feature appears to be very slim:
So, here is (an enlarged version of) the icon:
(If this does not appear on your toolbar, you need to customise it it (right click > customise toolbar > select "Drawing" from "Toolbar" drop-down, then scroll down "Commands" until you see "Glue points" and tick the appropriate checkbox.)
Once you have selected glue points, you will get a new "Glue Points" toolbar appear. The first button on this toolbar is for adding them - select any shape, press this button, then click the shape to place a point. Select and drag to move a point.
You can place glue points inside a shape (not just at the borders).
For non-rectangular shapes, you can place them outside the shape so long as that place is within the rectangular bounding box of the shape (marked by where the eight handles are). Attempting to placing outside this area will snap to the edge of the box.
To remove a glue point, ensure the glue point button is pressed (so toolbar is showing), but the insert glue point button (first on toolbar) is not pressed in, and then you can select a glue point (either click on, or click and drag for multiple) and press delete key. You can only delete glue points you have added (i.e. built-in ones cannot be removed in this way).
The next four buttons on the toolbar indicate which direction the connector can leave from. The buttons can be selected in combination. If you select "left" on a right-edge placed point, the connector stretches across the entire width of the shape.
The remaining buttons control how the glue point is positioned when the shape is resized - details here: http://help.libreoffice.org/Impress/Gluepoints_Bar
Update by atsag for newer LibreOffice versions:
Please enable the Glue Points Toolbar (View -> Toolbars -> Glue Points). Then, press the glue pistol icon (which is the left-most) - this reveals some icons on the right. The one with the '+' sign is the appropriate icon to click in order to add a glue point.