Modern digital displays (like LCDs) that use DVI use the DVI-D flavor. That means there are no pins above and below the blade. The included HDMI-to-DVI is DVI-D, so that it can plug into such monitors (or monitor extension cables). If it were to-DVI-I, it would not fit because there are no sockets for the pins around the blade on many/most digital displays.
To convert DVI to VGA, the other end has to be either DVI-A or DVI-I, because those are the only two that carry the analog signals. Once you have used an HDMI-to-DVI-D adapter, those DVI-to-VGA adapters won't work.
Furthermore, there are no simple HDMI-to-DVI-A (or -I) cables/adapters, because they are not directly compatible. HDMI does not carry an analog signal. You need an active converter; HDMI-to-VGA converters exist, but are in the $100 range.
I've never seen a DisplayPort to DVI-I adapter. Even if such a thing existed, you might as well skip the extra conversion step and get a Mini DisplayPort to VGA adapter. Not necessarily the $29 one made by Apple, although it might be more likely to "Just Work".