Network traffic includes a lot of extra bytes for network headers etc. etc. Does this get measured as part of data usage by ISPs?
Generally yes, but it also depends on the ISP. I did not hear about ISP that excludes headers from data usage.
Excluding headers may not be an easy task to do. Also, what headers should be excluded if they want to do so (IP/TCP/UDP headers). This requires more inspection on all flowing traffic which will be expensive in terms of processing for a gateway to do it unless they really need it.
...a lot of extra bytes...
In practice it adds up to at maximum ~1% of TCP usage (look at the specifications of header length and data length in a packet), or "more or less unnoticeable" (10MB per 1GB). You also have resent packages and the like from collisions, ACK packets and other things. It is nothing to be concerned over.
And as was said: headers and everything mentioned above is also data. If you have a bad cable at home so that many packets have to be resent, why should the ISP give this data "for free"? You are charged for data, not "effective data" as per some unclear standard.