112

When attaching data to my emails, I noticed that Thunderbird calculates the total size of the resulting email as much bigger than the files I attached.

Here's a recent example: two images, one at 13MB and one at 3.6MB should in total be approximately 17MB. There were four lines of text. Thunderbird then asked me if I really wanted to send an email with a total size of 22MB.

Where is that difference coming from? 5MB of text sounds like a bit much.

  • 2
    Note that this often affects things like maximum size. If I'm not mistaken Google mail usually allows email of at most 25MB, but the 25MB are computed after encoding, so you cannot send a 25MB image with an email, because when encoded it would actually be too big. – Bakuriu Oct 27 '16 at 6:26
  • 4
    @Bakuriu's comment applies to Outlook+Exchange server as well. I suggest that the underlying question is actually Why do mail clients (often -- Tbird seems better than outlook again) report only the local file size when it's the base64-encoded size that matters? – Chris H Oct 27 '16 at 10:00
  • @MarcksThomas I don't want to argue against the appeal of having one all-including easily searchable source of knowledge against just having all knowledge easily searchable. But is it necessary? I don't think so. - I don't think that the question isn't useful at all, I just think it doesn't fulfil the basic requirements to keep the site free of unnecessary questions and makes it harder to find the really important stuff, that isn't answered anywhere else. That's what we should be doing! - arc_lupus, as I only lurk on this site, usually, my downvote doesn't cout, yet. But as it is, it stands. – Alexander Kosubek Oct 28 '16 at 6:33
  • Related to: superuser.com/questions/568506/… – glenneroo Nov 1 '16 at 21:30
214

Your data was 17 MiB. There are 1024 KiB in an MiB. There are 1024 B in a KiB. There are 8 bits in a byte. So that's 142,606,336 bits.

Base 64 encoding encodes every six bits as a separate byte. So we need about 23,767,722 bytes. Dividing by 1024 twice gets us 22.67 MiB. So that's where the 22 MiB comes from.

Email is a pretty old technology and doesn't assume an 8-bit clean pipe.

  • 79
    To decode that last line a little: base-64 is a way to encode attachments as text using a limited set of "guaranteed safe characters" that wouldn't get garbled by some intermediary equipment, such as a-z, A-Z, 0-9 – Yorik Oct 26 '16 at 21:53
  • 64
    And, once you understand the math in David's excellent answer, you can just multiply the size of the attachments by 4/3 to get the size of the mail message that will be sent (plus the actual text). – Kent Oct 26 '16 at 23:13
  • 12
    Even if email knew it has a full 8 bit pipe there would have to be encoding as it's fundamentally a text stream--some characters serve control functions and thus must not occur in your data. That being said, there are better encoding techniques but they haven't been adopted. – Loren Pechtel Oct 27 '16 at 4:24
  • 3
    @LorenPechtel you can happily have an application/octet-stream part in a MIME message. All you have to do is choose a boundary that doesn't occur in the data. – OrangeDog Oct 27 '16 at 8:51
  • 8
    what base64 actually does, is using 4 bytes for every 3 original bytes. While this sounds similar, it is important because the length is always a multiple of 4, and also because there is no reason to the bit level. – njzk2 Oct 27 '16 at 15:44
50

Why is the email bigger?

Because the data is encoded in base64 which encodes groups of up to three bytes as groups of four printable ASCII characters. Typically, these groups of printable characters are then split into lines.

The result is that the encoded data is just over 1⅓ times the size of the original data.

Why is base64 used?

Email has a long history and was originally designed to carry text. Only byte values representing ASCII printable characters could reliably pass through the wide variety of email systems on the planet.

So MIME divised two schemes for encoding other data as ASCII text - "quoted-printable" designed for mostly ASCII text with a few other bits, and "BASE64" for arbitrary binary data.

There have been extensions to the SMTP protocol to try and remove these restrictions. First, 8BITMIME in 1994, which allowed higher octet values but unfortunately didn't remove limits related to line lengths and line endings, so was not suitable for arbitrary binary data; and then BINARYMIME in 1995, which allowed transfer of messages containing arbitrary binary data.

However, these standards have not seen widespread adoption. One problem is, what happens if one hop in the mail chain supports them but the next hop doesn't? The mail server then can't send the mail on as-is, it must either reject it as undeliverable and bounce it (which is unlikely to be acceptable to users), or convert it (which requires significant extra code in the mail server). Conversion is made especially painful by MIME rules regarding not using content transfer encodings on multipart types.

  • 1
    I wonder why yEnc, on the other hand, was quite successful in Usenet at displacing UUE. Possibly because binary newsgroups put a much higher pressure on ISPs than an occasional binary email? – igorsk Oct 30 '16 at 19:09
  • 2
    @igorsk: plus Usenet/NN was presented and understood as lossy, where you could publish an article and not all subscribers on all servers would necessarily receive it. There were (and largely remain) customs about quoting in a followup 'enough' of the previous article(s) that your followup can be understood by someone who didn't get the previous article(s). In contrast most (nonspammer) email senders expected 'the system' would get their message to the named recipient(s), although sometimes after hours or days; today people complain about even short delays. – dave_thompson_085 Oct 31 '16 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.