I was reading up on Base64 encoding, and found this on Wikipedia:

Base64 encoding schemes are commonly used when there is a need to encode binary data that needs be stored and transferred over media that are designed to deal with textual data.

... and the example given is sending binary files via email.

I'm trying to understand why base64 is needed. Since the binary data is a bunch of bytes, wouldn't it be directly translatable to ASCII, which is textual data? Why is base64 needed at all? Or does email have a problem with the control characters in ASCII?

  • What do you mean by "directly translatable"? In what sense is base64 not "direct"? – David Schwartz Mar 18 '12 at 21:38
  • Why do you think it's direct? – Cookie Monster Mar 18 '12 at 21:41
  • 4
    It's not that I think it's direct, it's that I think "directly translatable" is an oxymoron. If "direct" can include a translation process, then what makes base64 not direct? It's just a translation process. – David Schwartz Mar 18 '12 at 21:44

There is a good Wikipedia article on this.

The earliest iterations of NCP as used by ARPAnet were more like bit streams than byte streams, or attempts to negotiate a convenient byte size; the 8-bit byte was only standardized on much later. There were also several attempts at creating file transfer protocols that would work in different machines (mail was initially a function of the FTP protocol, primarily as the MAIL and MLFL commands, then split into MTP, later SMTP.). Those machines often had differing character encodings – ASCII vs EBCDIC – or even different byte sizes, 8-bit bytes vs 6-bit vs ...

Therefore, mail transfer functions were initially defined for transferring relatively short messages in plain text; specifically, "NVT-ASCII". For example, RFC 772 says:


Mail is transferred from a storage device in the sending host to a storage device in the receiving host. It may be necessary to perform certain transformations on the mail because data storage representations in the two systems are different. For example, NVT-ASCII has different data storage representations in different systems. PDP-10's generally store NVT-ASCII as five 7-bit ASCII characters, left-justified in a 36-bit word. 360's store NVT-ASCII as four 8-bit EBCDIC codes in a 32-bit word. Multics stores NVT-ASCII as four 9-bit characters in a 36-bit word.

For the sake of simplicity, all data must be represented in MTP as NVT-ASCII. This means that characters must be converted into the standard NVT-ASCII representation when transmitting text, regardless of whether the sending and receiving hosts are dissimilar. The sender converts the data from its internal character representation to the standard 8-bit NVT-ASCII representation (see the TELNET specification). The receiver converts the data from the standard form to its own internal form. In accordance with this standard, the sequence should be used to denote the end of a line of text.

Even though eight bits was being transmitted over the wire, the 8th bit would often get discarded or mangled, since there was no requirement to preserve it; in fact, some protocols required the 8th bit to be set to zero, such as the initial SMTP RFC as quoted below. In other words, the software was not 8-bit clean.

Data Transfer

The TCP connection supports the transmission of 8-bit bytes. The SMTP data is 7-bit ASCII characters. Each character is transmitted as an 8-bit byte with the high-order bit cleared to zero.

This persisted for a long time even after 8-bit ISO-8859-# character encodings became widespread. Even though some servers were already 8-bit clean, many others weren't, and blindly sending 8 bit data would have resulted in mangled messages.

Later, "Extended SMTP" was published, allowing mail servers to declare SMTP extensions they supported; one of them was 8BITMIME, indicating that the receiving server could safely accept 8-bit data. MIME message parts can have "Content-Transfer-Encoding: 8bit", indicating that they are not encoded in any way.

However, the SMTP protocol remained line-based and has the 998 octet line limit, as well as using a . line (0D 0A 2E 0D 0A) as the "end of message" indicator. This means that even though most binary files could be sent unaltered, it is still possible that files containing this octet sequence would be interpreted as the end of transferred message, and the rest of the file as a SMTP command, possibly causing damage. Similarly, a "line" longer than 998 octets might be cut off by the receiving server.

In 2000, the "BINARYMIME" ESMTP extension was published as RFC 3030, allowing transfers of raw binary data over SMTP. The message is now transferred in chunks of pre-indicated length, with a zero-length chunk used as the terminator, and Base64 & similar encodings are not needed anymore. Unfortunately, few SMTP servers support this extension; for example, neither Postfix nor Exim4 advertise CHUNKING in reply to EHLO. To take advantage of BINARYMIME, it would have to be supported by all servers in the message path, which can be more than just one or two.

See also:

  • 1
    Exchange servers within an organization send email as binary using the BDAT command, but they don't do this for SMTP servers outside of the organization. – james.garriss Mar 14 '13 at 18:43

Some older e-mail systems and software were not 8-bit clean, the 8th bit was used as a control character. This was enough to muck up binary files, thus Base64 (or other encoding schemes) were needed.

  • So are we wasting 2 bit for every byte just because some legacy email system of 90s might not be able to understand message correctly. These legacy system in the era of gmail might be less than 1%. I am thinking why are we wasting so much of bandwidth for handful of people? and is Base64 limited to mails only ? – vaibhav.g Dec 27 '18 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.