0

I understand an operating system like Linux or Windows is written in C/C++ and compiled to a given architecture (for example AMD64) to produce a machine code that is suitable for that architecture.

My questions -

  1. Why a separate specification is needed for the binary code - Linux uses ELF and Windows uses Portable Executable Format?
  2. Can an operating system and a program that runs on that operating system be created without this binary format specification?
  3. Is the binary format: architecture dependent, OS dependent or both?
  4. Is the binary format applicable to only executable files or it is applicable to the operating system code as well?
3

As you may have guessed, executable formats contain more than just machine code. For example, they can:

  • Specify metadata for the OS, e.g. which architecture the executable is intended for. This metadata comprises the header of the file.
  • Specify the program layout in memory. On modern OSes, most executable are not loaded into memory in a single chunk - they usually have many separate regions/sections/segments. Some of these segments will contain executable code. Some of them will contain immutable data, like text strings. Some of them will be designated as writable memory, for the program heap(s).

    Different programs will have different requirements (requests) for the sizes of these sections. This is all specified in the header.

  • Some formats will also allow you to embed a digital signature, which allows verification of where the binary came from.


  1. Why a separate specification is needed for the binary code - Linux uses ELF and Windows uses Portable Executable Format?

The reasons are primarily historical, and OSes tend to stick with their existing 'native' (or 'default') format unless there is compelling reason to switch (as has happened e.g. from the DOS MZ format to PE in NT 3.1, and from a.out to ELF in Linux 1.2, and from COFF to ELF in various Unixes over the years).

It should be noted that the underlying machine code depends on the CPU architecture but is otherwise (syscalls and linked libraries aside) largely portable between OSes. In fact, modern Windows and Linux can run both executable formats: ELF executables will run on Windows via WSL, and PE executables will run on Linux via WINE.

  1. Can an operating system and a program that runs on that operating system be created without this binary format specification?

We go back to the primary purpose of these formats. Without the metadata telling the OS where to load parts of the program, most modern executables cannot run. Some very old formats like COM pretty much contain pure code, but are not particularly flexible and have fallen out of favour.

In practice, there is no need for an operating system to even exist. At the hardware level, assuming the existence of a (legacy) BIOS, the BIOS will simply start executing at a specific location on the disk (MBR), which can be arbitrary machine code that then takes over and either starts an OS or does anything else it likes. (You could view the MBR itself as a binary format, though it is not directly related to the executable code.) The more modern UEFI, however, does specify a more complex executable format (PE).

  1. Is the binary format architecture dependent, OS dependent or both?

Depends on the format, really. Some formats assume a specific architecture. Others allow you to select the architecture from a list of specified "magic numbers" in the header. Still others are completely architecture-agnostic (e.g. Java and .NET/CIL bytecode).

Similarly, the format generally imposes no restriction on the OS, though the OS will be restricted in which formats it can (natively) recognise and execute. Of course, compatibility layers on top of the core OS can execute other formats (e.g. JVM, .NET/CLR, WSL&WINE, etc.) that the core OS may not recognise.

  1. Does binary format applicable to only executable files or it is applicable to the operating system code as well?

A large portion of most modern operating systems is just a collection of "normal" executable files. However, some parts of the operating system are "special" and will not necessarily use the same format as the rest. Usually, this applies to just the bootloader and kernel.

To pull a specific, very common, example, the legacy BIOS bootloader will not be in either the ELF or PE formats used by Linux and Windows. The Linux kernel is generally built in an ELF-derived format that the GRUB bootloader can load, but it could be in a different format to be compatible with the bootloader used. The Linux kernel also supports an EFI Stub mode, which contains a minimal PE/COFF header to be compatible with direct UEFI boot.

3

The main use of the format is to load programs in memory, which is done by a loader.

  1. Why a separate specification is needed for the binary code - Linux uses ELF and Windows uses Portable Execution Format?

Polite answer: because the OS is different and has different requirements. Unix person answer: because Microsoft likes to reinvent the (square) wheel.

  1. Can an operating system and a program that runs on that operating system be created without this binary format specification ?

Yes, but only the very trivial ones, a fairly basic OS that runs trivial programs that don't need to "relocate" and don't need to link to any external code.

  1. Is the binary format architecture dependent, OS dependent or both ?

None, the ELF format is used with many operating systems and architectures.

  1. Does binary format applicable to only executable files or it is applicable to the operating system code as well ?

In Linux, the bootable kernel may not be ELF, but the kernel modules use the ELF format.

  • 1
    "Unix person answer: because Microsoft likes to reinvent the (square) wheel." Funny you should say that, considering PE (NT 3.1, ~1993) is older than ELF (Linux, ~1995 migration from a.out). In the modern world, PE is the closest to COFF, which was also previously used by Unixes. – Bob Feb 5 '18 at 1:07
  • Yes, and almost everyone moved to ELF (except Windows and macOS). – xenoid Feb 5 '18 at 1:39
  • 2
    Which is in no way MS reinventing the wheel. The decision to (not) change to a new format after release is very different from inventing a new one "just to be different". Pedantic? Yes, probably. But your answer is not being very objective there, jokingly or not. – Bob Feb 5 '18 at 1:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.