Is it possible to boot compressed kernels with a compressed initrd with PXELinux?

First, a little background:

We created a custom Linux distro, for diskless OpenCL computing nodes. We want those nodes to fetch their OS from the network. Our Distro is composed out of a kernel (duh) and a large initrd which is loaded into RAM and everything is executed from there.

We chose to run everything off the initrd for two reasons:

  • NFS was not an option to serve the filesystem's extra contents
  • Fast file access from RAM.
  • No persistent storage needed, data and config is pulled dynamically through a SOAP service.

Now our initrd is about 450M in size. At our network speeds, it takes about two to three minutes to load a single client. Will compression speed up te downloading, and if yes, which one should be used? Is LZMA supported by PXELinux, or do we need to stick to bzip2 or gzip?

Because of the 2-3 minutes loading time, booting 15 nodes over the same network link takes quite a lot of time. We decided not to use hard drives or CD/DVD drives, for financial reasons (cheapest HDD @ €30 times 15 is a lot of money saved ;-) )

So, our question is: what compression options are available for this setup? And how do we do this?

Thank you for your time!

Yvan Janssens

  • Have you optimized your distro yet? Delete all the documentation, libraries, includes, locales you don't need, strip the binaries and libraries of debug symbols, recompile with optimizations for size... – Marcin May 29 '11 at 16:07

How did you make the initrd? Most systems that I know compress them in the last step:

> file /boot/initrd-
/boot/initrd- gzip compressed data, [...]

The Kernel needs to support the commpression:

> cat /boot/config- |grep CONFIG_RD_

But a 450MB RAM disk means 450MB less memory - and without a HDD you don't have swap.

You should seriously look into a networked file system, there is more than NFS: gPXE can load from iSCSI, AoE and even HTTP.

|improve this answer|||||
  • The nodes all have 2GB ram, and the OS and apps run on 256MB, 512MB peak. So that leaves a ramdisk of 1250MB left ;-). The 450MB is reduced till 250MB in the meanwhile, by stripping unnecessary stuff. That is compressed size, the image is 1024MB. – friedkiwi May 28 '11 at 22:02
  • Unless you use squashfs, the uncompressed image will be in RAM. Does not leave much room in your case. – Turbo J May 28 '11 at 22:10
  • yeah, I know. 2GB RAM, 1GB Ramdrive, 1GB Working mem. Should be enough for console-only, right? – friedkiwi May 28 '11 at 22:39

Have you though about implementing casper and a squashfs. That way you can keep the initrd to a minimal size, maybe around 25 MB and send the squashfs separately. Squashfs supports lzma compression, and if you look at the way parted magic handles it, some times they put the squashfs in the initrd... however in the past I've played with pxelinux, and you can instead send it as a separate file, and the casper scripts will still find it.

|improve this answer|||||
  • Removing more date from the image, by stripping debugging symbols and development programs/libraries did the job. Image is compressed to 260M now, which is acceptable for me, since it's transfered through a GBit connection (takes about 25sec to transfer). – friedkiwi Jun 1 '11 at 21:48

1) a typical Linux kernel is compressed, the same for its initrd, check your building chain.

2) your figures say you don't need squashfs

3) pxelinux does not care about your kernel/initrd compression method if any.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.