1

I am running ext4 on an ubuntu variant. In an effort to conserve disk writes on a limited ssd I am writing data to a mounted microsd card. Can someone explain how journaling would work in this situation?

Would my local filesystem keep a journal when writing to an microsd of undefined format (fat32, ntfs, ext4)? Would the journal files exist on the microsd or on my local ssd? Can you dynamically change where a journal is stored, if so how? Is there a way to change it at mount time?

1

It turns out the journal is actually one of the most flash-friendly write patterns there is. It's just a continuous sequential write. The issue with some flash devices is that they have a very cheap FTL, or Flash Translation Layer. This is the system which takes disk reads and writes to a particular LBA (logical block address, or HDD sector number) and maps it to a particular location on a particular flash chip inside the flash device.

Most SSD's have a decent enough FTL that the journal really isn't an issue. The problem comes with eMMC devices in phones, or microsd cards, where the manufacturers care about shaving every last millicent of their materials cost. (If you are making millions of phones or sd cards, saving fractions of a penny makes a huge difference to your profit.) So the really simple FTL's have real limitations to how many parts of the disk you can write to at any one time, and the really cheap flash devices may be using the cheap flash chips to begin with, which is where you really get in trouble.

As an aside, it turns out I'm mentoring a graduate student at CMU who is working on a "SMR friendly journalling for ext4", that will also be ideal for ext4. This suppresses the final writeback of the random 4k writes to update the allocation bitmaps and inode table blocks, and uses the copy of the metadata block in the journal as the authoritative copy. This is really only going to make a difference for what I call "cheap flash devices" --- although it will make a pretty huge difference.

In answer to your question Linux won't use the sdcard for anything automatically. You can mount it on a particular directory, in which case writes to that directory will go to the sdcard, using whatever file system was originally on the sdcard (unless you reformat it). By default, most sdcards these days usually have FAT32 or NTFS, and the distribution probably dealt with it automatically. In that case, it won't be using journalling at all, and it won't affect how the system writes to the rest of the system.

It is possible to use an external block device, including an sdcard, as an external journal. If you do that, you won't be able to use the sdcard for anything else, and the sdcard must be present after an unclean reboot to help recover your primary file system. After all, that's the whole point of the journal --- to minimize file system corruption and the attendant data loss after a crash or power failure. But it's tricky to set up, and depending on your SSD, it might not make much of a difference at all. I don't know what you mean by "limited SSD", but if it is an SSD at all (instead of a eMMC or micro sdcard) it probably has a better FTL than the micro sdcard. Sure, the journal write patterns really easy even for the crappiest of crappy FTL's, so it's not insane to use a micro sdcard as a journal. But if the SSD's FTL is at all competent, it might not help the SSD at all. And, setting up an external journal is pretty complicated, and not that user-friendly, since it's an advanced topic that generally only makes sense on very specialized setups.

In answer to your question, you can change where the journal is stored, but it requires making changes to the file system so it knows where to find the external journal device. So it's a lot more complicated than just some mount options that you can change at mount time. That's because the really tricky bit is making sure that you can find the external journal device after a crash or power failure, during the boot sequence, so you can recover your file system. Depending on your distribution, and whether it uses an initial ramdisk, and whether it needs to load modules in order to support the micro sdcard reader, it may require making changes to how your system boots.

Also, keep in mind that if you drop your device, and the micro sdcard goes flying out of the machine, with a missing journal the device will immediately crash, and you will need to find and replace the sdcard so you can recover the system afterwards. So I really wouldn't recommend using an external, removeable storage device for a journal unless you really, really, really know what you are doing, and are aware of all of the consequences.

It's like walking around with a glass bottle full of nitroglycerin. Sure, it can be done. There may even be reasons why experts would need to do it. But it might not be something I'd recommend to a newcomer to a chemistry lab to do....

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.