How to allocate limited SSD space in LVM for optimal performance

Question

I just got a new SSD, and I'm looking for advice on how best to incorporate it into my existing LVM setup. I have the following logical volumes (mounted at the obvious places):

# lvs
  LV          VG        Attr   LSize   Origin Snap%  Move Log Copy%  Convert
  home        maingroup -wi-ao  75.00g                                      
  opt         maingroup -wi-ao   4.00g                                      
  swap1       maingroup -wi-ao   1.00g                                      
  swap2       maingroup -wi-ao   1.00g                                      
  tmp         maingroup -wi-ao   5.00g                                      
  usr         maingroup -wi-ao  25.00g                                      
  var         maingroup -wi-ao  15.00g                                      

I've got 108.26g in the physical volumes associated with the new SSD. I'm going to use pvmove to migrate some of these LVs to the SSD. The question is, which LVs to move?

The machine in question is basically a home workstation. I do some light development (source code lives in home), run some very low-load server processes (apache, etc.), and do a bit of image and video editing from time to time. I run Gentoo on x86 if that makes a difference.

Answer 1

For my desktop, I moved the files that I used frequently on the SSD device, leaving other files on hard disks.

• Having the system on the SSD helps a lot. It's not modified very often. Move usr on your SSD.
• Your home directory is also used quite often. Move home on your SSD. If it's too big, try to isolate files used less frequently and leave them on the hard disk (symlinks helps in this task)
• The /var directory is often accessed by daemons (appending to log files in most cases). Some people try to log on a remote server or on a ramdisk. It can be complex and may not be worth the trouble. I moved var on the SSD.
• The use of the /tmp directory depends on the applications you are using.
• The swap usage also depends on your applications and physical memory. For me, swap is not used very often so it really does not bless to have it on the SSD (it's best for swap performance).

For the partitions you may not be sure (tmp, swap1, swap2, opt), you can try without moving them and use the iostat -p command to see how often they are accessed.

Check Installing an SSD device on Ubuntu

Answer 2

Based on what I've read, LVM may not be the best choice with SSD since currently there's currently no TRIM support if you use LVM:

http://www.fedoraforum.org/forum/showthread.php?t=256068

(there are various other suggestions re: SSDs in that thread as well)

Answer 3

why not simply use the SSD as a cache?

• bache
• flashcache

Answer 4

I'm looking into something like this myself at the moment. In addition to the bcache and flashcache options that Javier mentioned, you can instead identify "hot" extents and pvmove them to your SSD:

https://bbs.archlinux.org/viewtopic.php?id=113529

To mitigate the lack of TRIM you could use less than the full SSD capacity, then later move extents and manually discard sector ranges with hdparm:

# TRIM 1000 sectors starting at sector #1
hdparm --trim-sector-ranges 1:1000 /dev/sdb

That is obviously extremely dangerous and any error could trash your data!

Answer 5

I agree with using the SSD for some caching operations, but you should probably critically examine the exact use case. If you didn't buy a high-end SSD, wear and reliability are a bigger issue. In this case, I wouldn't use it for temp directory ops, like /tmp, /var/tmp, /var/run, and swap. I would venture to use memory based file systems for this, but it's a bit harder to set up, and can be a little risky if you don't know what you're doing.

Definitely have your A/V editing on the SSD! This is probably your home directory, but might be a special directory elsewhere, even a mount point under your home. The SSDs shine where you can have random read/write without having to move disk heads. This screams editing, and high-bandwidth applications like A/V work. /usr might be the next place, if you have enough space for it. Most of your binaries and libraries are in /usr, and can benefit from the random read order the SSD will provide.