Questions about storage virtualization

Question

I am trying to understand and compare storage virtualization methods, including RAID and LVM. I hope I could get some general idea and big picture for the relation between various concepts.

1. I was wondering if various storage virtualization methods can be classified into virtualization at the device (disk), partition or filesystem levels, as following

   • RAID belongs to virtualization at the device/disk level, which replace physical disks with logical/virtual disks.
   • LVM belongs to virtualization at the partition level, which replaces partitions with logical/virtual partitionss (also called logical volumes).
   • There is also vitualization at the filesystem level, which replaces filesystems with logical/virtual filesystems, for example, Network-attached storage (NAS).

2. If my above understanding is correct, does virtualization at each level also implement virtualization at all lower levels? For example, virtualization at partition level also implements virtualization at device level, and virtualization at filesystem level also implements virtualization at both partition and device levels?

3. How do different levels of virtualization affect/determine their different areas of applications? For example, are there applications suitable for RAID but not for LVM, and for LVM but not for RAID?

4. There is a Wikipedia article for storage virtualization, where there are two main categories of methods, block virtualization (which can further be classified into storage device-based and host-based and network-based) and file virtualization.

   Compare the article with my understanding in part 1,:

   • Is it correct that storage device-based block virtualization is same as virtualization at the device level. Host-based block Virtualization is same as virtualization at the partition level. File virtualization is same as virtualization at the filesystem level.
   • But in Host-based block Virtualization#Specific_examples, it looks like Host-based block Virtualization includes virtualization at the filesystem level? How shall one understand what is File virtualization then?
5. I would rather to single out network-based from block virtualization in the aforementioned Wikipedia article, because for storage virtualization over network, I think we can also classify the various methods into the levels of device, partition and filesystem? For example, can I say Storage Area Network (SAN) belongs to the level of device, and Network-attached storage (NAS) to the level of filesystem?

Thanks and regards!

Answer 1

Might as well provide an answer. Refer to @soandos 's answer for more detailed answering of your specific questions.

LVM vs RAID

RAID, as many have mentioned, is a standard of technologies in which multiple disk drives are allocated together as an array of disks, providing varying level of performance and reliability benefits. For example, RAID 0 provides the best performance one can possibly get with the harddrives, and is extremely sensitive to disk loss (one loss = essentially total loss) whereas RAID 6 provide redundancy even when rebuilding array in a one drive loss scenario. RAID array are usually seen as one single drive to the OS.

One can say that RAID is a many to one mapping.

LVM, on the other hand, allows logical "disk drives" (block device to be accurate, but anyways) to be formed by parts of different disk drives. They exist in a "many-to-many mapping" manner. While one can use LVM to accomplish what can be accomplished by RAID, LVM is actually something that can accomplish much more. For example, to add another disk drive to a RAID array it would likely be necessary to rebuild the whole array from scratch. With LVM, it is just adding a disk drive to the machine, adding the disk drive mapping to a logical volume, and using it (the actual configuration is a little bit more complicated but certainly less than rebuilding a whole array).

Answer 2

1. RAID is a backup technology that insure that in the case of drive failure, all data remains intact, and LVM is Logical Volume Manager that can do many things.
2. It's not.
3. That seems like an odd question. RAID is a way of separating data across drives, so that if one fails no data is lost. LVM is a volume manager that can be used to change the way a user/OS looks at all the hard drives. They have nothing to do with each other (though LVM can implement RAID 1 and RAID 0, that is not its primary focus).
4. The first means you don't have to know what physical device the data is on, and the second means that you can store for lack of a better word the links between files in a more abstract way.
5. As stated above there is no "device level" or "partition level" to talk about so no, you can't refer to them as such.

Answer 3

The article is not well written.

One of the biggest problems is that there are multiple layers of abstraction built into the full stack of storage, and "virtualization" is a fuzzy enough word as to be hard to definitively assign a place to put it. For a good look at the many layers of abstraction in storage, I'll point you at a blog piece I did last year (read it here for the gory details).

In marketing-speak, "Storage Virtualization" is just introducing abstraction where there previously hasn't been any. That can happen at many points depending on the market segment. But that's just marketing. Time for technical.

The storage stack (somewhat simplified):

1. Disk
2. RAID controller
3. Software RAID
4. Volume manager
5. Filesystem
6. Network filesystem
7. Network filesystem client

Disk, even old school spinning magnetic disks, do a level of virtualization. They present a logical view of the actual blocks on the platters (or storage cells for an SSD), and this has been this way since the mid 80's or so. Magnetic drives reserve a certain number of blocks for reassigning blocks that go bad, and the logical view is how this is abstracted away from the disk controller. Technologies like SMART can catch this in the act and report that the drive is "pre-fail" so you can plan your transition accordingly. This has been in place in some form since the 80's.

RAID cards provide another abstraction layer, hiding the true shape of storage from an operating system. This has been in place since the first RAID cards came out in the late 80's, and they've only gotten more complex since then. Cards with write-caches on them provide still another abstraction layer, as writes can be reported as committed before they're actually on a disk somewhere. The really fancy ones (such as those in Storage Area Network arrays) can even write to two separate disk arrays for realtime replication, and the OS is none the wiser.

Once you get into the operating system things get a lot more murky, as each does their own thing. Software RAID (md in Linux) is typically implemented as a low level storage driver that presents the logically combined storage to higher storage layers. As with the RAID cards, you can do all sorts of interesting things here. Some of the "Storage Virtualization" products you see out there are implemented at this stage.

Going higher you get to the volume managers (LVM) can provide for some seriously complex configurations. Where the next layer down aggregates disks into a single virtual volume, the volume managers can combine multiple volumes into a single bigger volume... or split a pool of volumes into an arbitrary number of volumes. Again, some of the Storage Virtualization products you see have a presence in this layer as well.

The next step up is the filesystem. This is the layer where the well known abstractions of "file" and "directory" come into existence. Some filesystems (btrfs, zfs) have volume-manager like features built into them which allows things like snapshotting, deduplication, replication to other devices, and even migration of files between storage tiers. That last bit is not in many filesystems yet, but is definitely a target for Storage Virtualization vendors.

The next step up is the network filesystem. This is things like Samba/CIFS, NetATalk/Appletalk, NFS, and others. If written the right way, these network filesystems can further abstract storage. One product I'm thinking of, Novell's Open Enterprise Server and their ShadowVolumes, takes multiple volumes on different storage (presumably differing speeds/cost) and presents them as a single volume to the network user, and then migrates files between the volumes based on usage statistics. Some of the "Storage Virtualization" appliances you can buy actually do their heavy lifting at this layer.

The last stop on our trip up the storage stack is the network filesystem client in the client machine. It is at this level that the Distributed File System (DFS) exists, which allows a single logical presentation of a filesystem to exist on multiple network filesystems. The client knows that this is a DFS share, and that specific object is a DFS link, and when following it present the specified network-share as a sub-directory of the parent directory. There have been other examples of abstraction at this level, but DFS is perhaps the most common.

---

One thing to keep in mind is that through all of this, each layer of the storage stack is independent of those above it. Many layers are already doing block-level abstraction, so adding one more doesn't change a whole lot. File-level abstraction has to happen near the top of the stack (for that's where the file-systems live) and the impacts lower down are highly decoupled to the point that it may not even be noticed.

---

At it's core "Storage Virtualization" is still mostly a marketing term for something that has been happening since the dawn of the PC era (if not earlier), only this time the new abstraction layers are happening when virtualization is the buzzword of the moment.

The one new abstraction layer I know of is something called a "Storage Router", which you'll only ever see on large Storage Area Networks. This device has several different storage arrays behind it, and presents those separate arrays as single array with multiple LUNs. The fancier ones can do interesting block-level abstractions like moving rarely used blocks to slower/cheaper storage and moving the highly used blocks to SSD layers, or handle realtime replication between storage arrays that normally wouldn't allow that kind of thing.

P.S.: RAID is not just device-level virtualization. I'm working with a storage array right now that takes slices of disks and assigns them to different RAID groups. It is working just fine (I'm doing it right now), and I have both RAID1 and RAID5 volumes on the same disk device. Lose two drives and the RAID5 volumes are toast, but the RAID1 volumes on the same disks are just fine.