I'm looking for virtualisation solutions for storage and OS for a home server. A sort of private cloud where I manage the storage space independently of the VM one.
This question focus on storage management. (I have another question related to the VM/compute instance management).
Here my environement and wishes.
- Server: HP Proliant MicroServer with 8 GB RAM (AMD Turion dual core with AMD-V technology) with 1 250GB system disk and up to 4 HDD (2 TB) for "data"
- OS types: only Linux (perhaps a *BSD VM in the future)
- Linux distributions do not matter, I'm familiar with RHEL, Fedora, Suse, Ubuntu, but any other recommandation will be fine
- The 4 HDD is going to be a software RAID array, probably RAID 5.
- storage should be "virtualised/cloudified":
- easy to extend: if I add a NAS on the network, I can include the NAS space capacity within this storage space as one virtual disk. This can be a NAS, an external HDD or another server.
- cluster FS or S3 style space or OpenStack block storage? Whatever is easier to manage/maintain and easy to integrate/plug to VM/compute instance.
I would prefer free (libre, as in a free speach) and open source tools. But it does not have to be free as in a free beer.
Note: the VMs I intend to run on top of this server are one dedicated to backup, one for a "owncloud/dropbox"-like service and perhaps one for media server (hosting video and photos). I'm not sure if traditional VMs or compute instance are the most suitable for this.
EDIT 1, re-focusing the question
As understood and after some own research, storage virtualisation and storage hypervisor are rather new technologies with few providers of software stack appart for Entreprise-grade solutions, which won't fit my budget and needs for 1-man personal time availability.
So I will go for more down to earth solutions, but here is the new question: what storage technologies (device mapper, logical volume manager, cluster FS, iSCSI, etc.) to choose for a home server with 4 disk bays, than could be expended with a NAS or another such server in the future?
- Software RAID using MD + LVM + iSCSI? (not sure if that is even feasible)
- Software RAID using MD + iSCSI?
- Software RAID using FS supporting it such as ZFS or BTRFS?
- GFS2 or OCFS2 cluster filesystem? On top of MD RAID?
- etc. (you suggest/advise)
EDIT 2, thin provisioning
It seems that my answer would revolved around MD and LVM2 as core technologies. Both supports (experimental) thin provisionning so I could use this feature once it is stable.
I would need to be advised for good user space tools to monitor such technologies and perhaps if there are some tools (web based) to configure remotely the LVM (other than ssh).
Is the lvm "virtualsize" part of the thin provisioning experimental feature, or is it something I could start with while the other technology mature?
Note 1: I can set-up the MD and LVM volumes in the command line, but it would be cool that "maintenance" could be done via GUIs remotely.
Note 2: The storage would be used for backup, so I cannot use experimental or even just stabilised technologies (like btrfs), I want proven solutions. Though, if I can upgrade to this newer technologies when they are mature, that would be nice!
EDIT 3, Linux LIO
I know nothing really about iSCSI or Linux LIO. I've seen that Synology, QNAP and other NAS vendors have implemented the latter in their NAS solutions. How can this help my goal? Is there some administration tools that take advantage of frameworks/libraries like libStorageManagement or Targetd? Is there other alternatives?