1

I intend to buy a PogoPlug for a small server. I want to install Debian or Arch on it and I will use it for sharing and backing up photos between computers and as a HTTP server or/and event storages with query databases (aka. CQRS with Event Sourcing). The photos require about 15GB, the event storage part require at most 10GB of storage space. The Linux and the applications will be on an SD card, while the stored data on a SATA2 or USB3 storage device.

By my applications it does not really matter whether the left side of the HTTP app <- query database <- event storage data flow is on the computers or on the server. Syncing the query database by reconnecting to the event storage requires more code, but I can live with that if necessary. The only write intensive part which must be on the server is the event storage. The most write intensive application will be a weather sensor log, which will log the time, temperature, humidity in every second. I can buffer or preprocess that before writing the data to the disk to reduce write frequency if necessary, but I think it will be the most write intensive part no matter what I do.

There will be other applications as well, e.g. training scheduler, etc.. by which the write frequency depends on how frequent I use them, so I guess they are not an issue compared to the sensor logger.

  1. What data storage solution do you suggest by this system? I would like to use something cheap and quiet, that's why I thought of a 32GB USB flash drive or an SSD. Can these do the job, or do I require a HDD because the write intensity?
  2. Is there a big difference between an USB flash drive and an SSD about wear out time?
  3. Should I move the query databases from the server to the computers, since they increase the writing at least double fold?
Improve this question

1 Answer 1

Reset to default
1

If you are logging that type of data 1/second speed is a non-issue.

Write intensity? No your situation doesn't qualify. The weather data you suggest is less than 128 bytes per second, it is laughably low, basically none.

The cells won't wear out as much because your just using new cells, and not re-writing old cells. Writing 1 10mb photo will be more intense that your logging. When your pounding your storage with 20mb/s writing then your needs will qualify for LOW. Cheap USB sticks won't get much more than 5mb/s, but every generation is getting faster. USB3 stick capable of 20mb/s-60mb/s are easy to come by for a price, but dollar stores are selling the left over slow ones.

Depends on how cheap the USB stick is $9.99 for 64gb its going to wear out much quicker. You get what you pay for.

The SSD will always win on the speed factor and durability, there are about 6 usb stick that even come close. The SSD has much higher quality chips and most of them are rated for 40TB+ of writing before failure. (at 4k, minimum block size, per second I come up 340 years, for a SSD, to wear out. Based upon your weather logging data needs)

Assuming you don't force linux to write the data out immediately, it should auto buffer it.

Improve this answer
3
  • I can do further enhancements by writing the data only by changes. I assume humidity and temperature does not change in every second... :-) Thanks! That part is solved in my puzzle.
    – inf3rno
    Aug 14, 2015 at 4:07
  • Can you at least estimate the write durability of an average usb3 stick compared to an ssd? (I meant sandisk, adata, kingston, etc... not something very cheap.)
    – inf3rno
    Aug 14, 2015 at 4:11
  • 1
    Bundle your writes in 4k increments come as close as you can without going over. You should get at least 3-5 years. However, 5-7 is not impossible based on your useage. I would get a new USB stick every 3 years any way because there quality,size, performance and etc are going to double or more in that time.
    – cybernard
    Aug 14, 2015 at 4:34

Not the answer you're looking for? Browse other questions tagged .