0

I am thinking about how to best setup a couple of environments on AWS. I would like to setup two servers:

  • Application server (running web application and REST services)
  • Tools server (running several ETL jobs)

The end result should be two application servers and two tools servers (i.e. one appserver/toolserver on each availability zone)

  • All environments will be talking to the same database
  • Each environment will have its own EBS storage
  • A single external shared storage (EFS) which will be accessible from any of the 4 environments.

I am a little confused in terms of how to setup the storage. The objective is to ensure that if an EC2 instance is lost, i don't loose any data. To achieve this, i would like to set it up so that code is deployed on the local storage (EBS) and dynamic content is stored on the external shared storage (EFS).

All appliations will be in a folder with the following structure

├─ application
    ├─ bin
    ├─ lib
    ├─ config
    ├─ logs
    ├─ data
        ├─ processed
        ├─ failed   

In an EC2 instance dies, i would like to retain config, logs and data folder. The other folders (bin and lib) will contain only binary code which can be rebuilt from GIT. To achieve this, i am thinking of the following storage

EBS on appserver 1 and appserver 2

├─ applications
    ├─ finance
        ├─ appname1
            ├─ bin
            ├─ lib
            ├─ config --> /storage/finance/config
            ├─ logs --> /storage/finance/logs
            ├─ data --> /storage/finance/date

The last three will be symbolik links to folders on the external storage.

EFS (Shared storage)

├─ storage
    ├─ finance
        ├─ appname1
            ├─ config
            ├─ logs
            ├─ data

The above structure can be a maintenance nightmare but i cant think of other alternatives. With the above approach, code is deployed on each appserver but data is shared between two appservers. If one appserver on one AV zone dies, the other will still be functional.

Other options i have considered include having everything on EBS storage and rely on backups/images. This will not work because the configuration files are dynamic and the image might not always have the latest data.

Reading about EFS storage, it looks like writing to it might be slow which is something that is slightly worrying me. Questions:

  • Are there alternatives to the above?
  • Are there any other disadvantages to the above approach?
  • Is there a way i can have everything (code + data) on local EBS storage and still be able to recover (to the second) in case of a disaster.

Thanks in advance

  • Have you reviewed any Amazon documentation on high availability? If so, please reference which ones and narrow your question down to a single specific question. There is no way we can address all the possible scenarios that could exist in your specific environment and Amazon has already provided you the platform to deliver highly available services. For instance, have you considered that EBS storage is already highly available to the AZ? I'm not sure how this question can be answered succinctly as it stands. – Appleoddity Oct 22 '17 at 14:01
  • What is your RPO for the data on the EBS disks? Basically, how much data are you prepared to lose - eg none, 1 hour, 24 hours, etc. That will probably drive the solution. – Tim Oct 22 '17 at 21:11
0

In looking at the AWS pages for EFS, it mentions that "Each Amazon EFS file system object (i.e. directory, file, and link) is redundantly stored across multiple Availability Zones." Similar wording on the EBS page. So there's some redundancy there. Here's an Amazon PDF on high availability.

For EC2, using multiple availability zones will lower your risk exposure. I'd put each instance in a separate availability zone. And the PDF recommends using snapshots of EBS with a standby EC2 instance for HA. Snapshots are available within regions so they are easily accessed for disaster recovery. Doing it that way would answer your question about putting everything into AWS EBS. And it's fairly simple to copy EBS volumes between regions to add redundancy and keep everything sync'd. You could, with that, have a production environment, a standby environment and a replacement environment that could be spun up quickly in the event of a failure, leaving you with 2 environments available at almost all times.

Check out that PDF and link plus Amazon's docs on high availability and see what works best for your business needs and staff levels.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.