I am thinking about how to best setup a couple of environments on AWS. I would like to setup two servers:
- Application server (running web application and REST services)
- Tools server (running several ETL jobs)
The end result should be two application servers and two tools servers (i.e. one appserver/toolserver on each availability zone)
- All environments will be talking to the same database
- Each environment will have its own EBS storage
- A single external shared storage (EFS) which will be accessible from any of the 4 environments.
I am a little confused in terms of how to setup the storage. The objective is to ensure that if an EC2 instance is lost, i don't loose any data. To achieve this, i would like to set it up so that code is deployed on the local storage (EBS) and dynamic content is stored on the external shared storage (EFS).
All appliations will be in a folder with the following structure
├─ application ├─ bin ├─ lib ├─ config ├─ logs ├─ data ├─ processed ├─ failed
In an EC2 instance dies, i would like to retain config, logs and data folder. The other folders (bin and lib) will contain only binary code which can be rebuilt from GIT. To achieve this, i am thinking of the following storage
EBS on appserver 1 and appserver 2
├─ applications ├─ finance ├─ appname1 ├─ bin ├─ lib ├─ config --> /storage/finance/config ├─ logs --> /storage/finance/logs ├─ data --> /storage/finance/date
The last three will be symbolik links to folders on the external storage.
EFS (Shared storage)
├─ storage ├─ finance ├─ appname1 ├─ config ├─ logs ├─ data
The above structure can be a maintenance nightmare but i cant think of other alternatives. With the above approach, code is deployed on each appserver but data is shared between two appservers. If one appserver on one AV zone dies, the other will still be functional.
Other options i have considered include having everything on EBS storage and rely on backups/images. This will not work because the configuration files are dynamic and the image might not always have the latest data.
Reading about EFS storage, it looks like writing to it might be slow which is something that is slightly worrying me. Questions:
- Are there alternatives to the above?
- Are there any other disadvantages to the above approach?
- Is there a way i can have everything (code + data) on local EBS storage and still be able to recover (to the second) in case of a disaster.
Thanks in advance