How to Implement AWS EFS to Share File System between EC2 Instances: A Comprehensive Guide

Hey there! Setting up shared file storage for EC2 instances is a common need, and that‘s where EFS comes in handy. In this in-depth guide as an AWS expert, I‘ll show you how to create an EFS file system and mount it on multiple EC2 instances.

Why Shared File Storage and EFS

First, let‘s understand why shared file storage is important. When you have multiple EC2 instances like web servers or application hosts, often they need access to common files like web assets, media files, log aggregates etc.

Of course, you can spin up a separate file server with NFS or SMB sharing enabled. But now you have to maintain that infrastructure. EFS makes this dead simple by providing a fully managed elastic NFS file system.

Here are some key advantages of using EFS instead of managing your own NAS/file server:

• Fully managed service – AWS handles all the undifferentiated heavy lifting of maintaining the distributed file infrastructure.
• High availability – Data is stored redundantly across multiple AZs.
• Auto scaling – You can grow or shrink the storage capacity on demand.
• Multi-AZ access – Accessible from 100s of instances across AZs.
• Scalable performance – Scales throughput automatically as request rate increases.

As per AWS docs, EFS storage capacity can scale all the way from gigabytes to petabytes, while supporting thousands of concurrent NFS connections.

So in a nutshell, EFS provides a resilient, scalable, sharable file system without you having to manage servers or NAS appliances.

Prerequisites

Before we get hands-on with creating and mounting an EFS system, make sure you have:

• An AWS account with proper IAM permissions to manage EFS resources.
• Existing EC2 instances that will need access to the shared file storage. Recommended to have instances across 2 or more AZs for HA.
• Ability to connect and administer your EC2 instances (e.g. SSH access).
• An existing VPC and subnets to deploy the EFS system.

Ideally your instances should be in the same VPC/subnets as EFS for fast in-network connectivity.

Okay, ready? Let‘s dig in and see this in action.

Step 1 – Creating the EFS File System

Log into your AWS console and navigate to the EFS service. Click on "Create file system" to launch the creation wizard.

Select VPC and subnets: Choose the VPC and subnets you want EFS deployed in. As a best practice, spread it across multiple subnets in different AZs.

EFS creation wizard

Choose performance mode: EFS provides two performance modes depending on your usage patterns:

• General purpose (default) – Recommended option for most use cases. Auto scales throughput from 50 MB/s up to the 100s of GB/s range based on usage. Can burst up to 100,000s of IOPS. Latency around 1-2ms.
• Max I/O – Optimized for massively parallel workloads accessing very large datasets with millions of IOPS. But has higher latency around 10-30ms. Also far more expensive.

Unless you need IOPS in the millions or 100s of TBs of storage, general purpose works great.

EFS performance mode selection

Review settings and create: Optionally tag the file system, enable encryption etc. Then review all the configuration options selected and click "Create file system" to deploy it.

Within a few minutes your EFS file system should be ready for use!

Now let‘s look at mounting it.

Step 2 – Install NFSv4 Client on EC2

Our EFS system uses the NFSv4 protocol to expose the file shares. So we need to install the NFSv4 client utilities on each of the EC2 instances that will access EFS:

# Ubuntu/Debian
sudo apt update
sudo apt install -y nfs-common 

# RHEL/CentOS  
sudo yum update
sudo yum install -y nfs-utils 

This installs mount, nfsstat and other tools required to mount and manage NFS file systems.

Step 3 – Mount EFS on EC2 Instances

With the NFS client available, we can go ahead and mount EFS.

1. Get DNS name: In the EFS console, access your file system details and copy the DNS name from the "Attach" instructions. This will be used for mounting on EC2.

2. Create mount point: On the EC2 instance, create a directory that will be used as the mount target:

sudo mkdir /efs-mount-point

3. Run mount command: Use the mount command to mount the EFS file system onto the EC2 instance:

sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-abcd1234.efs.us-west-2.amazonaws.com:/ /efs-mount-point

Replace the DNS name with your EFS system‘s address. You can tune the NFS options like timeout and retries as needed.

4. Validate: Check that the EFS file system is now mounted on the EC2 instance:

df -h

You should see the EFS volume mounted on /efs-mount-point.

5. Make persistent: To persist the EFS mount after reboot, add an entry in /etc/fstab.

Awesome! Now any files written to /efs-mount-point will be stored on EFS rather than the EC2 local disk.

Repeat this process on all other EC2 instances that need access to the shared file storage. They will all connect to the same EFS system.

Step 4 – Test File Access

As a simple test, we can create a file on one EC2 instance and verify access from the other instances as well.

On EC2 instance 1:

echo "This is a test" > /efs-mount-point/test.txt

Now from any other instance, check that the file is visible:

cat /efs-mount-point/test.txt

You should be able to access the test file from all other servers. This confirms that everything is configured correctly.

Now you can begin using this shared EFS file system to store application data, media assets, logs etc as needed.

Scaling EFS Performance

One of the key benefits of EFS is the ability to scale throughput and storage capacity on demand. Here is how performance scales:

	General Purpose	Max I/O
Use cases	General file sharing	Big data, media processing
Throughput scale	Up to GBs/sec	Up to 10s of GBs/sec
IOPS scale	10s of Ks	Millions
Latency	1-2 ms	10-20 ms

For General Purpose:

• Throughput scales based on usage from as low as 50 MB/s up into the multiple GBs/sec range.
• Latency is very low, around 1-2 ms.
• Can burst to 100,000s of IOPS for short periods.

So for normal use cases, it will automatically meet your performance needs.

For Max I/O:

• Massive throughput scaling – 10s of GBs/sec
• Millions of IOPS – but higher latency around 10-20ms
• Ideal for large scale parallel access across AZs

You can monitor EFS metrics in CloudWatch to spot any bottlenecks. Metrics like throughput, IOPS, latency will help identify issues.

Beyond this auto-scaling, here are some tips:

• Distribute workloads by mounting on instances across AZs
• Ensure fast networking between VPC and EFS
• Use Provisioned IOPS EBS for best performance
• Parallelize reads/writes if processing large files

By following these best practices, your EFS shared file system can easily scale to meet your workload needs!

Additional EFS Capabilities

Beyond the basics, EFS provides several other useful capabilities:

Encryption: You can enable encryption at rest for EFS file systems using AWS KMS. A safeguard for security compliance.

Lifecycle management: Policies can be set up to transition inactive data to lower cost Infrequent Access storage class. Helps reduce costs.

Hybrid access: EFS can be accessed from on-premises servers over Direct Connect for hybrid environments.

Backup: Integrates natively with AWS Backup service for periodic snapshots.

Sharing across accounts: Via Resource Access Manager, EFS can be securely shared across AWS accounts.

ECS/EKS: EFS volumes can provide shared storage for containers on ECS/EKS.

As you can see, EFS provides a full spectrum of file storage features to build upon.

EFS vs EBS

Compared to EBS, EFS offers shared file storage while EBS offers block storage for individual instances. Some key differences:

	EBS	EFS
Access	Single EC2 instance	Multiple instances & servers
Use cases	Boot volumes, instance-specific storage, databases	Shared storage for files, logs, data
Management	Unmanaged, administer storage on each instance	Fully managed service
Scalability	Manually resize volumes	Auto-scaling capacity & throughput
Availability	Single AZ	Stored across multiple AZs redundantly

Generally:

• EBS for instance-attached block storage
• EFS for shareable file storage across a fleet

So in summary, once you begin having multiple instances that need to access common files or data, EFS becomes a convenient managed service for that need.

Wrap Up

That covers a comprehensive walkthrough of how to create an EFS file system and make it accessible to your EC2 instances.

The key takeaways:

• EFS provides a simple, scalable elastic NFS file system
• Super handy for sharing files, data, logs across EC2 fleet
• Fully managed, highly available and durable
• Near infinite scale for storage capacity and throughput

We went through steps like:

• Creating EFS filesystem via AWS console
• Mounting EFS on instances using NFSv4
• Validation, security, optimizations
• Scaling, encryption, backup capabilities

With this knowledge, you should be able to easily get started with EFS and eliminate the need to run your own shared file server! Let me know if any part needs more explanation. Happy to help you use EFS like a pro!