Introduction
We moved an application into ECS from EC2 and were so pleased with the results that we decided to share them here. In this article we describe some of the client requests and our path to deliver along with the results so others can learn some of the things we did along the way.

Request:

Castlerock was engaged by a client to modernize an outdated web application. The application was based on a php 5.4 runtime and hosted on Ec2 using an Amazon Linux 1 based image. Several goals were identified for this project:

• Modernize the base operating system and simplify maintaining a current base operating system
• Provide a platform with a modern php runtime environment, and simplify future maintenance of the php runtime
• Deliver a simple build and deploy structure that plugs into an existing deployment management tool managed by another team
• Deliver the application using docker containers within AWS
• Begin to unbuild monolithic applications so they can break out into smaller chunks within docker containers

Note: Actual PHP code changes were handled by the clients software development team, not Castlerock

Challenges:

Our first challenge was to deliver a base image built from scratch with Docker to layer the additional components. We started with debootstrap, a Debian installation tool and AWS CodeBuild to create a base image. This completed the goal of developing a maintainable and upgradeable operating system image: the base image is versioned and upstream builds can easily select which base image version is used in the build, deploy, and quality assurance (qa) lifecycle.

The client in question had a very complex build and deploy based on different Git branches for different pre-production and production environments. Building a new version of the php application required a separate build in every environment. This led to over a dozen separate builds during each release cycle with the potential for more as the product grows. We set out to deliver a simple build and deploy by identifying environment specifics and moving them into Secrets Manager and the SSM Parameter Store. We then made sure that these specifics were integrated at container runtime in the task definition. This permits a single build that can be deployed to any runtime environment.

As happens with a lot of products, over the years, the client’s application environments’ nomenclature fell out of parity with each other. This resulted in different AWS accounts and runtime environments having their own peculiarities; these needed to be accounted for as fixing naming conventions was out of scope for this current project.  Keeping scope in mind, we had to design our Infrastructure as Code (IaC) templates, built in CloudFormation, to use complex IF statements in order to account for these peculiarities. 

What skills/tools we leaned on:

We learned some new things along the way. For example, in Cloudformation there is an extension for ForEach loops that will loop through a list and perform some action. In our case, we designed the template to help with future additional ECR repositories.  We passed a list of repository names into our CloudFormation template. This allowed us to repeat the same 30 lines of code for each repository in the list. Now, when we need to add a new repository, all we have to do is add 1 name to a parameter list. We can also, for example, pass a list of ports into common security groups rules.

After the CloudFormation infrastructure was in place, we moved on to the next big goal of building and deploying the application. To accomplish this objective, a pair of bash scripts based on the AWS CLI V2 were developed. These can run on any computer with the AWS CLI V2 and proper credentials in AWS.  The build script allows the release manager to securely run CodeBuild and create a new version of the image. This image is stored in a separate account from all of the application accounts. Doing this allows for a single build per release cycle. After the image has been built, we looked at the deployment script. This script takes the new image from the build stated above and deploys it in the individual environment.  These scripts are a temporary fix until the release process is added to an internal release automation tool the client uses. 

Results:

We had some pleasantly surprising results in application performance improvements

There were three components to this migration:

• Moving EC2 instances to ECS containers while maintaining consistent CPU and Memory values
• Replacing the base operating system Amazon Linux Version 1 with a Debian 12 based docker image
• Upgrading PHP 5.4 to PHP version 8.1

The charts below show data from Load Balancers and Target Groups handling production traffic. The old EC2 instances are split between the Orange and Green lines, handling different components of the application. In the ECS workload, these lines have been combined into a single Target Group represented by the Blue line.

Note: The blue response times all the way to the right is lower than the combined green and orange response times on the left.

To add to the above response time chart, on EC2 we had capped the number of requests per target to 150 and had autoscaling add more instances to maintain that level. In the chart below, we increased the number of requests per target to 600, 4x the ec2 workload, and still maintained the lower response time above.

In addition, the remaining charts illustrate the number of healthy hosts, and the CPU and Memory utilization of each. The ECS workload is overall more stable than the EC2 workload, where we saw a lot of instance failures during high traffic times. In ECS, containers remain stable throughout those periods, due to improvements in both the PHP runtime and the underlying operating system.

Showing that CPU and memory are both well below the ability of the tasks.

This migration demonstrates that the old adage “better, faster, cheaper; pick two” does not always apply. Here, by changing to ECS, a modern base OS, and a modern runtime environment, we have delivered on all three, reduced use of resources, lowered cost, and improved performance. It is hard to pinpoint exactly which component provided the most gain since we changed everything at once.  

Conclusion

There are many modern, build and deploy patterns available to application builders today. Some prove to be a flash in the pan, and some will withstand the test of time. This article addresses one such pattern: our team stays abreast of the current delivery mechanisms, watching for the short-lived shiny new things and delivering on what works, now and for the future for our customers.

Would you like a modernization review of your systems and architecture?