Migrating a Legacy Perl app to AWS

Migrating a Legacy Perl app to AWS

Cloud computing is a growing fact of life in websites and web services.

John Napiorkowski (Jnap) is perhaps best known as a lead developer of the Catalyst web development framework, at least until recently. His breadth of experience includes cloud-based web solutions, such as Amazon Web Services, Docker, Linux, and of course Perl. (He’s also part of The Perl Shop’s extended family.)

I sat down with Jnap to talk about a project he’s been migrating from dedicated hardware to AWS.

Infrastructure as code

The client has a Catalyst-based intranet application, running on dedicated internal hardware. They had made the decision to move to Amazon Web Services (AWS), as some of their other systems were already running there. The migration actually started before Jnap came on board, beginning with an “abortive attempt,” as he describes it.

“The original developer created an instance on AWS, and just started installing stuff. As he went along, he noticed old versions of packages, and said, ‘Well I’ll just upgrade everything.’ Eventually, he got something that just sort of worked, through a lengthy manual process.”

I think this is part of what we mean when we speak of “dependency hell.” Such installations tend to be fragile and difficult to maintain. The only way to upgrade them is by working on a live production box. And good luck spinning up an identical instance for development or staging.

Jnap continues: “I recommended what I usually recommend, which is if you’re going into the cloud, the best thing to do is to have code that represents your infrastructure, not just your application.”

Change as little as possible

He started with Docker, as a way to deploy to dev or Amazon Elastic Container Service (ECS), but he ran into issues. For one, the application persists data in the filesystem. In order to address that issue, he’d have to simultaneously migrate from dedicated hardware to a cloud platform and from file storage to some other form of storage.

That violated one of his key rules regarding migrations: Change as little as possible.

“That’s why Chernobyl exploded, because they removed the safety and then ran on overload. You really have to drink the full gallon of Kool-Aid. You really can’t store state in the container, for example. You can’t store uploaded files or generated files in the filesystem. You really have to use Docker-attached storage or something.”

Indeed, many disasters have been caused by a confluence of factors, any one of which would not have caused a problem. A quote from an old TV show about engineering disasters comes to mind: “If you look at the major disasters of history, you see in retrospect a sequence of mistakes, each one of which at the time appeared to be innocent and inconsequential, which all combined in a very unfortunate way at the time of the disaster to create a disaster.” (Roger L. McCarthy. Modern Marvels: Engineering Disasters 14. November 23, 2004.)

Of course, most of the issues we face as developers are not on the scale of devastation of the Chernobyl meltdown. Even so, we do experience our own little disasters: downtime, lost weekends, angry management. And as professionals, we talk about separation of concerns and API contracts, in an effort to manage the complexity of a modern software system. And we talk about leaving working systems in a working state. “Don’t fix it if it ain’t broken,” and such.

Jnap decided to go back to Vagrant, without Docker.

“With a VM, I could mimic the environment as the application expects to run, whereas with Docker, I needed to change the application to fit the environment. I did make some changes. I moved sessioning from files to use the database, so that you could use more than one box. I moved quite a few things to the database. But we’d certainly need to do a lot more, moving generated files to a shared filesystem.”

And he used Packer to manage his VM configurations.

The true hero of the story

Packer is a tool (a member of the HashiCorp suite, which also includes Vagrant) that creates identical machine images for multiple platforms from a single source configuration. With Packer, you can use the same configuration to generate a development image that runs in a VirtualBox under Vagrant and a separate AWS image for production.

With Packer, he installed everything from top to bottom, using Git to manage versions of the build.

“It takes a week or two to slog your way through the build process. And you end up with code that configures your environment. We have a good, working Vagrant box. We’ve been using it for two months to build features.”

But that slogging… Jnap explains: “It sounds easy to say, ‘You have code that represents your infrastructure.’ But there’s some nebulousness. For example, where do you put your cpanfile?”

The cpanfile conventionally goes with the application, even though it represents machine-image configuration. Do you go with the convention and put it inside the image with the app, rather than in the configuration? Or do you put it in the meta-configuration, and separate it from the app that it describes? It’s a design tradeoff.

“What I do,” he says, “is to put the cpanfile in with the application, and then annotate it that it’s associated with certain versions of the machine image.”

And then there’s the question of upgrading a running box. Especially for development environments, do you force the user to upgrade the entire environment with a new machine image? Or do you allow them to do in-place upgrades of individual components or libraries?

“How do you version your machine images as they change? With Docker, everything’s ephemeral. But with this app, I allow for people to upgrade in place without forcing them to upgrade the full machine to a new version. Probably following a manual process. If that’s too onerous, we’ll revisit.”

Where do we go from here?

They’ve been using the distribution from the Vagrant box, and they’re happy with it. The next step is deploying to Amazon’s Elastic Compute Cloud (EC2).

“I’ve created more steps to the Packer script to create the EC2 image. There are some tools that Amazon wants you to install to turn your local image into an Amazon Machine Image (AMI). They have a service that converts your image into an AMI, and Packer can trigger that as a post-process. I need to script the Amazon setup, and I’m using CloudFormation for that.”

CloudFormation is an Amazon service that provides a common language for you to describe and provision all the infrastructure resources in your cloud environment, using a single configuration file.

“Your Perl code is not really the application. This is my approach for dealing with legacy systems. If I were doing something from scratch, I’d probably use ECS or something that avoids bare metal. People don’t know about some of the tools. They can’t figure it out. There’s a learning curve.

“But 2 or 3 years from now, all of this is going to be different. Legacy is probably going to be Docker applications. Whatever you’ve got, you have to not force it into being something that it’s not.”

If you have a legacy Perl applications that you’d like to migrate to the cloud, get in touch and we can advise or manage the process.

Tim King is Lead Developer at The Perl Shop. Tim got his start writing real-time embedded software for high-speed centrifuges the 1980’s and went on to do embedded software for Kurzweil Music Systems and Avid Technology. He has been developing for the web since the web existed, and brings discipline and skills honed from embedded systems to enterprise software. His expertise is in designing for software quality, achieved through automated code testing, test-first development, and risk managed refactoring, all through an agile process. This approach naturally lends itself to working with legacy code, such as successfully and safely refactoring a 465-line legacy function used in a video streaming application into a structurally sound design. Or designing for maintainability, through cleanly layered architectures, like a web service that can handle multiple RPC protocols using a common controller and a thin view layer, that can easily be supplemented to handle additional protocols. Tim is skilled in Perl, JavaScript, and other programming languages, in Internet protocols, in SQL, and is familiar with the internals of a variety of open source applications. Tim also writes and performs music, and has authored and published a number of inspirational books.

3 Replies to “Migrating a Legacy Perl app to AWS”

    1. From what I know of the project, I doubt there was a DarkPAN involved. It most likely involved automating the install from CPAN into a machine image.

    2. The original website author was very careful to separate custom from CPAN code. As a result we were lucky to not have to cope with locally introduced changes that never made it upstream to CPAN

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.