GitHub Actions grew to become my software of alternative for automating duties round software program improvement. To execute jobs, GitHub Actions depends on runners. By default, jobs run on GitHub-hosted runners. However there are good causes to make use of self-hosted runners.
- Lowering prices by using your cloud or on-premises infrastructure.
- Accessing personal networks (e.g., RDS linked to a VPC).
- Customizing the setting by pre-installing libraries or instruments.
Within the following, I’ll share three approaches to self-host GitHub runners on AWS and focus on their professionals and cons.
Internet hosting GitHub runners on EC2 situations
The best approach to host a GitHub runner on AWS is that this.
- Launch an EC2 occasion.
- Set up the runtime setting and instruments required for jobs.
- Set up and configure the GitHub runner.
The GitHub documentation describes how you can add self-hosted runners intimately.
The strategy comes with two downsides. First, the answer doesn’t scale. Throughout peeks, jobs pile up and decelerate software program improvement. Second, the idea just isn’t safe. When tasks or groups share a digital machine to execute their jobs, there’s a excessive danger of leaking delicate data (e.g., AWS credentials).
Scaling GitHub runners with auto-scaling
To keep away from prolonged ready instances, scaling the variety of EC2 situations operating GitHub runners with the magic of auto-scaling is an apparent thought.
- An Auto Scaling Group launches and terminates EC2 situations based mostly on an AMI with GitHub runner, the runtime setting, and instruments pre-installed.
- A CloudWatch alarm will increase or decreases the specified capability of the Auto Scaling Group based mostly on a metric just like the job queue size.
A facet notice: it isn’t trivial to make sure the Auto Scaling Group doesn’t terminate an EC2 occasion that executes a long-running job (see lifecycle hooks). Additionally, discovering the precise metric to scale is difficult.
Whereas this strategy addresses the scaling problem, it nonetheless has a significant draw back: jobs from totally different tasks or occasion groups share the identical digital machine. There’s a excessive danger of leaking delicate data (e.g., AWS credentials).
Occasion-driven EC2 situations for GitHub runners
Right here comes a easy strategy that addresses each challenges: constructing a safe and scalable infrastructure for GitHub runners by executing every job by itself EC2 occasion.
- The GitHub webhook sends occasions indicating {that a} job was queued and is ready for a runner.
- The API Gateway receives an occasion and invokes a Lambda perform.
- The Lambda perform launches an EC2 occasion and fingers over a just-in-time runner registration by way of person knowledge.
- The EC2 occasion begins the GitHub runner.
- After the GitHub runner exits, the EC2 occasion terminates itself.
This answer does offload the problem of scaling an infrastructure to the on-demand capability supplied by AWS. And by the way in which, the strategy may be very cost-efficient as you aren’t paying for idle assets.
Apart from that, as every job runs on its digital machine, which ensures excessive isolation and implements the Safety hardening for GitHub Actions finest follow of utilizing just-in-time runners.
There is just one small catch: beginning a brand new EC2 occasion for every job provides a delay of ~1 minute for each job. For my part, a delay of 1 minute per job is price the advantages when it comes to scalability, price, and security.
I’m glad to announce that we simply launched HyperEnv for GitHub Actions: Self-hosted GitHub Runners on AWS. This product implements the event-driven answer described above. HyperEnv for GitHub Actions is out there on the AWS Market.
