Introduction: This is the first of a 2-part series on Hosting your own GitHub Runners on Kubernetes. This article delves into the benefits and why you want to do this. GitHub Actions offers an integrated CI/CD solution that aids seamless workflow development and maintenance. It's easy to set up and manage, and works with the same platform where your code is stored, increasing the likelihood that developers will incorporate CI/CD building and testing into their pipeline.
However, as projects grow, the native GitHub hosting may introduce limitations in scalability and cost-effectiveness. A robust alternative? Self-hosted runners. They use the same workflows as GitHub Actions, but you are entirely in control of the configuration. This gives you all the power of GitHub Actions with the benefits of your own infrastructure.
At Trunk, we've adopted this strategy for scalable pipelines, ensuring we use resources efficiently.
Why Self-hosted Runners are Vital for Scaling Projects
There are three primary rationales for migrating to self-hosted runners:
1. Cost Efficiency:
GitHub’s hosted runners may appear cheap for small-scale projects. However, as your demand scales and you start to parallelize your runs, your costs rise, hitting thousands of minutes per month.
For instance, GitHub's pricing for an 8-core, 32 GB machine is $1.92/hr. If you were to provision the same machine on AWS, it would cost merely $0.1468/hr. That is a 13X markup you will pay for every minute by hosting on GitHub. If we just look at the Linux box costs, here's how they stack up against AWS:
And that's just for Linux. Windows and Macs cost more on GitHub. You can spend $0.32 per minute on a 12-core Mac. If you need long-term access to large macOS machines for your builds or testing with GitHub Actions, it's $19/hour and $460 per day. You can get a refurbed Mac Mini for the same cost you would spend in a day.
Because you can bring your own machine (whether on-prem, a VM, a container (as we use), or even your local machine), you can choose more powerful hardware or optimize resource allocation based on your project's needs. This can lead to faster build and test times, especially for resource-intensive tasks.
Self-hosted runners also allow you to fully customize the environment, including the underlying hardware, software, and network configurations. This can be particularly useful if you have specific dependencies or configurations that are unavailable or unsupported in the default GitHub-hosted runners.
GPUs: GitHub's default infrastructure is CPU-only. Self-hosted runners allow for GPU-intensive jobs, like machine learning pipelines.
CPU-to-memory ratios: GitHub's larger runners use a N-cores, 4xN GB model. Self-hosting lets you customize, like having 8 cores with 128 GB memory.
Disk usage: GitHub's offerings can be expensive for jobs needing significant storage but minimal memory. Self-hosted runners allow tailored disk-to-compute ratios.
3. Enhanced Caching:
Caching environments and dependencies for builds and tests can help with throughput. With GitHub-hosted runners, the Actions cache is slow. As a software-level caching mechanism, the cache is stored elsewhere on GitHub servers, so you'll always have long response times.
Self-hosting on your own infrastructure allows you to use a local hardware cache that will be much more performant. You can persist a warm cache so there is no need to download assets repeatedly onto a blank slate runner, and these assets are available on the machine in milliseconds.
At Trunk, using self-hosted runners allows us to customize our CI infrastructure better, making our tests and builds faster and for less by running directly on GitHub. But the key to good performance is the right infrastructure at the right time. We'll cover how to host your GitHub Runners on Kubernetes in our next article. For now, if you want to see how much time you spend on your CI workflows, check out CI Analytics.
Part 2: Self Hosting Github Runners is now live.