Search
moon
sun

Closing the GitHub Actions Monitoring Gap: Observing ARC Runners with an Exporter

URL
생성 일시
2026/11/17 11:01
최종 편집 일시
2026/11/17 11:01
태그
당근
파일과 미디어
Hi, I’m Uno from the Delivery part of Karrot’s SRE team. Our team’s mission is to run Kontrol, our internal deployment platform on Kubernetes, and manage the company-wide CI system that mostly relies on GitHub Actions. In this post I’ll introduce the exporter we built to see which Jobs are waiting and which Runner is running each Job. I’ll also share how we improved it in production. A Workflow defines what happens when an event such as a push or pull request occurs. Each Workflow is split into Jobs such as test and build that run on execution environments called Runners. If no Runner is free, a Job waits until one becomes available. name: Simple # 1. Workflow: defines the automation on: workflow_dispatch: jobs: run_shell_command: # 2. Job: the unit that gets a Runner and runs name: Actions are amazing runs-on: arc-test-runner steps: # 3. Step: the commands the Job executes - name: Say hello run: | echo "Hello!"Runners can be GitHub-hosted or self-hosted. At Karrot we mainly use self-hosted Runners on our own Kubernetes clusters. These are managed by a Kubernetes operator called ARC (Actions Runner Controller). ARC organizes Runners into groups called Runner Scale Sets and scales each set according to Job demand. Each set defines the Runner image and resources, along with minimum and maximum Runner counts. A Workflow can specify a Runner Scale Set in runs-on so its Jobs are assigned to Runners in that set. Source: https://docs.github.com/en/actions/concepts/runners/actions-runner-controllerEven with autoscaling, Jobs don’t always start immediately. A GitHub outage or a spike in Job requests can leave them stuck in the Queued state. While operating ARC, we often heard developers say, “My Action is stuck in Queued and won’t run.” Answering that question wasn’t easy with the data we had. We often had to wait for an announcement on GitHub’s status page or infer th