CARVIEW |
Navigation Menu
Do jobs in self-hosted runners run in parallel? #26769
-
The document says "Jobs run in parallel by default.Β ". When I tried workflow in github-hosted runners, jobs do run in parallel. However, when I try to run the workflow in self-hosted runners, the jobs do not seem to run in parallel. Iβve tried in both macos and centos with the latest version runner v2.165.2. Is it that jobs only run in parallel in github-hosted runners? My work flow is as follows:
And the runner output in CentOS is as follows:
|
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 12 comments · 11 replies
-
HiΒ @dahuokolmostarΒ , Jobs should run in parallel for both hosted runner and self-hosted runner. But for self-hosted runner, there are some comments:
Thanks. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 15 -
π 2 -
π 3
-
Well that's a bummer finding out after setting everything up π₯² |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 4 -
π 2
-
If you are on AWS you might be interested in RunsOn, which handles the runner autoscaling for you. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 2
-
So... if a single running can only run a single job at a time, consecutively and NOT concurrently, then "Jobs should run in parallel for both hosted runner and self-hosted runner." seems like a generally false or at best misleading statement. Each single runner will only run a single job at a time. The only concurrency which can occur is if there are multiple runners, one for each job. This is not just annoying but counter-intuitive to all the current documentation. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 39 -
π 1 -
β€οΈ 7
-
Are there any plans to change this? That is so that a runner can run multiple jobs at the same time (concurrently). It would be really nice, since a lot of the time the jobs are small and donβt require the full machine to run and could run multiple jobs at the same time. Plus most of the time the jobs are spun up in containers so just having fewer runner machines with multiple cpus to spin those up vs having multiple smaller machines or having to run multiple instances of the actions runner service per machine to achieve thisβ¦ For more cpu/resource intensive jobs you could select a runner via labels so that it would go onto a machine that is dedicated to that job? |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 4 -
β€οΈ 1 -
π 1
-
Fact is that currently, the GitLab runner is much better than the Github actions runnner. Hopefully this will change. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 41 -
π 2
-
Yes, those are years apart, I was using gitlab and it's way better... |
Beta Was this translation helpful? Give feedback.
All reactions
-
β€οΈ 10
-
yep, I used github in the past then got exposed to gitlab, and must say that their CI/CD is superb. Now my company tries to unify SCM and settled on github. Feels like a downgrade. I also don't like how github really had to design CI/CD to function like an app store. Completely unnecessary. |
Beta Was this translation helpful? Give feedback.
All reactions
-
@allir i agree with you man!!! |
Beta Was this translation helpful? Give feedback.
All reactions
-
β€οΈ 2
-
We can run Runner in docker and then we can run multiple runners on same server!!! |
Beta Was this translation helpful? Give feedback.
All reactions
-
any details on this? |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 1
-
Thanks @weide-zhou for the details. I think it makes a lot of sense to add these comments into the official documentation. |
Beta Was this translation helpful? Give feedback.
All reactions
-
if you want to have multiple runners on the same machine you can run them under different users:
|
Beta Was this translation helpful? Give feedback.
All reactions
-
π 17 -
π 4 -
π 1 -
π 2 -
π 2
-
This doc should include this information. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 1
-
The doc you linked to no longer contains this statement. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 5
-
should be possible to install the runner multiple times on the same machine with different users, no? |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 1
-
You can install multiple runners on the same machine with the same user. Just use a different folder location. |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 8
-
I've installed multiple runners in different folders, but this is quite frustrating to maintain between all the systemctl service definitions and special configs and permissions for each. I would love to see a single runner instance be able to run multiple jobs concurrently. |
Beta Was this translation helpful? Give feedback.
All reactions
-
Using ARC on a single node with microk8s might be a solution? |
Beta Was this translation helpful? Give feedback.
All reactions
-
I did try that but found the runner performance to be significantly worse than running on bare metal. Might be just me though... |
Beta Was this translation helpful? Give feedback.
All reactions
-
here is a script to automate adding runners on a single server #!/bin/bash
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <runner-number> <github-token>"
exit 1
fi
RUNNER_NUMBER=$1
TOKEN=$2
RUNNER_DIR="/home/$USER/actions-runner-${RUNNER_NUMBER}"
SERVICE_FILE="/tmp/github-actions-${RUNNER_NUMBER}.service"
# Remove existing directory if exists
rm -rf "$RUNNER_DIR"
# Create fresh runner directory
mkdir -p "$RUNNER_DIR"
cd "$RUNNER_DIR" || exit
# Download and extract runner
curl -o actions-runner-linux-x64-2.321.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.321.0.tar.gz
# Configure runner
./config.sh --url https://github.com/REPLACE_GH_USERNAME/REPLACE_GH_REPO \
--token "$TOKEN" \
--name "runner-${RUNNER_NUMBER}" \
--unattended \
--replace
# Create service file in temp location
cat > "$SERVICE_FILE" << EOL
[Unit]
Description=GitHub Actions Runner ${RUNNER_NUMBER}
After=network.target
[Service]
Type=simple
User=${USER}
Group=${USER}
WorkingDirectory=${RUNNER_DIR}
ExecStart=${RUNNER_DIR}/run.sh
Restart=always
RestartSec=10
KillSignal=SIGTERM
[Install]
WantedBy=multi-user.target
EOL
# Move service file and configure systemd (these need sudo)
sudo mv "$SERVICE_FILE" "/etc/systemd/system/"
sudo systemctl daemon-reload
sudo systemctl enable "github-actions-${RUNNER_NUMBER}"
sudo systemctl start "github-actions-${RUNNER_NUMBER}"
echo "Runner ${RUNNER_NUMBER} has been created and started" run the script as ./add-runner 1 TOKEN_1
./add-runner 2 TOKEN_2
./add-runner 3 TOKEN_3 |
Beta Was this translation helpful? Give feedback.
All reactions
-
π 5 -
β€οΈ 6
-
Appreciated! |
Beta Was this translation helpful? Give feedback.
All reactions
-
+1 for more documentation on parallel workflows. I set up two Actions Runners under NixOS: Nix config services.gitea-actions-runner.instances = let
labels = [
# From NixOS example config
# provide a debian base with nodejs for actions
"debian-latest:docker://node:20-bookworm"
"debian-bookworm:docker://node:20-bookworm"
"debian-bullseye:docker://node:20-bullseye"
# From Gitea example config
# (see https://gitea.com/gitea/runner-images)
"ubuntu-latest:docker://gitea/runner-images:ubuntu-latest"
"ubuntu-22.04:docker://gitea/runner-images:ubuntu-22.04"
"ubuntu-20.04:docker://gitea/runner-images:ubuntu-20.04"
];
runners = lib.lists.range 0 1;
in
builtins.listToAttrs (map (i: {
name = "runner" + toString i;
value = {
enable = true;
name = "ACME CI amd64 (" + toString i + ")";
url = "https://10.5.0.11:3000/";
tokenFile = config.age.secrets.gitea_actions_runner_env.path;
settings = {
capacity = 4;
cache = {
enabled = true;
host = "10.5.0.1";
port = 3088;
};
};
labels = labels;
};
})
runners); On a single branch they did not become active concurrently. On different branches they did, but only one came up: the workflow includes a MinIO server (Docker image) binding to 0.0.0.0:9000 by default and a collision happens β it seems, the workflows are not fully isolated, even though container:
# Specifies the network to which the container will connect.
# Could be host, bridge or the name of a custom network.
# If it's empty, act_runner will create a network automatically.
network: "" (https://gitea.com/gitea/act_runner/src/branch/main/internal/pkg/config/config.example.yaml#L63) I expected each runner to have its own dynamic network by default. π€ Further, it was not clear how the cache is being handled in this case. It seemed to work, but each runner used its own cache. Not critical for my setup so far, all in all in works pretty great already with a single runner! π |
Beta Was this translation helpful? Give feedback.
HiΒ @dahuokolmostarΒ ,
Jobs should run in parallel for both hosted runner and self-hosted runner.
But for self-hosted runner, there are some comments:
Thanks.