Setup Docker containers as Build Slaves for Containerized Jenkins Master
In a distributed Jenkins environment, resource utilization of the slaves will be very less when the builds are not quite often. In this scenario, it is better to use ephemeral docker containers as your build slaves for better resource utilization. As you know, spinning up a new container takes less than a minute, every build spins up a new container, builds the project and will get destroyed. This way you can reduce the number of static Jenkins build VMs.
Prerequisites
- A Jenkins master server up and running.
If you want docker based Jenkins setup, you can follow this tutorial -> https://medium.com/@rakeshjain_17559/containerizing-jenkins-with-docker-on-ubuntu-ce766a59c779 - A Linux VM with docker installed.
Refer below tutorial for Docker installation on Ubuntu https://medium.com/@rakeshjain_17559/docker-installation-on-ubuntu-d04206626e8e
Here we will go through the steps for configuring docker container as build slaves for Jenkins.
The first thing we should do is set up a docker host. Jenkins server will connect to this host for spinning up the slave containers.
[Important] Configure a Docker Host With Remote API
Jenkins master connects to the docker host using REST API’s. So we need to enable the remote API for our docker host.
All the docker configurations are present in the file/lib/systemd/system/docker.service
. In that file, there is an ExecStart parameter.
Open the file/lib/systemd/system/docker.service
, search for ExecStart and add value as shown below.
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
The above command will bind the docker engine server to the Unix socket as well as TCP port 4243. “0.0.0.0” means docker-engine accepts connections from all IP addresses.
Note: Also, please add necessary firewall rules for your server to accept connection on port 4243
Now for all the changes to take place, you need to restart the daemon and Docker service. Execute the following commands to do that.
sudo systemctl daemon-reload
sudo service docker restart
Now, remote API is enabled on your docker host. To test this, there are a few ways.
How do I access Docker API?
You can use curl command-line utility or REST APIs to access the Docker API. Both ways are explained below.
Test using curl
Get the IP address of your Docker host where you enabled remote API and execute the following command from any terminal which supports curl. You can test with the localhost as well.
Note: replace the IP with your Docker host IP or localhost as shown below.
curl http://localhost:4243/version
curl http://172.42.42.201:4243/version
This command will output all the images in your docker host in JSON format.
jenkins@2e208792831c:~/logs/tasks$ curl http://172.42.42.201:4243/version
{“Platform”:{“Name”:”Docker Engine — Community”},”Components”:[{“Name”:”Engine”,”Version”:”19.03.11",”Details”:{“ApiVersion”:”1.40",”Arch”:”amd64",”BuildTime”:”2020–06–01T09:10:54.000000000+00:00",”Experimental”:”false”,”GitCommit”:”42e35e61f3",”GoVersion”:”go1.13.10",”KernelVersion”:”4.15.0–99-generic”,”MinAPIVersion”:”1.12",”Os”:”linux”}},{“Name”:”containerd”,”Version”:”1.2.13",”Details”:{“GitCommit”:”7ad184331fa3e55e52b890ea95e65ba581ae3429"}},{“Name”:”runc”,”Version”:”1.0.0-rc10",”Details”:{“GitCommit”:”dc9208a3303feef5b3839f4323d9beb36df0a9dd”}},{“Name”:”docker-init”,”Version”:”0.18.0",”Details”:{“GitCommit”:”fec3683"}}],”Version”:”19.03.11",”ApiVersion”:”1.40",”MinAPIVersion”:”1.12",”GitCommit”:”42e35e61f3",”GoVersion”:”go1.13.10",”Os”:”linux”,”Arch”:”amd64",”KernelVersion”:”4.15.0–99-generic”,”BuildTime”:”2020–06–01T09:10:54.000000000+00:00"}
Once you enabled and tested the API, you can now start configuring the Jenkins server.
Create a Jenkins Slave Docker Image
I have created a Jenkins Image. You can use this image or use its Dockerfile a reference for creating your own.
Make sure sshd service is running and can be logged into the containers using a username and password. Otherwise, Jenkins will not be able to start the build process.
Note: The default ssh username is jenkins and password is also jenkins as per the given Dockerfile. You will have to use these credentials in the below configuration.
Configure Jenkins Server
Step 1: Head over to Jenkins Dashboard –> Manage Jenkins –> Manage Plugins.
Step 2: Under the Available tab, search for “Docker” and install the docker cloud plugin and restart Jenkins. Here is the official plugin site. Make sure you install the right plugin as shown below.
Step 3: you will find dedicated cloud configuration under Manage Jenkins –> Manage Nodes and Clouds
Step 4: Under docker, you need to fill out the details as shown in the image below.
Note: Replace “Docker URI” with your docker host IP. For example,
tcp://172.42.42.201:4243
You can use the “Test connection” to test if Jenkins is able to connect to the Docker host.
Step 5: Now, from “Docker Agent Template” dropdown, click the “Add Docker template” and fill in the details based on the explanation and the image given below and save the configuration.
- Labels — Identification for the docker host. It will be used in the Job configuration. Here we use
java-docker-slave
- Name: Name of the docker template. Here we use the same name as label ie, java-docker-slave
- Docker Image — myjenkinsslave:latest or the image that you created for the slave.
- Remote Filing System Root — Home folder for the user you have created. In our case, it’s /home/jenkins
- Credentials — click add and enter the SSH username and password that you have created for the docker image. Leave the rest of the configuration as shown in the image below and click save. If you are using my Docker image, the user will be
jenkins
& password is alsojenkins
. - If image is already installed on docker host, then select pull strategy “never pull”
Test Docker Slaves Using FreeStyle Job
Now that you have the slave configurations ready,
- Create a freestyle job, select “
Restrict where this project can be run
” option and select the docker host as a slave using the label. - Add a shell build step which echoes a simple “
Hello World
“
If you have done all the configurations right, Jenkins will spin up a container, builds the project and destroys the container once the build is done.
First you will see a pending notification as Jenkins tries to deploy a container on run time and establishes an SSH connection. After a few seconds, your job will start building.
You can verify on Docker slave node as well -
root@kworker-rj1:~/jenkins-slave# docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
afb9819cd2de myjenkinsslave:latest "/usr/sbin/sshd -D -…" 7 seconds ago Up 6 seconds 0.0.0.0:32769->22/tcp affectionate_hugle
Errors You may encounter:
- Jenkins will not be able to deploy containers on the host:- Please make sure you have proper connectivity to the docker host on API port.
- Jenkins builds goes in pending state forever:- Make sure you have Docker ports access from Jenkins to docker host.
Conclusion
We learnt here the process of setting up dynamic jenkins slaves using Docker. It can be further customized to fit your specific use cases.
Hope you like the tutorial. Please let me know your feedback in the response section.
Happy Learning!