Jenkins Agents: A Comprehensive Guide to Scaling Your Jenkins Environment

Jenkins Agents: A Comprehensive Guide to Scaling Your Jenkins Environment

Day 28 : #90DaysOfDevOps Challange

Jenkins is a popular automation server that is widely used in software development teams to automate the building, testing, and deployment of software. It offers a flexible and customizable platform that can be extended with a vast library of plugins. However, as your team grows and the number of projects increases, it can become challenging to manage all the workload on a single Jenkins server. This is where Jenkins agents come in.

What are Jenkins Agents?

Jenkins agents, also known as slaves, are separate nodes that connect to the Jenkins master and handle the execution of jobs. They can be physical machines, virtual machines, or containers. Each agent is assigned a unique label that identifies it.

When you create a Jenkins job, you can specify the agent on which the job should run. When you trigger the job, the Jenkins master schedules the job to run on the specified agent. The agent then runs the job and sends the results back to the Jenkins master.

Why Use Jenkins Agents?

By using Jenkins agents, you can scale up your Jenkins environment and distribute workloads across multiple machines. This allows you to run more builds simultaneously, reduce build times, and improve overall efficiency.

For example, suppose you have a large project with multiple components that require different environments for building and testing. In that case, you can set up multiple agents, each with a specific environment, to handle the build and test process for each component.

Another advantage of using Jenkins agents is that you can choose the type of machine or container that best suits your needs. For example, you can use a high-performance machine for CPU-intensive builds or a container for lightweight builds. This allows you to optimize your Jenkins environment for your specific requirements.

Setting Up a Jenkins Agent

To set up a Jenkins agent, you need to install the Jenkins agent software on the machine or container you want to use as an agent. Once installed, you can configure the agent to connect to the Jenkins master using a unique secret key. You can then assign the agent to specific jobs in your Jenkins environment using the label you created for it.

Jenkins supports several methods for setting up agents, including:

  • Launch agents via Java Web Start: This method allows you to launch agents on-demand from the Jenkins master using Java Web Start.

  • Launch agents via SSH: This method allows you to launch agents on remote machines using SSH.

  • Launch agents via JNLP: This method allows you to launch agents on remote machines using the Java Network Launching Protocol.

Adding and Removing Agents

One of the benefits of using Jenkins agents is that you can easily add or remove agents as your needs change. This provides a flexible and scalable solution for managing your Jenkins environment.

To add a new agent, you need to install the agent software on the machine or container and configure it to connect to the Jenkins master. Once connected, you can assign the agent to specific jobs using the label you created for it.

To remove an agent, you simply need to disconnect it from the Jenkins master. Any running jobs will be stopped, and the agent will be removed from the list of available agents.

Conclusion

In conclusion, Jenkins agents are a powerful tool for scaling up your Jenkins environment. By distributing workloads across multiple machines, you can increase efficiency and reduce build times. If you're using Jenkins and haven't explored the use of agents yet, it's definitely worth considering as your team and workload grow.

Jenkins offers a flexible and customizable platform for building, testing, and deploying software. By using Jenkins agents, you can scale up your Jenkins environment and distribute workloads across multiple machines. This allows you to run more builds simultaneously, reduce build times, and improve overall efficiency.

Setting up Jenkins agents is relatively easy, and Jenkins provides several methods for doing so. You can also easily add or remove agents as your needs change, providing a scalable solution for managing your Jenkins environment.

One thing to keep in mind when using Jenkins agents is security. Agents can potentially have access to sensitive data and resources, so it's important to ensure that proper security measures are in place. This includes restricting access to agents based on user roles and permissions and ensuring that agents are running in secure environments.

In addition to security, it's also important to monitor and manage your Jenkins agents. This includes regularly checking the status of agents and ensuring that they are running the latest software updates and patches.

Overall, Jenkins agents are a valuable tool for managing and scaling your Jenkins environment. By distributing workloads across multiple machines, you can increase efficiency, reduce build times, and improve overall productivity. If you're using Jenkins, it's worth exploring the use of agents to see how they can benefit your team and your workflow.

Task 1. Create an agent by setting up a node on Jenkins. Create a new AWS EC2 Instance and connect it to the master(Where Jenkins is installed). The connection of master and agent requires SSH and the public-private key pair exchange. Verify its status under the "Nodes" section.

  1. Log in to your AWS console and go to the EC2 dashboard.

  2. Launch two instances one for the master (jenkins-master) and the other as (jenkins-agent)

  3. Install java and Jenkins on jenkins-master and configure port 8080 as we have done so many times in our previous blogs

  4. Log in to the Jenkins master machine and generate a new SSH key pair using the command "ssh-keygen" in the .ssh/ folder.

  5. Copy the public key from the Jenkins master machine using the command "cat ~/.ssh/id_rsa.pub".

  6. Log in to the Jenkins agent machine using SSH and add the Jenkins master public key to the authorized_keys file using the command "vim~/.ssh/authorized_keys" and paste the copied public key.

  7. Save and exit the file.

  8. now by doing ssh ubuntu@<jenkins_agent>

  9. Exit from agent you can see change in ip adress below πŸ§πŸ§πŸ§πŸ§πŸ§πŸ§πŸ€“πŸ₯³

  10. We have succesfully connected Master with an agent

  11. Now lets do same using jenkins. So login to your Jenkins and click on " setup an agent"

  12. now add credentials jenkins and paste private key from master id_rsa into those credentials

  13. Save and refresh

  14. Oopsss 😣😣😣😣😣😣😣😣😣😣throwing errors but why

  15. Ohh Java is not installed on Agent

  16. So install java

  17. Lauch the node again

πŸ˜ŽπŸ˜ŽπŸ˜Žβœ¨βœ¨βœ¨πŸŽ‰πŸŽ‰πŸŽ‰πŸŽ‰ dev-agent in sync πŸ˜ŽπŸ˜ŽπŸ˜Žβœ¨βœ¨βœ¨πŸŽ‰πŸŽ‰πŸŽ‰πŸŽ‰

Task 2: let's Assign Task to our Agent

  1. Create a job pipeline of our node-todo App

  2. configure your pipeline

  3. Add pipeline and save

  4. Expose port 8080 and add webhook on GitHub to connect GitHub with Jenkins( you can check my previous blogs on Jenkins)

  5. Also Expose port 8000 on the agent and open the application on your browser

  6. Now click on build now

As you can see :

πŸ˜ŽπŸ˜ŽπŸ˜Žβœ¨βœ¨βœ¨πŸŽ‰πŸŽ‰πŸŽ‰πŸŽ‰running on dev-agent πŸ˜ŽπŸ˜ŽπŸ˜Žβœ¨βœ¨βœ¨πŸŽ‰πŸŽ‰πŸŽ‰πŸŽ‰

That's it! We have successfully set up a Jenkins agent on an EC2 instance and connected it to the Jenkins master.

Did you find this article valuable?

Support Dhananjay Kulkarni by becoming a sponsor. Any amount is appreciated!

Β