Adopting GitOps for Enhanced Operations

For the last 20 years, the software development landscape has changed dramatically. The way developers work on building the applications, testing and delivering became both fast and more autonomous with emerging new technologies. Would you be surprised if we remind you of the old-school development process where we had to compile locally and send compiled codes to the physical servers via FTP or copying and pasting them to the server’s desktops with RDP?

Now we leverage DevOps principles, CI/CD pipelines, Infrastructure as Code and containers to automate and standardize our workflows for maximum agility. These tools and methodologies bring a cultural shift towards collaboration, named as DevOps and GitOps. Let’s look at the underlying approaches and principles within GitOps first, and then the benefits of adapting GitOps practices.

Agile Software Development and DevOps Culture

With the shift from monolithic applications towards service oriented architecture and even microservices, developers now are able to specialize in their work collaboratively for overall success. Each developer within a particular team has their own specialties and responsibilities rather than working on the one giant application altogether. This shift also promoted a new era within software development, often called agile development processes, thanks to cloud. Traditional waterfall development usually caused slow steps and silos within the development lifecycle, where each step would require a long time due to distinct teams and operations, and sometimes becoming a bottleneck for the release. With agile development, software delivery became team work, where the whole team works together for successful delivery.

Besides the DevOps tools and methodologies that enable continuous integration and delivery, the end game of such an approach is known as DevOps culture. DevOps culture applies to the whole organization rather than separate attributes and it highlights the importance of collaboration across teams. By this way, everyone involved in the software development lifecycle has shared responsibilities and shared success metrics rather than separate priorities for each team. The codes can now be committed and deployed anytime during the development lifecycle with automated processes of CI/CD pipelines and your application is able to work in every environment from development to the production.

Infrastructure as Code

Extending the DevOps philosophy just a little further for greater velocity, there comes Infrastructure as Code approach. Infrastructure as Code is simply managing and provisioning your resources through code, rather than using configuration tools manually. As the complexity of environments increases, managing infrastructure becomes hard work and also much more vulnerable to human errors. With the IaC approach the infrastructure is treated as code, which enables standardization and speed across the processes. But still, infrastructure management is outside of the system rather than being inside.

Here is our previous blog post on Infrastructure as Code for more about its advantages and implementation:

Container Orchestration

Today we are all familiar with the term containerization, which enables us to bundle our software whole differently than traditional processes and with ease. When it comes to containerized applications, Kubernetes is the first thing that comes to mind for orchestrating our containerized application deployments. Kubernetes helps us manage our complex infrastructure with automatic deployment of committed files.

The underlying principle of Kubernetes is the declarative approach for desired states of objects. Creating, updating, deleting and managing objects are defined with configuration files within Kubernetes, which are also called manifests. What Kubernetes do for us that we don’t have to go and manually launch, configure and maintain infrastructure resources.

GitOps Practices to Enhance All Workflows

Yet, the operations part of the DevOps sometimes is not fully understood and utilized due to infrastructure complexity and management issues. This is where GitOps comes in. GitOps, a term popularized by Weaveworks, is a set of best practices for deployment, monitoring and management of infrastructure and provisioning resources. GitOps takes the DevOps philosophy and Infrastructure as Code approach together to handle infrastructure changes through automation and pull request based. By this way, software delivery and deployment of infrastructure resources are fully automated, just as the way developers are already familiar with: using Git repositories. Managing infrastructure through codes enables us to store our code in a source control system as Git, which gives its name to the term GitOps. This way, we can store our code in Git and let it automatically trigger updates to our infrastructure when a change in the source code is detected.

Here is a brief example of applying GitOps practices within your CI/CD pipeline on AWS:

Adopting GitOps with Kubernetes and Cloud-Native Stack

GitOps standardizes the deployment, configuration, monitoring and making changes on Kubernetes and all the applications that run on these clusters with IaC approach and automation. You can turn all your deployments on your clusters into code, store and version them in Git. Consider managing your thousands of containers for your applications. Besides the extreme amount of time needed, the service dependencies and infrastructure health becomes almost impossible to manage effectively. GitOps enables you to deploy your changes without much expertise in orchestration management. In this way, developers can work more productively and autonomously without worrying about container orchestration.

With Gitops for Kubernetes, we simply define our manifests for our desired infrastructure state and object definitions, and keep them in the Git repository. So, when we are working on our application and making changes, we don’t have to handle these resource launches and management. GitOps works with Pull Request basis, so when these pull requests are approved and merged, the changes on our container clusters are deployed automatically. Kubernetes’s controllers automatically converge the system to the desired state described in the manifest.

In this way, changes on infrastructure are orchestrated within the system itself, rather than us manually handling it from the outside. The result is completely integrated work processes and pipelines, and of course increased reliability! We just go on and describe our application topology and needs, and Kubernetes’ controllers handle the rest.

Here is a brief example of how to apply GitOps for your Kubernetes clusters on AWS:

Flux CD is a GitOps operator that runs on your Kubernetes clusters. You can configure it to monitor and detect changes within your Git repository and take actions to apply the changes for your infrastructure. You don’t manually update your application, the operator pulls the code changes from the Git repository within specified periods and updates the configuration of your clusters according to the desired states you have specified.

GitOps Principles

Declarative over imperative principles

GitOps help us to manage our infrastructure declaratively, rather than applying manual instructions for every single change. GitOps allows us to describe the desired state of our infrastructure and applications in the source control. By this way, it also aligns with the foundational declarative principle of Kubernetes.

Single source of truth

Declarative approach to the desired state of infrastructure also makes the Git repository your single source of truth for the infrastructure. By storing our code and manifests in Git, the repository becomes our single source of truth for what our infrastructure should ideally be like. The code in the repository should also be versioned for effective revert and rollbacks for our infrastructure, supporting the single source of truth.

Continuous convergence to desired state

GitOps allows us to adapt tools to ensure that our actual infrastructure matches with the desired state. Automation and alerting systems in this structure helps us achieve an auto-scaling and auto-healing infrastructure without complex infrastructure configuration and management.

GitOps: What are the benefits?

Workflow Standardization

Storing our Infrastructure as code and version controlling in Git repository allows us to standardize the desired states as a single source of truth. Applying Pull Request basis in each step of code commitment and deployment,, our CI/CD pipeline becomes fully automatized and reproducible via Git with the declarative descriptions.

Disaster Recovery

Using Git as the single source of truth for our infrastructure, it becomes possible to revert or rollback our change commitments in case of any issues. Although GitOps is not a brand new way of infrastructure management, it guides us to make our infrastructure reliable and consistent when things go wrong.

Productivity

Using Git to manage and deploy changes on our infrastructure reduces the time spent on infrastructure configurations, manual operations and fixing human errors. Developers can manage their application and infrastructure changes more rapidly with Git repositories. These practices also allow developers to experiment with new technologies and features easily without affecting the whole infrastructure.

Security and Compliance

The correctness of Git ensures you that you can track and monitor all changes across your infrastructure and clusters. With GitOps, you don’t need cluster credentials or human intervention to make changes on the system. The system can only be changed through Git as a segregated environment. The audit logs and monitoring also enables you to track changes on the infrastructure into the single users who made those changes and enhance your compliance processes.

Here are some of our favorite GitOps toolset:

• Flux: The GitOps Kubernetes operator

• Gitkube: Build and deploy docker images to Kubernetes using git push

• JenkinsX: Continuous Delivery on Kubernetes with built-in GitOps

• Helm Operator: The Flux Helm Operator, for declarative Helming

What is yours?