GitOps Boot Camp

Part 1: Introduction

1. Required knowledge
   1. Git: Committing code and creating pull requests
   2. Kubernetes: Deploying a service to Kubernetes and basic checks with kubectl
   3. Docker: Pushing an image to a Docker repository
   4. CI/CD: GitOps reverses the traditional understanding of continuous integration/continuous development.
2. Core concepts: A quick introduction
   1. Immutable infrastructure
   2. Infrastructure as code
   3. Orchestration
   4. Convergence
   5. CI/CD
3. What GitOps is not
   1. GitOps is not infrastructure as code.
   2. GitOps doesn't replace continuous integration (CI).
4. The use case: Deploying a highly available microservice
   1. Deploying a microservice to Kubernetes, with all the surrounding infrastructure to make it available

Part 2: Setting up the Tools

1. Kubernetes
   1. In advance: Setting up a cluster from scratch is time-consuming, even if you use a managed solution like EKS. Pre-allocating a cluster per person is something you can do in advance.
2. Preparation: Setting up kubectl
   1. Every participant should have credentials to connect to the cluster using kubectl.
3. Preparation: Access a cluster through kubectl/k9s.
   1. Check running pods.
      1. Check deployments.
4. Repository
5. Preparation: Infrastructure repository
   1. An empty repository in GitHub/GitLab to use for deploying infrastructure
6. Application repository
   1. Strictly speaking, you don't need to separate the application and infrastructure, but it's easier to understand what goes where this way.
      1. A sample application that serves a web server with a hello world response as the baseline (NodeJS-based, for instance)
7. ArgoCD
8. Why ArgoCD?
   1. ArgoCD is tightly integrated with Kubernetes and closely follows the GitOps mindset. Therefore, it's a good tool to showcase GitOps.
9. Exercise: Add ArgoCD to the cluster.
   1. Create namespace.
   2. Deploy ArgoCD to the cluster.
   3. Access ArgoCD using the CLI.

Part 3: Deploying a Microservice

1. Exercise: Prepare a simple microservice to be deployed in k8s.
   1. Build sample application as a Docker container (Dockerfile can be provided in advance)
   2. Push service to a Docker registry (cloud-native, docker.io, or quay.io)
2. Exercise: Create a k8s deployment.
   1. Create a Kubernetes deployment definition in code for the application (here's a sample).
   2. Push code to infrastructure repository.
   3. Create an application in ArgoCD.
   4. This time, you'll use the ArgoCD CLI so you can see that part. You'll move to use Git from here on, which is more aligned to GitOps.
   5. Sync the application.
   6. Again, use the CLI.
   7. Test: Use kubectl check to ensure that deployment works.
3. Automated synchronization
   1. Pull versus push: How ArgoCD can read from a repository and automatically apply the changes
   2. Exercise: Activate synchronization so that further changes happen when you push code to the infrastructure repository.
4. Exercise: Create a k8s service. (A deployment alone doesn't expose the microservice, so let's build on that.)
   1. Create service definition.
5. Pull request
   1. This is an opportunity to introduce the pull request aspect of the flow. You can extend pull requests so that extra checks are performed, using something like GitHub Actions.
      1. Implementing CI with something like GitHub Actions isn't part of the exercise, although it's something that you can complete as an extra exercise. (See the bonus section at the end of this post.)
      2. Test: Carry out a kubectl check to prove that service was deployed.
6. Exercise: Create a load balancer. (You still can't access service from the outside.)
   1. Create loadbalancer k8s definition for cloud provider.
   2. Pull request
   3. Test: Curl to load balancer address to ensure that service is actually online.
7. Exercise: Update the application.
   1. Change something in the application, such as the body of the response of a route in the application.
   2. Rebuild container with a new tag and push it to Docker registry.
   3. Update k8s deployment to use new tag.
   4. Pull request
   5. Test: New version of the app should be deployed.
8. Exercise: Update the infrastructure. (Why do this? So you can demonstrate that changing the application and the infrastructure results in blurry boundaries.)
   1. Update k8s deployment to be highly available (more than one replica).
   2. Pull request
   3. Test: kubectl shows that there are multiple pods running.
9. Wrap-up: This covers the workflow of deploying an application and then performing updates and changes on it.
   1. This is the core of GitOps!
   2. There are also other, more advanced use cases to cover.

Part 4: Promoting Changes Through Different Environments 

1. In advance: Prepare a second cluster.
   1. As with the first cluster, this is something to have prepared in advance.
2. Preparation: Register the cluster in ArgoCD to allow deployments to it.
3. From development to production
   1. Which options are there to represent different stages?
   2. This is an open discussion, as there's no set recipe to do environment promotion, with different options:
   3. Use different infrastructure repositories.
   4. Use different folders in the same infrastructure repository.
   5. Use branches.
4. Exercise: Promotion of a version
   1. Set up a second cluster (production) to read from a different folder.
   2. Copy the infrastructure created for the first folder into this one.
   3. Pull request
   4. Test: Second cluster should have the service available as well.
5. More advanced deployment scenarios (Controlled release is an important part of releasing traffic, especially to production. It's worth talking about the options that you have that can be based on the exact same building blocks as explained before.)
6. Exercise: Blue/Green
   1. Enable Argo Rollouts in cluster.
      1. Test: Observe rolling deployment with kubectl.
   2. Install argo-rollouts plugin for kubectl.
   3. Create rollout to apply to existing microservice.
7. Canary release
   1. Theory only (This can be a good lead-in to a discussion of the merits and tradeoffs of different deployment strategies.)
8.  Exercise: Error handling (This exercise shows that failure in infra deployment is expected and is handled through code changes—not panicked actions!)
   1. Introduce an error in the hello world application (this results in a thrown exception instead of starting the webserver).
   2. Rebuild the container with a new tag and push it to Docker registry.
   3. Update k8s deployment to use new tag.
   4. Pull request
   5. Test: Confirm with kubectl that deployment is failing.
   6. Revert a failed change through code.

Part 4: Security in GitOps

1. Accessing resources
   1. kubectl shouldn't replace observability, such as logging and monitoring (similar to secure shell—SSH—into a production server)
2. Secrets
   1. No plaintext secrets should ever be stored in Git.
3. Vault
   1. This is theory only because it's probably too much to do for a practical exercise.
4. Exercise: Sealed secrets
   1. Depending on time, this can be treated as theory or as an exercise. Furthermore, you can split it in two depending on how much time you have.
      1. Modify microservice to read the secret and make it available through a request.
   2. Provision secrets in the infrastructure repository.
   3. Use secrets from either the cluster or the application.
   4. Install the sealed secrets controller.
   5. Inject an encrypted secret in the infrastructure repository.
   6. Modify Kubernetes deployment to inject a secret into the microservice.

Part 7: Recap

1. Core concepts: Infra as code, Git as the source of truth, pull model, converging changes
2. Core flow: declare infrastructure, commit it, pull request for review, merge to apply
3. Next steps
   1. Automated promotion (If a deployment to a staging environment succeeds, then trigger a deployment to production.)
   2. Observability (microservices that export metrics, logging aggregator, and monitoring)

Bonus exercises

1. Replace manual steps (push Docker container, build application code) with CI, such as GitHub actions.
2. Introduce templating (Jsonnet, Helm) to foster reuse of Kubernetes resources.
   1. Exercise: Parameterize deployment so that port can be defined as configuration. Deploy a second copy of the same service with a different name, running on a different port.
3. Install more advanced resources in the cluster using the same mechanism, such as an ingress controller.

Developers who want to understand how to deploy their applications to a target environment are the ideal audience for this course. These developers want to take ownership of the surrounding infrastructure while following best practices. 

Professionals who benefit from this course include:

• Software Engineers
• Software Developer
• Site Reliability Engineers
• Systems Engineers
• Network Engineers
• Anyone involved with DevOps-style workflows
• Anyone involved with IT infrastructure

• Set up an environment to do GitOps (You'll use ArgoCD for that.)
• Deploy and update an application to Kubernetes using GitOps, including the required auxiliary infrastructure
• Scale the GitOps approach to multiple environments
• Address operational concerns, such as security