Skip to content

Instantly share code, notes, and snippets.

@thealmightygrant

thealmightygrant/A-Journey-into-Helm.html

Forked from ryanj/Fork-This-Talk.html
Last active January 7, 2022 03:33
Show Gist options
  • Save thealmightygrant/fb40cb3b372ed339cc3d90bb67d8d1ba to your computer and use it in GitHub Desktop.
Save thealmightygrant/fb40cb3b372ed339cc3d90bb67d8d1ba to your computer and use it in GitHub Desktop.
Download ZIP
A Journey into Helm
Raw
A-Journey-into-Helm.html
<section>
<h1>A Journey into Helm</h1>
<h2>Grant Sherrick</h2>
</section>
<section data-transition="linear">
<section id="where-we-started">
<h2>We started with a git repo.</h2>
<br>
<pre><code>the-greatest-kafka-consumer
├── Dockerfile
├── README.md
├── package.json
├── src
│   └── index.js
├── test
│   └── index_test.js
├── dev
│   ├── the-greatest-deployment.yml
│   ├── the-greatest-cm.yml
│   └── the-greatest-job.yml
├── local
│   ├── the-greatest-deployment.yml
│   ├── the-greatest-cm.yml
│   └── the-greatest-job.yml
└── the-greatest-gif.gif</code></pre>
</section>
<section id="issues-in-our-repo">
<h2>We had some issues in that repo.</h2>
<ul>
<li>Not DRY</li>
<li>No local development environment</li>
<li><b>Dirty</b>, persistent Kafka</li>
<li>Manual, inconsistent deployments</li>
</ul>
</section>
<section id="templating-options">
<h2>We looked into some options</h2>
<ul>
<li class="fragment">Bash script all the things</li>
<li class="fragment">A Custom Templating Service in Golang</li>
<li class="fragment">Jsonnet</li>
<li class="fragment">Helm</li>
</ul>
</section>
<section id="helm-overview">
<h2>Helm</h2>
<span class="fragment">
<p>Helm is a tool for managing Kubernetes charts.</p>
<p>Charts are packages of pre-configured Kubernetes resources.</p>
</span>
</section>
<section id="k8s-resources-explanation">
<h3>Kubernetes Resources are:</h3>
<ul class="fragment" style="float: left; padding-left: 6.5em;">
<li>pods</li>
<li>deployments</li>
<li>jobs</li>
<li>configmaps</li>
<li>secrets</li>
<li>and many more</li>
</ul>
</section>
<section id="k8s-manifests-explanation">
<h3>Kubernetes Manifests are:</h3>
<ul class="fragment">
<li>pod.yaml</li>
<li>deployment.yaml</li>
<li>job.yaml</li>
<li>configmap.yaml</li>
<li>secret.yaml</li>
<li>and all of the other k8s yamls</li>
</ul>
</section>
<section id="what-is-a-chart">
<h2>So...what is a Helm chart?</h2>
<br>
<h3 class="fragment" style="text-align: left; padding-left: 2.4em;">A Helm chart contains:</h3>
<ul>
<li class="fragment" style="text-align: left;">a set of templated kubernetes manifests.</li>
<li class="fragment" style="text-align: left;">collective versioning information about the chart.</li>
<li class="fragment" style="text-align: left;">a list of charts that this chart depends upon.</li>
</ul>
</section>
<section id="the-greatest-kafka-consumer-chart">
<h2>We created a helm chart for <code>the-greatest-kafka-consumer</code>:</h2>
<br>
<pre><code>the-greatest-kafka-consumer
├── Chart.yaml
├── NOTES.md
├── bin
│   ├── build-and-push-image
│   ├── run-integration-tests
│   ├── deploy-from-helm-repo
│   └── push-to-helm-repo
├── charts
│   ├── local-kafka-0.2.1.tgz
│   ├── the-greatest-kc-dev-0.1.1.tgz
│   ├── the-greatest-kc-local-0.1.1.tgz
│   ├── the-greatest-kc-prod-0.1.1.tgz
├── local-charts
│   ├── the-greatest-kc-dev
│   │   ├── Chart.yaml
│   │   └── values.yaml
│   ├── the-greatest-kc-local
│   │   ├── Chart.yaml
│   │   └── values.yaml
│   └── the-greatest-kc-prod
│   │   ├── Chart.yaml
│   │   └── values.yaml
├── requirements.lock
├── requirements.yaml
├── templates
│   ├── _helpers.tpl
│   ├── the-greatest-deployment.yaml
│   ├── the-greatest-cm.yaml
│   └── the-greatest-job.yaml
└── values.yaml</code></pre>
</section>
<section id="the-greatest-kafka-consumer-templates">
<h2>All of our k8s manifests were templated!</h2>
<br>
<pre><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: {{ template "name" . }}
annotations:
checksum/theGreatestConfigMap: {{ include (print $.Template.BasePath "/the-greatest-cm.yaml") . | sha256sum }}
labels:
app: {{ template "name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
maxSurge: 3
replicas: {{ .Values.theGreatestKafkaConsumer.replicaCount }}
template:
metadata:
annotations:
app: {{ template "name" . }}
labels:
app: {{ template "name" . }}
release: {{ .Release.Name }}
spec:
restartPolicy: {{ .Values.theGreatestKafkaConsumer.restartPolicy | quote }}
imagePullSecrets:
- name: {{ .Values.theGreatestKafkaConsumer.imageSecretName | quote }}
containers:
- image: {{ .Values.theGreatestKafkaConsumer.image | quote }}
imagePullPolicy: Always
name: the-greatest-kafka-consumer
volumeMounts:
- mountPath: /root/.aws
name: aws-creds
env:
- name: BOOTSTRAP_SERVERS
value: {{ .Values.theGreatestKafkaConsumer.kafkaServers | quote }}
- name: SCHEMA_REGISTRY_URL
value: {{ .Values.theGreatestKafkaConsumer.schemaRegistryUrl | quote }}
- name: CONNECT_REST_ADVERTISED_HOST_NAME
value: {{ template "name" . }}
ports:
- containerPort: 4563
protocol: TCP
volumes:
- secret:
defaultMode: 420
secretName: {{ .Values.theGreatestKafkaConsumer.s3SecretName | quote }}
name: aws-creds</code></pre>
</section>
<section id="the-greatest-kafka-consumer-manifests">
<h2>Our live k8s resources were versioned!</h2>
<h3>using the helm server (Tiller)</h3>
<br>
<pre><code>$ kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
the-greatest-kc 1 1 1 1 23h
tiller-deploy 1 1 1 1 1d
$ helm upgrade --install the-greatest-kc-dev ./the-greatest-kafka-consumer --set tags.the-greatest-kafka-consumer-dev-values=true
Release "the-greatest-kc-dev" has been upgraded. Happy Helming!
LAST DEPLOYED: Tue Jan 30 13:52:55 2018
NAMESPACE: data-processing
STATUS: DEPLOYED
RESOURCES:
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
the-greatest-kc 1 1 1 1 21h
==> v1/ConfigMap
NAME DATA AGE
the-greatest-cm 1 21h
$ helm ls
NAME REVISION UPDATED STATUS CHART NAMESPACE
the-greatest-kc-dev 2 Mon Jan 29 23:06:46 2018 DEPLOYED the-greatest-kafka-consumer-0.1.1 data-processing</code></pre>
</section>
<section id="the-greatest-kafka-consumer-local-kafka">
<h2>We got a <b>clean</b>, ephemeral, local Kafka platform!</h2>
<br>
<pre><code>local-kafka
├── Chart.yaml
├── templates
│   ├── _helpers.tpl
│   ├── kafka-deployment.yaml
│   ├── kafka-service.yaml
│   ├── rest-proxy-deployment.yaml
│   ├── rest-proxy-service.yaml
│   ├── schema-registry-deployment.yaml
│   ├── schema-registry-service.yaml
│   ├── zookeeper-deployment.yaml
│   └── zookeeper-service.yaml
└── values.yaml
</code></pre>
</section>
<section id="the-greatest-kafka-consumer-versioned-charts">
<h2>Our templates were versioned!</h2>
<h3>via Chart.yaml</h3>
<br>
<pre><code>apiVersion: v1
description: A Helm chart for the Greatest Kafka Consumer
name: the-greatest-kafka-consumer
version: 0.2.1</code></pre>
</section>
<section id="prism-long-term-storage-env-specific-charts1">
<h2>We got environment specific, templated k8s resources!</h2>
<br>
<pre><code>$ cat local-charts/prism-lts-local/values.yaml
exports:
environment-specific-data:
theGreatestKafkaConsumer:
kafkaServers: "local-kafka:9092"
schemaRegistryUrl: "http://local-schema-registry:8081"
s3SecretName: "s3-creds"
restartPolicy: "Always"
theGreatestJob:
name: "health-metrics-kc-job"
s3BucketName: "local-kc-job"</code></pre>
</section>
<section id="the-greatest-kafka-consumer-env-specific-charts2">
<h2>We got packaged, environment specific, templated k8s resources!</h2>
<br>
<pre><code>$ cat requirements.yaml
dependencies:
- name: local-kafka
version: "0.2.0"
repository: "file://../local-kafka"
tags:
- the-greatest-kafka-consumer-local-values
- name: the-greatest-kafka-consumer-local
version: "0.1.1"
repository: "file://local-charts/the-greatest-kafka-consumer-local"
tags:
- the-greatest-kafka-consumer-local-values
import-values:
- environment-specific-data
- name: the-greatest-kafka-consumer-dev
version: "0.1.1"
repository: "file://local-charts/the-greatest-kafka-consumer-dev"
tags:
- the-greatest-kafka-consumer-dev-values
import-values:
- environment-specific-data
- name: the-greatest-kafka-consumer-prod
version: "0.1.1"
repository: "file://local-charts/the-greatest-kafka-consumer-prod"
tags:
- the-greatest-kafka-consumer-prod-values
import-values:
- environment-specific-data</code></pre>
</section>
<section id="prism-long-term-storage-env-specific-charts3">
<h2>We got easily modifiable, packaged, environment specific, templated k8s resources!</h2>
<br>
<p>There is a priority order for setting templated values:
<ol>
<li>--set or -f <i>other-values.yaml</i> using the <code>helm</code> CLI tool</li>
<li><i>values.yaml</i> in required charts (including other envs)</li>
<li><i>values.yaml</i> in the main chart</li>
</ol>
</section>
<section id="using-a-chart1">
<h2>Not only can we chart up our stuff, there's public charts too</h2>
<br>
<code class="fragment">helm install stable/prometheus</code>
</section>
<section id="using-a-chart2">
<h2>How do I find out what is configurable?</h2>
<br>
<h3 class="fragment" style="text-align: left; padding-left: 1.5em; padding-bottom: 0.5em;">1. <b>look at its values.yaml file</b></h3>
<pre class="fragment"><code>nodeExporter:
## If false, node-exporter will not be installed
##
enabled: true
# Defines the serviceAccountName to use when `rbac.create=false`
serviceAccountName: default
## node-exporter container name
##
name: node-exporter</code></pre>
</section>
<section id="using-a-chart2">
<h2>How do I configure a public helm chart?</h2>
<br>
<code class="fragment" style="text-align: left;">helm install stable/prometheus --set nodeExporter.enabled=false</code>
<br><br>
<code class="fragment" style="text-align: left;">helm install stable/prometheus -f other-values.yaml</code>
</section>
<section id="helm-disadvantages">
<h2>What doesn't Helm do well?</h2>
<br>
<p>NOTE: all of these are my opinions</p>
<ul>
<li class="fragment"><code>helm upgrade</code></li>
<li class="fragment">Doesn't follow the Unix philosophy</li>
<li class="fragment">Obfuscates some kubectl errors and logs</li>
<li class="fragment">Conventions abound (chart versioning, checksum, exports, helper functions)</li>
</ul>
</section>
<section id="helm-advantages">
<h2>What does Helm do well?</h2>
<br>
<p>NOTE: all of these are my opinions</p>
<ul>
<li class="fragment">templating (especially across environments)</li>
<li class="fragment">intra-chart checksum-based redeployment</li>
<li class="fragment">a great set of docs and a very active community</li>
<li class="fragment">dependency resolution between projects</li>
</ul>
</section>
<section>
<h2>Thanks!</h2>
<h1>Have a helm of a Day!</h1>
</section>
</section>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment