Snipper: Better YAML Templating

I got an itch a few days ago to try out an idea. This was, in part, motivated by a few conversations about Helm templates for Kubernetes. Helm Charts rely on weaving templating into your Kubernetes object definitions. The templating that Helm uses hails from the same tradition as ERB templates, mixed PHP/HTML pages, etc.

I started thinking on what it would be like if I could template Kubernetes definitions using something resembling Lift snippets - where the template and the instructions that lead to the final result are totally separated. And, since I’m working on getting a bit more familiar with Go - I decided to build it using Go. What I ended up with is Snipper.

Getting Snipper

I’ll start by saying that Snipper is very much a proof of concept.

Snipper is available on GitHub. Pre-built binaries are available on the releases page, or you can get it using go install if you have a Go development environment set up.

Once it’s installed you can use it by providing a template YAML file - which is just a regular old YAML file. You can then invoke snipper with the template and any number of transformers like so:

$ snipper template.yaml transformer1.yaml transformer2.yaml...

The resulting YAML will get sent to standard out.

Example

Let’s say that I’ve got this Pod definition pulled mercilessly from the Kubernetes documentation.

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command:
    - 'sh'
    - '-c'
    - 'echo The app is running! && sleep 3600'

I can use snipper to modify this pod to my heart’s content. Let’s say that I would like to add a team label to the container. I might define this in the transformer YAML file like so:

'metadata:labels:team': Farmdawg Nation

Transformer YAML files follow some special conventions. Specifically:

• The top-level keys are all in “selector format”
  • : separates selectors. metadata selects the metadata node at the root. metadata:labels selects the labels under metadata, but would not select the labels under spec if there were one.
  • Ending a sector with + causes snipper to append a value to a string or a list instead of replacing the value
  • Using [] as a selector selects all array members at the level you’re selecting at. spec:containers:[]:command:[] would select all components of the command array for all of the containers above.
• Selectors are only parsed at the top level of keys. Anything nested inside the selectors are assumed to be values that you would like to drop into your template.

These rules are still heavily in flux. They will likely change, but let’s take a few more examples of things we can do.

We could suffix the pod name with the number 2:

'metadata:name+': 2

We could change the image to alpine:

'spec:containers:[]:image`: alpine

We could add an additional container:

'spec:containers+':
- name: myapp-hello
  image: hello-world

And of course, a single transformer file can have multiple rules:

'metadata:labels:team': Farmdawg Nation
'metadata:labels:alerting': enabled
'spec:containers+':
- name: myapp-hello
  image: hello-world

All of these instructions remain separate from the actual template itself. You no longer have to litter a single file with the template and what needs to happen to it. With Snipper, these concepts are separate.

But what about logic?

You’ll notice that there aren’t any logic constructs. That’s actually an intentional decision. Snipper by itself isn’t a total replacement for a templating engine. However, when combined with an orchestration layer that conditionally applies transforms, you wouldn’t be far off. I have not (yet) written such an orchestration layer, but I could imagine a tool that defines, say, a:

• pod_template.yaml
• local.yaml (transforms for local deployment)
• staging.yaml (transforms for staging cluster deployment)
• prod.yaml (transforms for production cluster deployment)

What’s next?

I’m not sure. Part of this was scratching an itch. Part of it was wanting to develop some better informed opinions about Go. What I do next with this project will depend heavily on whether or not folks find it useful / interesting. So, if you find it useful / interesting please star it on GitHub, file bugs and feature requests, or share with your friends.