Jupyter for SRE

Jupyter is a tool that came out of the data science community. In science, being able to replicate experiments is of the utmost importance -- so a tool where you can "show your work" is helpful. However, being able to show your work -- have colleagues validate what you have done, repeat it if needs be, and learn new techniques -- is also useful in the world of Site Reliability Engineering and DevOps.

The Jupyter console allows us to experiment (carefullly!) with APIs, running one function at a time, and validating the results. It allows building the needed automation from simple atoms, all the while learning how to use them.

The Jupyter Python kernel is popular in the data science community because so many data science libraries are available for Python. Luckily, the same is true in the SRE/DevOps community.

import github3
import os

The GitHub API has several client libraries in Python. My personal favorite is github3 -- I find its interface allowing remarkably idiomatic Python.

fname=os.path.expanduser("~/.git-token")
with open(fname) as fin:
    token = fin.read().strip()
gh = github3.login(token=token)

I have prepared a GitHub authentication token in a file. This allows the NoteBook to be published widely, without leaking access credentials. Never put usernames and passwords in Jupyter notebooks!

repositories = list(gh.organization('twisted').iter_repos())
repositories[:3], len(repositories)

([<Repository [twisted/txmongo]>,
  <Repository [twisted/twisted]>,
  <Repository [twisted/txaws]>],
 26)

This is a list of the repositories in the Twisted GitHub organization. It is nice to be able to validate we got a reasonable value by checking the first three. In previous versions of the notebook, my usage of github3 had an error -- and this was a list of all repositories I had access to, including private ones! Being able to inspect values interactively meant I could fix the bug easily.

repositories[1].branch('trunk').commit.sha

'51d23b6cb004e60ed699b3b100b4c476f5aebc4e'

As an example of why this might be useful, we are checking the commit hash of the trunk branch. This can be used in validating which version we are running somewhere, or checking if there have been new commits.

The GitHub API is big, and this is not meant to be an exhaustive guide to it. However, this approach is powerful -- it can be used, for example, to automatically create pull requests for a list of repositories. This comes in handy when needing to change the build structure, for example.

from fabric import api as fabpi

The Fabric library, (here used in its fabric3 fork) is an all-purpose ad-hoc library for operations.

Again, a full tutorial is beyond the scope of this article.

fabpi.local("uname -a", capture=True)

However, the advantages of running Fabric from Jupyter notebook are big. Because Fabric is specifically designed for ad-hoc operations, there is often no way to know exactly what someone did. Even with a fabfile, the logging is often lacking.

Running the functions from a notebook means an official log of what was done -- that can easily be attached to the relevant ticket, or to the post-mortem. (Either the post-mortem that the operations were meant to fix, or the ones they inevitably caused).

import docker
client = docker.DockerClient(base_url='unix://var/run/docker.sock')

The Docker client is also available as a Python library. Once again, the possibilities are endless.

[im for im in client.images.list() if (im.tags or [''])[0].startswith('debian')]

[<Image: 'debian:latest'>,
 <Image: 'debian:stable-slim', 'moshez/debian:2017-10-26T10-58-56-455022'>]

import boto3

The boto3 library is an API for the Amazon Web Services -- which includes everything from load balancers, through orchestrating containers, to sending e-mail.

The Jupyter console is a great adjunct to the AWS web console -- while results can often be inspected in the web console, any actions done from the notebook can be repeated, tweaked, and automated.

ec2 = boto3.client('ec2', region_name='us-west-2')