Fifty Shades of Ver

Sat 19 September 2020 by Moshe Zadka

Computers work on binary code. If statements take one path: true, or false. For computers, bright lines and clear borders make sense.

Humans are more complicated. What's an adult? When are you happy? How mature are you? Humans have fuzzy feelings with no clear delination.

I was more responsible as a ten year old than as a three year old. At 13, I reached the age when I was responsible for following Jewish law myself. At 18, the legal system trusted me to could drink alcohol and drive, and trusted me that I will keep the two activities distinct. In the US, you cannot become a senator before you are 30.

At what age are you responsible "enough"?

Software is written by humans, not computers. Humans with feelings, hopes, and dreams. We cry, we strive, we feel accomplished at times, and disappointed at others.

If you were designing a version system for computers, SemVer, or "Semantic Versioning", would make perfect sense. Each part number in a three-part version number is given a specific, distinct, definition:

  • Increment MAJOR for backwards incompatible changes.
  • Increment MINOR for changes which add functionality.
  • Increment PATH for bugfix-only backwards compatible release.

But software is not made by computers. It is made by humans.

Start small

A journey of a thousand miles begins with a single step. The first version of the Linux kernel printed As and Bs to the screen. The first version of Python didn't have modules. SemVer, to its credit, acknowledges that.

In versions like 0.x.y. SemVer defines the semantics:

Anything MAY change at any time. The public API SHOULD NOT be considered stable.

When something is small and fragile, it should be able to change. Every UNIX programmer knows the story of why Makefile treats tabs and spaces differently. In retrospect, causing a dozen of people a small amount of pain would probably have been better than staying with the problem.

Once the software is mature enough, the SemVer reasoning goes, just release 1.0.0 and commit to API stability.

Grow slowly

Given the amount of projects that have stayed on ZeroVer for a long time (or forever!) the assumption that commiting to API stability once the project matures is easy seems to not pan out.

Remember: software is written by humans. Fuzzy humans, in complicated social structures, who work together as best they can, using brains evolved to figure out politics in a fifty-person tribe in order to stay alive.

Once a social structure is in place and working, changing it is hard. In the ZeroVer days, there was no reason to figure out which changes broke API compatibility. There was no reason to clearly delinate which parts are "public" API and which are not. After all, there was no need.

Switching out of ZeroVer requires building all of this. Not switching out of ZeroVer does not require complicated social engineering. It is not surprising that it is hard. It's almost as if humans work better with slow changes, and not sudden revolutions.

Small commitments

Lately, I have been frustrated with some aspects of my life. COVID-19 did not cause them, but helped bring them into sharp focus. As the least embarassing example to admit in a public forum, I realized that while my book shelves are so full of books that shoving another one requires my 80s-kids Tetris skills, I have not read a single fiction book in the last three years. I used to be a voracious reader!

How do you change habits? I used to read, easily, 200 pages of fiction a day in my 20s. I have not gotten worse at reading. I could commit to reading 200 pages a day, and track my progress. If you have ever done that, you know what the outcome is. Every day, you look at the task, and you decide it is too big. You never begin.

Instead, I decided I will read 20 pages a day, and feel good about it. Feel good? I even decided to reward myself for every week where I hit this goal five out seven days.

The result? The last few weeks, I have been consistently been reading 20 pages a day, missing only one or two days.

When you are not good at something, as a person or as a group, and you want to get better, small commitments frequently achieved are the way to go.

SemVer does not work that way. It is all or nothing. SemWay or the Highway. Perhaps it is better to have a versioning system for humans, not a fictional alien race, if we assume software will keep being written by humans.

Deprecation policy

It is an easy change to say that no single change can "just" break an API. One change to deprecate, and one change to break. This is straightforward to verify. It is reasonable to have a policy for exceptions, but document the exceptions carefully.

Note that this change does not help potential users all that much by itself. After all, two PRs in close succession can land, and there is no reasonable upgrade path.

Should we feel good about making a small change that does not help anyone? Absolutely. Because it is small, and it is on the right path.

Now that this change becomes ingrained in the developer group, we can start mandating a minimum time between deprecation and breakage. At first, we can have a 0day policy: you can break, as long as the deprecated software has been released. This causes more releases to happen, making the team better at releasing. It helps users only minimally. However, at least with careful version pinning, there is an upgrade path.

Now, we can start making the number 0 a bit bigger. First, a week. Then, a month. Eventually, a quarter or a year. If the project is big, the number might be different for different parts.

But at that point, the project has a clear deprecation policy. A deprecation policy that can slowly grow the more mature the project is. Not a binary, true/false, mature/new. Shades of maturity. Levels of reliability.

The calendar

A minute has 60 seconds. An hour has 60 minutes. A day has 24 hours. Our time measurement system is still based on the Babylonian base-60 system, though the actual digits used by the Babylonians are studied only by specialists.

Humans organize their lives by their calendar. Kids learn that their birthday happens when they are a year older. Every seven days, we have a weekend. Every month, utility bills need to be paid.

Humans make plans that depend on time. They wait for their tax refund on April 15th to make purchases.

A time-based deprecation policy takes advantage of those skills. If the time between deprecation and breakage is one week, then the policy is clear: better make sure to upgrade weekly. If it is one year, do it when returning from the end-of-year holidays. If the policy is incompatible with the expected value of the maintenance effort, then this can be known in advance. This might mean that that dependency is not mature enough to be used.

Versioning for adults

A versioning scheme needs to remember two things:

  • The people writing the software are humans
  • The people using the software are humans

If there is one thing that humans are good at, it is communicating with other humans. Humans can communicate feelings, fuzzy boundaries, and plans for the future.

Calendar-based versioning, and a clear deprecation policy, give them the ability to communicate those. Not in a way that is suitable for computers. Not in a way that will help your dependency-resolver decide which version is "compatible". But in a way that lets you communicate with people about your needs in a mature way, and figuring out whether you can work together, or part on friendly terms.

Summary

If the main consumer of version numbers was the dependency resolver, and the producer of version number was a top-down military structure used to following orders, SemVer would work well.

For real software projects, used by humans, depending on documents written by humans for humans, and often "managed" in extremely loose ways, even for commercial projects, let alone volunteer-led ones, a versioning system that helps adults work with adults is best.

Credits


Universal Binary

Thu 20 August 2020 by Moshe Zadka

I have written before about my Inbox Zero methodology. This is still what I practice, but there is a lot more that helps me.

The concept behind "Universal Binary" is that the only numbers that make sense asymptotically are zero, one, and infinity. Therefore, in order to prevent things from …

read more

The Hardest Logic Puzzle Ever (In Python)

Fri 24 July 2020 by Moshe Zadka

The Labyrinth is a children’s movie. The main character is 16 years old, and solving a logic puzzle that will literally decide if she lives or dies. In fiction, characters are faced with realistic challenges: ones they can solve, even if they have to make an effort.

So, it …

read more

Hey, Back Off!

Sun 12 July 2020 by Moshe Zadka

The choice in parameters for back-off configuration is important. It can be the difference between a barely noticable blip in service quality and an hours-long site outage. In order to explore the consequences of the choice, I wrote a little fictional ditty about a fictional website.

I hope you enjoy …

read more

A Labyrinth of Lies

Fri 03 July 2020 by Moshe Zadka

In the 1986 movie Labyrinth, a young girl (played by Jennifer Connelly) is faced with a dilemma. The adorable Jim Henson puppets explain to her that one guard always lies, and one guard always tells the truth. She needs to figure out which door leads to the castle at the …

read more

Conditionally Logging Expensive Tasks

Sat 13 June 2020 by Moshe Zadka

(I have shown this technique in my mailing list. If this kind of thing seems interesting, why not subscribe?)

Imagine you want to log something that is, potentially, expensive to calculate. For example, in DEBUG mode, you would like to count the classes of the objects in gc.get_objects() and …

read more

My Little Pony -- DevOps is Magic

Sun 26 April 2020 by Moshe Zadka

(This article is based on the one I originally published on OpenSource.com.)

In 2010, the My Little Pony franchise was rebooted with the animated show My Little Pony: Friendship is Magic. The combination of accessibility to children with the sophisticated themes the show tackled garnered a following that cut …

read more

Numbers in Python

Sun 26 April 2020 by Moshe Zadka

Numbers in Python come in all shapes and forms. The reason different kind of representations of numbers exist is because they all have different trade-offs. These trade-offs are often surprising!

Integers

The most surprising things about integers is how easily they stop being integers. Dividing two integers, for example, 4 …

read more

Goodbye, John H. Conway

Tue 21 April 2020 by Moshe Zadka

John H. Conway passed away ten days ago, and I think it's only now I can write a proper eulogy.

I was first introduced to his work, if not his name, when I was at the end of elementary school. I am sure everyone has heard about the Game of …

read more

Using Twisted to Massively Parallelize Web Clients

Mon 13 April 2020 by Moshe Zadka

The Twisted Requests (treq) package is an HTTP client built on the popular Twisted library that is used for web requests. Async libraries offer the ability to do large amounts of network requests in parallel with relatively little CPU impact. This can be useful in HTTP clients that need to …

read more