Computers work on binary code. If statements take one path: true, or false. For computers, bright lines and clear borders make sense.
Humans are more complicated. What's an adult? When are you happy? How mature are you? Humans have fuzzy feelings with no clear delination.
I was more responsible as a ten year old than as a three year old. At 13, I reached the age when I was responsible for following Jewish law myself. At 18, the legal system trusted me to could drink alcohol and drive, and trusted me that I will keep the two activities distinct. In the US, you cannot become a senator before you are 30.
At what age are you responsible "enough"?
Software is written by humans, not computers. Humans with feelings, hopes, and dreams. We cry, we strive, we feel accomplished at times, and disappointed at others.
If you were designing a version system for computers, SemVer, or "Semantic Versioning", would make perfect sense. Each part number in a three-part version number is given a specific, distinct, definition:
- Increment MAJOR for backwards incompatible changes.
- Increment MINOR for changes which add functionality.
- Increment PATH for bugfix-only backwards compatible release.
But software is not made by computers. It is made by humans.
A journey of a thousand miles begins with a single step. The first version of the Linux kernel printed As and Bs to the screen. The first version of Python didn't have modules. SemVer, to its credit, acknowledges that.
In versions like
SemVer defines the semantics:
Anything MAY change at any time. The public API SHOULD NOT be considered stable.
When something is small and fragile, it should be able to change.
Every UNIX programmer knows the story of why
treats tabs and spaces differently.
In retrospect, causing a dozen of people a small amount of pain
would probably have been better than staying with the problem.
Once the software is mature enough,
the SemVer reasoning goes,
and commit to API stability.
Given the amount of projects that have stayed on ZeroVer for a long time (or forever!) the assumption that commiting to API stability once the project matures is easy seems to not pan out.
Remember: software is written by humans. Fuzzy humans, in complicated social structures, who work together as best they can, using brains evolved to figure out politics in a fifty-person tribe in order to stay alive.
Once a social structure is in place and working, changing it is hard. In the ZeroVer days, there was no reason to figure out which changes broke API compatibility. There was no reason to clearly delinate which parts are "public" API and which are not. After all, there was no need.
Switching out of ZeroVer requires building all of this. Not switching out of ZeroVer does not require complicated social engineering. It is not surprising that it is hard. It's almost as if humans work better with slow changes, and not sudden revolutions.
Lately, I have been frustrated with some aspects of my life. COVID-19 did not cause them, but helped bring them into sharp focus. As the least embarassing example to admit in a public forum, I realized that while my book shelves are so full of books that shoving another one requires my 80s-kids Tetris skills, I have not read a single fiction book in the last three years. I used to be a voracious reader!
How do you change habits? I used to read, easily, 200 pages of fiction a day in my 20s. I have not gotten worse at reading. I could commit to reading 200 pages a day, and track my progress. If you have ever done that, you know what the outcome is. Every day, you look at the task, and you decide it is too big. You never begin.
Instead, I decided I will read 20 pages a day, and feel good about it. Feel good? I even decided to reward myself for every week where I hit this goal five out seven days.
The result? The last few weeks, I have been consistently been reading 20 pages a day, missing only one or two days.
When you are not good at something, as a person or as a group, and you want to get better, small commitments frequently achieved are the way to go.
SemVer does not work that way. It is all or nothing. SemWay or the Highway. Perhaps it is better to have a versioning system for humans, not a fictional alien race, if we assume software will keep being written by humans.
It is an easy change to say that no single change can "just" break an API. One change to deprecate, and one change to break. This is straightforward to verify. It is reasonable to have a policy for exceptions, but document the exceptions carefully.
Note that this change does not help potential users all that much by itself. After all, two PRs in close succession can land, and there is no reasonable upgrade path.
Should we feel good about making a small change that does not help anyone? Absolutely. Because it is small, and it is on the right path.
Now that this change becomes ingrained in the developer group, we can start mandating a minimum time between deprecation and breakage. At first, we can have a 0day policy: you can break, as long as the deprecated software has been released. This causes more releases to happen, making the team better at releasing. It helps users only minimally. However, at least with careful version pinning, there is an upgrade path.
Now, we can start making the number 0 a bit bigger. First, a week. Then, a month. Eventually, a quarter or a year. If the project is big, the number might be different for different parts.
But at that point, the project has a clear deprecation policy. A deprecation policy that can slowly grow the more mature the project is. Not a binary, true/false, mature/new. Shades of maturity. Levels of reliability.
A minute has 60 seconds. An hour has 60 minutes. A day has 24 hours. Our time measurement system is still based on the Babylonian base-60 system, though the actual digits used by the Babylonians are studied only by specialists.
Humans organize their lives by their calendar. Kids learn that their birthday happens when they are a year older. Every seven days, we have a weekend. Every month, utility bills need to be paid.
Humans make plans that depend on time. They wait for their tax refund on April 15th to make purchases.
A time-based deprecation policy takes advantage of those skills. If the time between deprecation and breakage is one week, then the policy is clear: better make sure to upgrade weekly. If it is one year, do it when returning from the end-of-year holidays. If the policy is incompatible with the expected value of the maintenance effort, then this can be known in advance. This might mean that that dependency is not mature enough to be used.
Versioning for adults
A versioning scheme needs to remember two things:
- The people writing the software are humans
- The people using the software are humans
If there is one thing that humans are good at, it is communicating with other humans. Humans can communicate feelings, fuzzy boundaries, and plans for the future.
Calendar-based versioning, and a clear deprecation policy, give them the ability to communicate those. Not in a way that is suitable for computers. Not in a way that will help your dependency-resolver decide which version is "compatible". But in a way that lets you communicate with people about your needs in a mature way, and figuring out whether you can work together, or part on friendly terms.
If the main consumer of version numbers was the dependency resolver, and the producer of version number was a top-down military structure used to following orders, SemVer would work well.
For real software projects, used by humans, depending on documents written by humans for humans, and often "managed" in extremely loose ways, even for commercial projects, let alone volunteer-led ones, a versioning system that helps adults work with adults is best.
- Thanks to Amber "Hawkowl" Brown for her inspiration in among other things, her excellent talk, Releasing Calendar-Versioned Software.
- Thanks to Mahmoud Hashemi for his work on CalVer, ZeroVer, and blog post about CalVer as well as design notes.
- Thanks to Glyph Lefkowitz for insights into how software development happens.
- Thanks to Jay Looney and Paul Nathan for feedback on earlier drafts of this article. Any issues or errors that remain are my responsibility.
I have written before about my Inbox Zero methodology. This is still what I practice, but there is a lot more that helps me.
The concept behind "Universal Binary" is that the only numbers that make sense asymptotically are zero, one, and infinity. Therefore, in order to prevent things from …read more
The Hardest Logic Puzzle Ever (In Python)
Hey, Back Off!
The choice in parameters for back-off configuration is important. It can be the difference between a barely noticable blip in service quality and an hours-long site outage. In order to explore the consequences of the choice, I wrote a little fictional ditty about a fictional website.
I hope you enjoy …read more
A Labyrinth of Lies
In the 1986 movie Labyrinth, a young girl (played by Jennifer Connelly) is faced with a dilemma. The adorable Jim Henson puppets explain to her that one guard always lies, and one guard always tells the truth. She needs to figure out which door leads to the castle at the …read more
Conditionally Logging Expensive Tasks
(I have shown this technique in my mailing list. If this kind of thing seems interesting, why not subscribe?)
Imagine you want to log something that is,
expensive to calculate.
in DEBUG mode,
you would like to count the classes of the objects in
My Little Pony -- DevOps is Magic
(This article is based on the one I originally published on OpenSource.com.)
In 2010, the My Little Pony franchise was rebooted with the animated show My Little Pony: Friendship is Magic. The combination of accessibility to children with the sophisticated themes the show tackled garnered a following that cut …read more
Numbers in Python
Numbers in Python come in all shapes and forms. The reason different kind of representations of numbers exist is because they all have different trade-offs. These trade-offs are often surprising!
The most surprising things about integers is how easily they stop being
Dividing two integers, for example,
Goodbye, John H. Conway
John H. Conway passed away ten days ago, and I think it's only now I can write a proper eulogy.
I was first introduced to his work, if not his name, when I was at the end of elementary school. I am sure everyone has heard about the Game of …read more
Using Twisted to Massively Parallelize Web Clients
The Twisted Requests (treq) package is an HTTP client built on the popular Twisted library that is used for web requests. Async libraries offer the ability to do large amounts of network requests in parallel with relatively little CPU impact. This can be useful in HTTP clients that need to …read more