Popular products often have a "Getting Started" tutorial in order to guide you in using them for the first timees. Those guides are easy to follow, pleasant to use, and lead you do make bad design choices.
Follow the "Getting Started" guide to get a feel for the product. Then, skip to the end: in the "Advanced" section, you will find how the people who really use it in production use it.
The Physics of "Getting Started"
What do flask.Blueprint, requests.Session, and ansible.plugins have in common?
They all share two curious properties:
- In the official documentation, they are relegated to the "Advanced" section.
- Any production use case of the relevant technologies (flask, requests, and ansible) would use them.
All of these technologies are popular for their niche. All of them have friendly "Getting started" guides.
Those guides do not cover the relevant piece, the piece that makes this technology usable in most realistic cases. Most people who use these technology start with "Getting Started", and follow the path it suggests. The path it suggests eventually breaks down.
When this path breaks down, often people will think to themselves "I don't need the advanced stuff. My use case is pretty vanilla." They will shim and work around the limitations, until either giving up on the whole enterprise, or asking an expert.
At the point where they asked an expert, the expert will be exasperated. "Just use flask.Blueprint/requests.Session/ansible.plugins for anything beyond a toy example. This is the way to really use the technology."
The expert is exasperated because this is the fifth (or the hundred and fifth!) time this has come. Someone is about to give up in disgust, only to learn that the solution was simple all along: the first three paragraphs in the "Advanced" section cover how to do this.
Why does this happen?
I chose these three examples because I know them well. I can give similar examples from almost every popular technology: following the "Quickstart" guide leads to a huge mess, while a fairly simple way to do it in a better way is presented in a section titled "Advanced" or "Developer" or something similar.
These things happen so often that this cannot be a coincidence. Just like the common shape of whales and fish, despite separate evolutionary origins, the explanation is the shape of the dynamic systems at play.
Whales and fish look similar because they both have to solve a similar problem: moving through water efficiently. Having an appropriate aquadynamic shape is the right solution.
Technologies, open source or commercial, live and die by their adoption. A technology that is hard to adopt is a useless relic.
There are many things that can impact the ease of adoption of a technology. One of the easiest levers to pull, by the core development team, is the "Quickstart" guide. This guide tries to get a toy example up and running as fast as possible.
At this point, any extra complexity is a big turn off. Any extra step will have a large fraction of the users dropping off. This exponential decay means that to survive, the Quickstart guide needs to be heavily optimized.
It does not matter whether it's "design" (developers explicitly optimizing the guide) or "evolution" (technologies which don't do that do not end up surviving long enough to cause problems). It is probably a combination of both factors.
As can be predicted from general optimization problems, you can optimize more when you drop constraints. Removing the constraint "must lead to a proper production solution" from the requirements for the Quickstart guide leads to a guide more optimized to win hearts and minds.
At the point the project is popular, this will end up being the shape of the documentation:
- A Quickstart guide, generating a toy example with a serious scaling flaw.
- An "Advanced" section with the explanation of how to really use the technology.
What can you do?
Being upset with Quickstart guides, as I have for many years, is like being upset with gravity, which I also have for many years. Those phenomena are annyoing, but they are the results of immutable laws of nature.
In order to fly, or use technologies, you need to first understand the physics involved and then figure out how to use them to accomplish your goals. Now that the shape is understandable, how can it make adopting technologies easier?
The quickstart guide is still useful. Follow it. Generate a toy example. Play around with it. Get a feel for the technology. See if it's at all a good fit.
Put all of these attempts in a source-controlled repository. Now move all of the code written so far into a subdirectory, experiments. Go to the advanced section, and figure out what are the 1-3 things that you need to do in order to properly use the technology. Alternatively, consult someone with experience.
Use the notes from the experiments, plus the understanding from the "advanced" section, to build your first "real" proof of concept. From this point on, this is the "toy example" you iterate on.
The experiments serve as research notes for how to do the specific things you needed. Never use the experiments directly.
Good technologies will gently nudge you to using them badly. This is not good or bad: this is a law of nature. Understand it, and start using platforms better!
Thanks to Alex Scammon for his feedback on an earlier draft of this post. Any mistakes that remain are my responsibility.
Three Problems with a Tool
Better Outage Retrospectives
Originally published on Enable Architect.
Modern computer systems supply business-critical services everywhere -- from Amazon providing shopping services to Healthcare.gov providing enrollment in health insurance plan. We all rely on such systems. But, unfortunately, these systems are complex and can fail in surprising ways.
By now, it is a well-understood …read more
Empathy vs. sympathy for Site Reliability Engineers (SRE)
This article was originally published on Enable Architect
Many people have had the insight that DevOps is about people. Often, they will summarize it as "DevOps is about empathy". I have found, however, that idealizing empathy is just as bad as thinking that DevOps is about a single technology.
I …read more
Minimal packing list
With in-person conferences starting to open up, I need to clear the dust off of some skills that have not been used in a while. One of those is how to pack for travel.
This list works for me. It will probably not work for you as-is. Among other things …read more
Post that PR
Sometimes you will be working on hairy and complicated feature in a shared repository. Maybe it's for work. Maybe it's an open source project.
As a responsible person, you are working on a branch. The usual way of working involves a lot of "intermediate" check-ins. Those serve, if nothing else …read more
Portable Python Binary Wheels
It is possible to work with Python quite a bit and not be aware of some of the subtler details of package management. Since Python is a popular “glue” language, one of its core strengths is integrating with libraries written in other languages: from database drivers written in C, numerical …read more
So you want to create a universe
A story about looking for a universe, and finding a pi(e)
This is fine. You need not feel shame. Many want to create a universe. But it is good you are being careful. A universe with sentient beings is a big moral responsibility.
It is good to start with …read more
Virtual Buffet Line
Many people have written about the logistical challenges of food in a conference. You trade off not just, as Chris points out, expensive food versus terrible food, but also the challenges of serving the food to everyone at once.
One natural method of crowd control is the buffet line. People …read more
DRY is a Trade-Off
DRY, or Don't Repeat Yourself is frequently touted as a principle of software development. "Copy-pasta" is the derisive term applied to a violation of it, tying together the concept of copying code and pasta as description of software development bad practices (see also spaghetti code).
It is so uniformly reviled …read more