French patent to M. Julien. by Robert Pittis Scott, Public Domain
Zen and the Art of Software Maintenance
Index
- An Inquiry into Values
- Gumptionology
- Out-of-Order Assemblies
- Forgetting something early in the re-assembly process, and having to take it apart again. Undoing potentially hours of work.
- Intermittent and Elusive Problems
- Problems that only show up sometimes, or only under very specific conditions. Such as only during rain, or when running the engine at a certain temperature.
- Parts or Material Failure
- While performing maintenance, having a part break unexpectedly or when replacing one part breaking another. Especially if the broken one is one that is hard to come by.
An Inquiry into Values
In 1974, Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig (1928 - 2017) was first
published after rejections from hundreds of publishing houses. Despite the initial set-back, the book
instantly became a best seller after a publisher dared to take a chance, selling over 50 000 copies in the
first three months.
The title is almost intentionally misleading, already within the first few pages the author explains that
the
book "should in no way be associated with that great body of factual information relating to orthodox Zen
Buddhist practice. It's not very factual on motorcycles, either."
The book is on the surface an autobiographical novel about a father traveling with his son on a motorcycle
through
the pacific northwest. The father and narrator tries his best to understand his son and mend their bond
which he can
feel
deteriorating. The narrator also recollects past events and educates us the readers via smaller,
philosophical
discussions intended to be easily digested.
These philosophical discussions, as well as the recollections, revolve around epistemology, history of
philosophy, and most importantly to us engineers, the philosophy of science and technology and concept of
Quality.1
But the subject I would like to highlight is one that concerns us the most as software maintainers, how we
interact with technology and how it affects our quality of life.
The topic we are going to discuss is brought up towards the end of Pirsig's journey with his son and spans
just about 20 or so pages. But
the advice and wisdom within these few pages could easily fill a 300-page of modern American self-help slop
speaking to just how densely Pirsig manages to pack wisdom in a book of 400 pages that also manages to
contain a story and whole system of metaphysics.
The subject matter in these twenty pages is "Gumption" and more specifically "gumption traps".
Gumptionology
Known categories of gumption traps
Gumption is an old Scottish word, roughly translating to motivation or enthusiasm.
As Pirsig puts it: "A person who is filled with gumption doesn't sit around dissipating and stewing
about
things. He's at the front of the train of his own awareness, watching to see what's up the track and
meeting
it when it comes."
I hope that I don't have to convince you that this isn't something that is esoteric. We've all
felt this feeling, this seemingly unlimited reservoir of motivation, your brain is working five steps ahead,
you are simply unstoppable. You might not have called it gumption, but that is exactly what Pirsig is
describing. When interacting with technology, whether it is repairing your motorcycle or debugging a flaky
test,
depending
on your level of gumption you will have more or less satisfaction during the process. It is also essential
to
performing work with Quality1 and avoiding mistakes.
Maintaining gumption can be seen as keeping a steady supply of "psychic gasoline", if you haven't got any
gasoline in your tank you can basically give up before you start since there is no chance that you are going
to get
anywhere anyways. However, if you do have it, and you know how to keep it, then there is no way that the
problem you have set out to solve isn't going to get solved.
I'd argue that maintaining a high level of gumption is what makes or breaks your workday. As such one should
avoid losing it, that is, avoiding stepping into what Pirsig calls "gumption traps".
Gumption Traps
Gumption traps are, as you might have guessed, things that drain enthusiasm and motivation, things
that
makes your day feel like it drags on and on, the things that makes you want to quit your job in tech and
move into the woods and never touch a computer again.
You are probably running into one or more gumption traps every day, examples of which will be listed,
but if you are unlucky, you might even have these built in your process at your company, effectively
causing you demotivation by (unintentional) design.
Pirsig goes to mention two major classes of gumption traps: Set-Backs and Hang-Ups.
Set-Backs
Set-Backs, also referred to as external traps, are gumption traps caused by external circumstances pushing back against your efforts. Traffic jams or slipped bike chains should give you the right idea of what a Set-Back is. Within the Set-Backs category we have several sub-categories, some of which software maintenance is more or less affected by.
As software maintainers we are spared from some of the external problems that might affect us, but only if
we take time to use high quality tools.
For instance, Out-of-Order Assemblies is a concern we in many instances can avoid by choosing
statically typed languages, which will throw errors and warnings if we would forget to fix a call site
when performing a refactor, and we can write quickly executable tests to check that we preserve the
behaviour throughout our re-assembly. And if all else fails, we have version control which in essence is
just time
travel we have invented for ourselves.
Of course Out-of-Order assemblies exists outside of the code base as well. Imagine if you have to, a
scenario where you built a whole feature and getting ready for release, but just as you are getting ready
your Product Person™, decides something should work slightly differently, but just different enough that it
cannot support the intended use case. This leaves you with two options, either rebuild it from scratch, or
make a low-quality hack, which Pirsig describes as "gumption desperation", which leads to more mistakes.
In general, avoiding the Out-of-Order assembly traps is about staying organized. Make sure you are
building the right thing by communicating with everyone involved, and doing your best to defer every
decision that can be deferred to the last possible moment. Leaving as many doors open as possible while
still being able to do deliver for as long as possible. You'd be surprised how long you can do this and how
many of those decisions never have to be made.
In contrast, Intermittent and Elusive problems are much more of a technical issue.
Having your system work sometimes is among the most frustrating parts of software maintenance.
Classic examples include, but are not limited to, race-conditions, flaky tests, and issues with
localization and time zones. According to Pirsig these first evolve into gumption traps when you thought
you had already fixed them, but they keep showing up again and again.
When diagnosing intermittent issues, it is of the utmost importance to correlate them to what other things
are
happening at the same time.
Are we taking backups at the same time as running scheduled jobs, are several users updating the same
entity at once? Are the customers affected by this issue always in Poland?
Dealing with intermittent issues can be difficult. Pirsig makes an important note that if you are the one
who
performs the maintenance and the one who is trying to diagnose the issue, you have the advantage of being
able
to replicate the issues yourself. If you were to leave it to a mechanic, you'd be no better off. Since the
mechanic would have much less context about the issue than you can possibly convey; they will more than
likely hand you back a still broken motorcycle that will have its issues resurface shortly again.
Similarly, we should try to run our own system ourselves as much as possible, and that means running in the
same environment or at least as close as possible to the environment that the intermittent issue appears in.
Run against real databases and emulate your cloud setup using MiniStack and Docker,
you might be able to find some of them when running both automated and manual testing locally before a real
user encounters the problem. But the most important part is being able to see and correlate correctly.
If you are not sure when or what is happening, you can forget about being able to figure out why.
So make sure your system is observable, using traces and logs and being able to aggregate them and make
sense of them.
If all else fails, and if you can tolerate it, then it might be that you might have to live with the
intermittent failure for a bit. To gather more information and correlate the failures as independent
incident
to get a bigger picture.
Parts or Material Failures reflect in the technology choices we make and in particular in
the vendors and dependencies we use. I think most software professionals have felt the recent degradation in
availability from provider such as GitHub and CloudFront, or the supply chain attacks in npm packages that
seem to show up every month; these are the parts in your motorcycle failing.
Getting new parts can be very expensive, or the replacements can be hard to come by,
or it might be that they are simply no longer produced.
Pirsig calls this type of set-back the most common, and one that one almost never prepares for. Outside of
paying the sometimes steep price of replacing the part, it also hard to ensure that the new part is
compatible and "fits" properly.
There is also the issue of having to communicate with different vendors, as some might be more cooperative
than others to help you be able to fit your needs and use cases.
There are two main ways of avoiding or reducing the impact of the materials failures according to Pirsig;
the
first is establishing a good relationship with the most cooperative "parts-man". The "parts-man" could be
everything from a maintainer of a package that you depend on or a cloud provider. In practice this could
mean establishing agreements, for instance in the form of SLAs with cloud provider or simply showing
interest in the package and sponsoring or in other ways contributing to its continued existence.
The other way is to simply start manufacturing your own parts, obviously this requires both time and money
to invest, and some parts you are probably never going to get as good as the ones you can get from
professionals. (Databases and compilers come to mind)
But if you have the time and money, the act of building your own parts is actually gumption building,
running software using more or less only your own code and being able to tune it to your exact needs gives
you a feeling that you can't get from anywhere else.
Hang-Ups
Hang-Ups in contrast to Set-Backs are internal gumption traps, that is to say that they
originate from inside yourself. They prevent us from performing quality work due to a poor fit between our
psychological state and the work we have set out to perform. Unlike the external gumption traps that we can
at least partly avoid by using abstractions, we cannot abstract away our own mental state.
Pirsig splits the Hang-Ups into three main categories, each with their own sub-categories.
These are the following:
Value Rigidity is according to Pirsig the most widespread and harmful of the bunch. If you are suffering the effects of value rigidity you are unable to revalue what you are seeing because you are completely committed to your current values. This means that you have to rediscover what you are doing as you are doing it. This might seem nonsensical, but consider the times you headed into a problem completely convinced of what you are supposed to be doing and what you're supposed to change, but once you apply the change it doesn't actually fix the problem. Unless you are aware of this value trap you are very likely to keep banging your head, which might eventually evolve into desperation, and if you are not careful you can make grave mistakes. The only way to avoid this trap is to give up your current opinions, and slow down, its not like you have choice anyways. But the slow down has to be deliberate and you should go over and re-visit where you have tread before re-evaluate what you thought was valuable and important, and just "stare at the machine". Often you are not that wrong of what the failure is, you know your system after all. You will often find that you are close, and the issue has a "friend" that is exactly what you are looking for. Rubber ducking2 is a great way to walk through and re discover what you are valuing, and helps you to level out your biases which is suppressing real signal of what is really important.
Example of value fluidity.
Ego is closely related to value rigidity, and is many times the cause of it. When you have an inflated ego the facts you are discovering as your maintaining your codebase are saying that you are doing it wrong and can be practically screaming in your face to stop, but you are unlikely to believe any of it, and on the flip side when false information is making you look good you are likely to believe it. If you are constantly trying to defend your ego you are going to suffer longer and harder from value rigidity than one who lets go of his ego. To get around this, even if you have a huge ego, you can simply act and pretend to yourself to be more modest according to Pirsig. Because you are eventually going to get humbled anyways, you might as well get a head start. "If you just deliberately assume you're not much good, then your gumption gets a boost when the facts prove this assumption is correct. This way you can keep going until the time comes when the facts prove this assumption is incorrect."
On the flip side of ego comes the trap of Anxiety. Anxiety in contrast to ego actually prevents you from doing anything useful because you are afraid of making it worse. This usually manifests as "pussyfooting" around the issue, inventing reasons for why you can't get started yet, finding any excuse not to get to heart of the issue. You might even start to fix things that aren't broken, living in the "What-if?" instead of objective reality. This eventually becomes a self fulfilling prophecy, as the amount of work you do to circumvent the actual issue starts failing, you confirm your own doubts on your competency, and now you are deeper in the hole. According to Pirsig there are two ways out of the trap of anxiety; the first is a concrete action. If you are feeling like you're out on the deep end simply read and familiarise yourself as best as you can with the task at hand. Consult books, videos, AI, or other people to build confidence and try to relate what you are doing to things you are already familiar with. If you are convinced that this is productive anxiety actually makes this step easy since it will calm you down the more you read and understand. The second way is to realise that every one makes mistakes, to err is human after all, and that is part of you learning. In some way you should be grateful for the opportunity to try and learn, every good engineer you've ever met has messed up on more than one occasion. That's just life as a software maintainer.
Boredom occurs when you have lost your beginners mind, when every project or feature feels like you
done it before, when everything feels like just another CRUD app.
Pirsig gives a stern warning about boredom, boredom is usually what happens before a big mistake. When you
are bored you are completely off the tracks when it comes to performing good maintenance, you stop doing
things with Quality in mind, take shortcuts, leave a mess that will eventually will come back to haunt you,
sooner or later!
When bored, it is best to just stop what you are doing for a while, have a nap, make some coffee, listen to
the birds sing, touch some grass. Simply do anything for a little while but the task at hand that is making
gumption leak out like a deflating balloon. If that doesn't work, it might mean that you are distracted,
because you know that there is more important work to be performed, if that is really the case, do those
instead, you are already subconsciously focusing on them anyways and they're disturbing your piece of mind.
If you find yourself being bored with tasks that have become a regular occurrence, Pirsig recommends turning
them into a sort of ritual. There is beauty to be found in doing new adventurous things as well as
activities
that have been done hundred times before and are familiar. See it as re-acquainting yourself with the
familiar, maybe you'll find something that is different and sticks out.
Impatience has it roots in inaccurate estimation of the time the task you have to do will take. Set-Backs are all but a certainty, and being under pressure while dealing with one is a recipe for anger and disaster. Ideally we would have an indefinite time to perform any task, but living in the world we do deadlines are a harsh reality, so if you can't get indefinite try at least for double. Try for a life with "Broad Margins".3 Dealing with impatience typically includes scaling down your overall goals in terms of importance and focusing on your immediate tasks, this might feel sad and underwhelming especially if the right thing to do is a boring task, but in the long run it will pay off in doing things right from the get-go compared to having to face a set-back down the line.
Truth Traps
Truth traps are concerned with the data you have gathered and the questions you ask about it. Most of the time, classical analysis using the scientific method is sufficient. But there are instances where it is not, such as the Yes-No truth trap.The Yes-No truth trap is one you step into when you are asking a question about your system and expecting a Yes-No answer, but the results are inconclusive. We are usually very hung up on the idea that things should be able to be answered or modeled in terms of Yes-No, true or false, 1 or 0. But not all questions are suited to answer. You might be able to relate quite well to this, how many times have you been asked a Yes-No question, but giving the classic "It depends." as an answer? You might've felt like you are antagonizing the inquirer, but you know that it is simply not the correct or honest answer to classify the question into two simple states. What you've actually done is much closer to what a Buddha would answer in a similar situation, you are answering using a third logical answer, which will instead expand our understanding in a unexpected directions. There is no such term in most of western tradition, instead we have to look east, in this case we use the Japanese term 無 ("mu"). Pirsig translates 無 to roughly meaning "No class", that is not Yes or No, not one not zero. It simply means that the context that the question is asked in too small for the available classes for either of them not to be erroneous. It asks instead for us to "Unask the question". When performing experiments and the answer is inconclusive it can be tempting to get frustrated and blame your stupidity for designing such an awful experiment, but getting a 無 answer can be the most useful and most enlightening answer you could have asked for, it means that your question is to small for the world to answer, and you must extend the context, zoom out if you will, and see that there are more hidden variables that you have to take into account before the world can give a satisfy your inquiry. These are more vital, this where you grow your understanding!
Psychomotor Traps
The final category of Hang-Ups are the Psychomotor traps, they are the traps that affect your ability to
perform actions while maintaining the system with accuracy and comfort.
Psychomotor Traps manifest as frustration with the tools and the environment you are currently situated in.
Starting with tools, these are the tools that you actually physically interact, analogous to the to the
mechanic's spanners and screwdrivers. As software maintainers these are both digital and physical,
e.g. your keyboard, screen, or text editor. Make sure to buy high quality and reliable tools that will
help in many situations, whether you are SSH:ed into your server or running locally on your machine.
Pirsig highlights avoiding cheap tools, especially when they can break at inopportune moments, since usually
you never plan on buying new tools in the middle of your maintenance work, having your IDE crash or your
keyboard stop registering your inputs can quickly make you feel like walking through mud.
Prefer instead tools that have stood the test of time, they are still in use for good reasons and there is
usually a vibrant community and ecosystem surrounding them.
After tools the working environment is one of the most common sources for psychomotor traps. Having bad
lighting
or noise disturb you while working can really throw off your momentum, and can cause you to make more
mistakes.
If your able remove the cause for the disturbances, pull the curtains, close the door (if you are lucky
enough
to have an office) before getting to work, it will make the work both easier and more enjoyable which will
lead
to a better result.
Conclusion
Even if you are aware of all the traps outlined, dodging set-backs, and being aware of your most common hang-ups, will everything be smooth sailing and will you be able to live a life with peace-of-mind from now on? Hopefully you understand at this point that this not the case, if you want to write perfect software you yourself have to become perfect, and that is a life long journey which requires constant upkeep and most, if anyone, will ever reach so a point. We are human after all. The gumption traps described by Pirsig is not the be-all and end-all, they are as he says "shortcuts to living right".What you are working on is not your motorcycle or your codebase, what you are working on is yourself.
References
1. Pirsig's Metaphysics of Quality https://en.wikipedia.org/wiki/Pirsig's_Metaphysics_of_Quality
2. Rubber duck debugging https://en.wikipedia.org/wiki/Rubber_duck_debugging
3. Walden; or Life in the Woods, Sounds (1854). Henry David Thoreau