The Relationship Between Knowledge and Planning in Software Development

I've always enjoyed the parable style, so that's what I'll use here. The story
is true, with names changed and details omitted to protect the innocent.

Ford and Arthur

Ford and Arthur had worked together for a while. One day they decided to work together on a program that would ease a developer pain point. More important than the specifics of the project, is that they decided to write this tool in a language they had minimal experience in, with new technologies.

When they started the project, they split up the tasks into the job queue and the frontend. Ford took on the frontend, and Arthur started on the job queue.

After a few weeks, they were close to ready, and slated to present this tool in two days. The interface was ready to go. The only remaining part was to finish up the job queue, which had proved difficult. They paired on the job queue for an hour, but couldn't quite get it to work that session.

Looking at it after, Ford started the system from scratch. After some tinkering, he had a working system by the end of the day.

So what did Ford do differently? Why did Arthur take two weeks to get an almost-working system, and Ford got it working in a day? To answer this, I think it's best to talk briefly about how experienced software engineers generally work, and when that breaks down.

When a new feature request came in for his project, Arthur would look at the requirements, and a plan would materialize. He knew he'd have to create some new components, write some utility functions, hit some new endpoints, etc. Once he started programming, most of the thinking had already been doneā€”the implementation was straightforward. This is how most experienced software engineers work, particularly specialists.

So when the task was to create a job queue with a new technology, Arthur naturally followed the same route that had worked for him in the past. Once he had read the documentation for the library, he thought through how he'd structure the system, and started coding.

Ford took a different approach. He ran through tutorials by copying code, then tweaked the code and saw whether the result matched his expectation. He tried to piece the things he was learning together, taking a bit from one code example and trying to fit it in with another. When he wasn't sure why something was written a certain way, he'd write it another way, and see why it errored out.

After this experimentation, when he started on the real task, it took Ford less than an hour to glue together the pieces to create a basic job queue.

The principle

The lesson this story illustrates, and that I've taken from my own experience is this: The less you know about the things you're working on, the less you should plan.

This scales all the way to no knowledge: when you know nothing, plan nothing.

When you start out programming, you naturally experiment more. There's really no alternative. You can't plan out the structure of your new website when you're still learning what a <body> tag is and what should go in it. You'd waste most of your time on scrapped plans.

This changes once you gain more knowledge. Ex. once you know Django well, you don't need to experiment with how to structure a certain relationship, or how to add an auth system; you know enough to plan out the pieces.

Arthur, being a specialist, never had the thought to not plan out his approach. It wasn't so much a decision, as it was the automatic response of an experienced engineer. It's this seemingly innocuous response to a problem that gets developers in trouble, because why wouldn't you think through the problem first? It's counter-intuitive to most that reach a certain level of competency in software.

So when you're doing something new, experiment first. Don't start on the real code until you understand the basics. The experimentation phase won't feel like progress, and constantly encountering errors won't feel good. But between the programmer that experiments, and the programmer that plans a course through unknown territory, my money is on the former. It could be the difference between a painful two weeks or a day of work.

Of course, once you do know what you're doing, you should absolutely be planning. But this, I think, is a fairly agreed-upon point and not much more need be said about it.

Show Comments