As Marc Andreessen pointed out six years ago, software has eaten the world. Many large companies from yesterday, once the flagships of their industries, have sunk into irrelevance. The corporate captains of today lose sleep because they know the sharks of Silicon Valley are circling. If you peer under the hood of these companies, you will find them heavily investing in technology. Now that everyone has software projects underway, the winners will be decided by who executes on them.
I will step through the key points to successfully executing on software projects, exactly what most companies get wrong:
Who is writing your software is most important. Looking back at past failures and successes, I have noticed two common themes. Software failures are written by engineers who are attempting to solve business problems they have been presented with. Software successes are created by business-minded individuals who have the passion and ability for solving problems through technology.
What do most companies do? They huddle up, create a plan for how they are going to make an amazing software solution, and then look for some nerdy types to code it up. They might hire a translator, also known as a project manager, because their engineers appear to speak a different language. This may actually be the case, because many businesses decide to outsource development after experiencing US engineer sticker shock. The general course of action is to bring in the engineering team after the entire solution has been planned out.
But software isn’t like a sandwich. You can’t hand over a recipe and expect something scrumptious to be slid back across the counter. If you have tried this approach, you will know that you usually get served something very different from what you ordered.
The reality is throughout the development process there are many decisions made while programming that have the potential to cause unwanted business outcomes. If the engineer has a blurry vision of the goal, these decisions will accumulate and amount to a poor product-market-fit. The result is months of wasted time and money. Using the Agile methodology is a good starting point, but it is not a cure-all solution. There is no substitute for engineers who are active participants in business-oriented discussions.
How do you find these business-minded types of engineers? Stop putting “10 years of professional development experience + computer science degree” on job descriptions. Another example of the type of engineer I am referring to is a tech entrepreneur. They enjoy programming, especially when they use it as a tool to build business value. Not every entrepreneur is looking to quit his day-job, source venture capital, and strike it out on his own. There are a lot of entrepreneurs who work on their own projects after hours.
These types of individuals will not settle for being task runners. They want to understand the why behind what they are building. In addition, they also introduce a valuable feedback loop into your project because they incorporate learnings from their own ventures into your business.
When it comes to filling out an entire team of these types of individuals, you might get pushback from your recruiters. Good engineers are hard to find, and they usually command higher prices. Modifying job descriptions and looking in non-traditional places can help with the first concern. The second is cancelled out by hiring fewer engineers.
Remind your recruiters that small teams of exceptional engineers can out produce larger teams of mediocre engineers. Countless studies and articles have provided evidence of this.
However, what is the status quo? Quantity over quality. The problem is that budget engineers tend to get less done and leave behind a trail of complexity. The problems that arise tend to deceptively look like they are due to understaffing, so even more engineers end up being brought on.
An odd phenomenon occurs. As the team’s size increases, its collective output decreases. However, in other professions this formula works: if you need to landscape more acres, you hire more landscapers. Why does this same method not apply to software? Due to the complexity of software projects, constant communication between team members is required and quickly becomes the bottleneck. As the pressure of approaching deadlines starts to be felt, developers start to act upon assumptions. These assumptions start accumulating, and once again, the trajectory of the project starts to diverge from its initial goals.
So now that you have your small group of exceptional business-minded engineers, how do you ensure incentives are aligned? Imagine engineer-business-involvement as a continuum that extends from completely uninvolved on the left to completely involved on the right. On the left, you might envision business rules being dropped on an engineer’s desk. The opposite extreme, on the right is embodied in startup founders who are also developers.
Larger companies tend to fall on the left side of the scale. However, startups tend to be toward the right. Why? Because startups are forced to build what users want. Every employee knows that if they don’t get this right they won’t have a job because the company will be gone. In a startup, business outcomes and engineer’s incentives are almost perfectly aligned.
While it might not be possible to shift a large company completely to the right, they can benefit from moving in that direction. The key is to remove the strict delineation between business and technical outcomes. In order for this to be done, engineers need to be brought in as active participants in key business meetings. If you notice the vocabulary shifting towards business results, not software deliverables, you are on the right track.
You might be wondering how anything gets accomplished by developers who are constantly interrupted by meetings? The answer lies in reconsidering whether or not you will permit meetings to be scheduled sporadically throughout the day.
Paul Graham, the founder of Y Combinator, wrote a compelling article on time management for technical teams. His article emphasizes the importance of dedicating large blocks of uninterrupted time for your team to work on their primary tasks. The impact of this practice on productivity is clear to anyone who has come into the office early in the morning or stayed after everyone else has left.
The negative effect of consistent meeting interruptions is particularly pronounced for software teams. The nature of software development requires engineers to hold multiple concepts in their heads at the same time. It can take a long time for a developer to get back to the same level of productivity after being derailed by a context switch. If you imagine one’s state of mind as a cup filled with concepts relevant to the task at hand, an interruption is equivalent to pouring out the contents. The cup will need to be refilled before any meaningful work can start again.
If you are trying to implement this same kind of blocked schedule at your company, you will likely run into a good amount of pushback. Try proposing this new schedule as an experiment. Go all-in for a month and see what kind of changes in productivity come out of it.
To recap, the formula for executing successful software projects is as follows:
While some of these points may seem common sense, they are far from commonplace.
 The Agile methodology encourages regular check ins from the business team to demo functionality built in each sprint (every week or so). However, new software projects tend to have many pieces that need to fall into place before any demo-able product exists. Couple this with human nature, not wanting to show a half-baked solution to the business stakeholders, and you wind up with the same misalignment of outcomes from expectations.
 One example: http://www.qsm.com/risk_02.html