What Software Developers Can Learn From Big Infrastructure Projects

I recently developed an interest in learning the stories behind the creation of some of the world’s largest infrastructure projects. I blame this on an incredible and highly recommended podcast called The Big Dig by WGBH media, which explores Boston’s Big Dig project from idea to completion. This led me to read some of Bent Flyvbjerg’s research about understanding the successes and failures around specific megaprojects. When digging into the stories of iconic projects like the Øresund Fixed Link, Sydney Opera House, and the Big Dig, I started to notice some common threads that eventually led to some of the high-profile issues like being behind schedule and over budget which made these famous projects a bit infamous.

Over the years, I’ve seen the same type of issues begin as the root of some major problems on a handful of projects that I’ve worked on as both a contributor and a leader. Modern agile processes and ceremonies were developed to handle the risk and uncertainty in software projects, but despite our best efforts to control risk and embrace change, there are some issues that can still threaten the success of large software projects.

This blog post isn’t going to talk about processes and following steps A-Z for a successful project. I wanted to focus on the more human and social aspects of both running and contributing to projects, and how some common human tendencies can set a project up for major trouble or even complete failure.

Pre-Project Optimism

When reading through some of these notable large infrastructure project retrospectives, I could see that every project started with loads of optimism or rosy outlooks on both project progress and how beneficial these projects would be to the populations that would serve them. One underlying commonality between a lot of the large and ambitious infrastructure projects was that there was pressure at all levels to choose the happiest path when in the designing and planning phases. This seemed to be because of a fear that a project might not get a green light if an accurate picture of the true project cost and time to completion were known.

Having worked on many software projects over the years, I’ve also encountered this optimism and pressure. This type of pressure sometimes could be felt coming from leadership or the team, but most of the time it came from me. When you’re excited about building a new thing, or taking on a new challenge, you may not want to discover something that could prevent the start of a project or cause some of the more exciting technologies to not be included. In our desire to work on a cool new project or see the completed version of a grand vision, we have a tendency to ignore or overlook things that might prevent the project from moving forward. This can result in not having a realistic picture of the true scope of a large endeavor.

Poor Communication During the Project

Another interesting piece of data that came from learning about high profile infrastructure projects was a reluctance of one or many people to acknowledge and communicate when things were starting to go wrong. This often resulted in small problems snowballing into much larger ones, and jeopardizing the success of the project.

In some cases, a small, covered up issue would result in a cascade of failures for multiple systems built on top of it. This is something that also often happens in software projects. Small problems get covered up, or they never get resolved. When these problems eventually turn into a failure, it can be catastrophic to the system, and be very difficult to repair. Testing early and often is a fundamental tenet of modern software development, but sometimes we forget to apply that concept to areas outside of the code.

Other important areas like technical feasibility, costs and staffing also need to be analyzed early and often to make sure that issues can be caught and resolved as soon as possible.

Doing What’s Never Been Done Is Rarely Easy

Completing a project on schedule and budget is exceedingly rare in cases where something truly groundbreaking is being built or a new type of technology is being used for the first time. This is somewhat related to too much optimism. We tend not to be open about risk beyond what has been calculated in a technical risk assessment. All stakeholders need to have a realistic expectation of the risks, and everyone needs to understand how to move forward when obstacles present themselves.

Once again, this is where expectations need to be set early on. Ambitious projects rarely get it done right on the first try. They also rarely ever achieve their goals cheaply. Everyone contributing to a project, and especially stakeholders, need to understand the very real challenges involved with groundbreaking development, and that there’s no guarantee that the project will be successful. This isn’t an excuse for not following modern software development methodologies and best practices, but just setting realistic expectations for all involved.

Key Takeaways

There are many common themes between the issues and challenges that can occur on large infrastructure and large software projects. While software development has come a long way and has created really well-refined processes, there’s still a lot to be learned from other industries. This is especially true for challenges that can’t easily be solved by just dropping it into an existing methodology. Whether you’re starting a brand new project, or just looking to improve the potential success of your current project, here are some things to keep thinking about:

• Be real with yourself, and encourage others on your team to do the same. If you think something likely won’t work, or can’t be done, make sure you speak up. If building a new application or adding a feature doesn’t make sense, help to make sure your team can openly talk about that before starting the project.
• When things go wrong and have the potential to jeopardize the project, make sure that the team knows and understands the implications. Covered up problems always come back to haunt you.
• If a project is groundbreaking, experimental, very large, or all three, make sure that stakeholders understand not just the quantifiable risk, but that there is unknown risk in every step of the process. Not all of the potential risks can be discovered until they’re impacting your work.

Being mindful of these dynamics, and understanding how they may impact your contributions, will help keep you moving forward and hopefully drive your project to successful completion.