Unknown Risks: Planning for the Inevitable in Software Development

Software projects never go exactly as planned. No matter how meticulously we plan, scope, and estimate, we must always be ready for a wrench to be thrown in the works. Requirements will change, technical limitations will be discovered, and bugs will occur. This isn’t the exception, it’s an expected byproduct of software development. As we strive to deliver a project on time and on budget, it’s crucial to have a process driven by strong communication to handle any issues that may arise.

Estimating Time is Difficult

Before we delve into specific communication practices, we must acknowledge a fundamental reason effective communication is so critical: 

Estimating time is one of the most difficult aspects of software development.

In general, we’re good at estimating how long things will take that have known risks. Take installing a refrigerator, for instance. It’s a task that’s been repeated millions of times, and typically plays out the same way each time. Maybe a water line needs to be replaced, or perhaps there’s a defective part—but we’re not going to encounter an issue that we haven’t seen before. Likewise, we have a pretty good idea about how long it takes to install Postgres, or validate the strength of a password.

However, we’re not installing a refrigerator, we’re inventing one. We’re experts in the tools required to build one. We’ve likely built many of its individual components in past projects, but we’ve never built this specific appliance before. Although many software projects can be distilled into elements that, by themselves, have known risks, we can’t foresee every possible edge case that could emerge from the unique way all of these elements will interact to solve a client’s specific problem. We’re inventing something new and, as such, there are unknown risks. This idea of unknown risk forms the foundation of our communication practices. 

Communicate Progress in Tiers

When you map out the features of a software project, there’s a hierarchical structure to it. So, we should convey our progress using that same structure—a flat list of GitHub issues simply will not do. We need to effectively communicate progress at every tier of our project’s hierarchy—from the bird’s-eye view to the individual task.

While the terminology may vary depending on the project management methodology or software you use, generally, at the top-most level you have milestones. You can think of milestones as checkpoints on your project’s schedule that typically map to specific releases (e.g. Public Beta). Milestones are comprised of features (sometimes called “epics”). This might be our user registration page, for example. Next, we have stories, which are the requirements that make up a feature. A story for our registration page might be “As a visitor, I can create an account using an email and password”. And at the lowest level, we have the individual tasks needed to complete story (e.g. “validate password strength”). While this approach may seem like common sense, it’s important to acknowledge why we do it. By organizing our development process into tiers, we’re able to clearly communicate the progress of the project at every level, which enables the client to make informed decisions on their direction and priorities. 

Let’s look at the unfortunate, yet not uncommon scenario of a project running behind schedule. From the bird’s-eye view, the client can see that our Private Beta milestone is in danger of being missed. Digging deeper, they can see that the culprit is our registration page feature, which has one uncompleted story: the ability to login via Facebook. Based on the number of tasks completed for that story, the client can determine how far away this story—and ultimately, this milestone—are from completion. They might decide that enough of the major use-cases for our registration page have been addressed to push this specific story to a later milestone and just go ahead and release. By using this tiered approach to communicating progress, we give the client the flexibility to make these kinds of decisions.

Separate Developer and Client Language

Most projects utilize some form of issue tracking for handling development tasks (e.g. GitHub Issues). While it may seem convenient to have developers and clients all sharing the same issue tracker (hey, it streamlines the process—the client reports the bug directly to the developers), we end up muddying our communication while simultaneously making it easier to creep out of scope or budget.

Developer issues commonly use a different language from client issues. What amounts to a simple “button doesn’t work” issue to the client might actually spawn multiple developer issues that go too deep into the weeds to be useful to the client. As a result, the progress of an issue isn’t as clear as it should be. So, it’s generally a good idea to have a separate place where clients can convey any thoughts or issues. These are then triaged into actionable development tickets. Additionally, having this separation makes it easier to determine whether or not a client’s request is in scope and budget, as we have now adopted a process that makes this a deliberate, conscious step.

Release Often

While we try our best during the discovery process to flesh out exactly what the client wants. Oftentimes it’s not until they’ve had the opportunity to see it live that they realize what they actually want is something different. Acknowledging this reality by opting for smaller, more frequent releases to a staging environment shortens that feedback loop and allows us to adapt to changing requirements without having to redo a bunch of work.

Continuous Integration and Deployment tools (such as Travis CI) make it easy to keep our staging environment up to date—if the build passes, it will automatically deploy. By utilizing such a system, the client is always able to test the latest version of the application. For more complicated projects with multiple features being developed in parallel, solutions such as Heroku Pipelines allow us to deploy multiple staging sites. We can have one for each branch that’s ready to be reviewed by the client. This alleviates the issue of being bottlenecked by a single staging environment.

It’s important to note that with a frequently updated staging environment, we need to stay on top of keeping the client aware of what’s available to test. Most popular project management tools have some sort of release mechanism baked in, but for smaller projects, sending the client a simple list of release notes can be just as effective.

Limit Crunch Time

Lastly, no matter how well we plan, we somehow always find ourselves in periods of crunch time. Heads are down, productivity is high, but communication tends to slow, necessitating extra effort from project management to enforce regular check-ins. It’s important to limit these periods as much as possible because they place a shroud over the development process. Although developers are hard at work, their progress isn’t as clear to the client. Extended periods of crunch time reduce the flexibility to adjust priorities and handle any issues that may arise.

—

While we’ve covered some general actionable steps one can take, there is no “one size fits all” approach to effectively communicating progress. Ultimately, the communication framework you use will be guided by the needs of the client and project. What’s important is that you use one.