Budget Creep: How Change Impacts Custom Software Development Cost
Software development remains an expansive, experimental, new… and immature field. That newness flies in the face of the (occasionally overwhelming) complexity behind what it builds. A quick search on just about any topic within software development is a deep, dark pit of competing opinions that change on a regular basis.
This newness is apparent all the way down to a lack of consensus on core decisions — like which HTML tag to use in a given situation. Except for the list of deprecated elements. We’re pretty sure it’s safe to count those out. (Maybe.)
Consequently, we use a lot of construction analogies. Houses seem to make more sense conceptually.
There are plenty of predictable costs in building or remodeling a home: materials, permits, equipment, and labor follow a plan and can be inferred from past structures. Even something as unpredictable as the weather can be reasonably modeled based on data. Surprise takes a few forms, but we’ll focus in on change.
Industry change, regulatory change, change of plans; all alter underlying expectations of cost.
Let’s stick with the weather variable. As a Florida native, I had a few family members stick it out through Hurricane Andrew back in 1992. Taking steps to prepare a construction site during hurricane season: relatively predictable. Changes to the building code post-Andrew, including hurricane straps and impact-resistant glass: surprise increases. And that’s just one-time costs, not factoring in the sharp uptick of yearly insurance prices.
Quotes in software development follow the same pattern. Requirements, deployment, team members — everything predictable follows a plan and can be inferred from past builds. In spite of optimizing process around this (agile, sprint, stand-up, scrum, and all the other lingo project teams use), custom applications continue to surprise and blow past estimates.
Diagnosing today’s problems helps avoid them tomorrow, so let’s take a look at the shifting winds of software expectations.
The Faces of Software Change
In the world of software development, change assumes three forms:
- Refinement of the industry. Given software’s importance in virtually every sector and the capital involved, there’s an incentive to move the needle toward maturity. Predictability in approach, lower failure rates, the whole nine. This takes shape as an implementation concern.
- Regulation catching up with technology. Law has notoriously lagged behind the speed of innovation, especially when it comes to privacy, consumer protection, and accessibility. As these rules come into effect (worldwide), new complexities are introduced on the implementation side.
- Refinement of the plan. No matter how meticulous, no set of software requirements survives from estimate to launch. This takes shape as a process concern.
The Pace of Implementation Quality
A side effect of the march toward maturity is the parallel march towards complexity. As the quality bar rises for anything user-facing, the specializations and requirements behind it have to keep improving their pole vaulting skills.
Webmaster was a popular title when I first dipped my toe in the pool of web development. Webmasters would handle everything: design, markup, styles, functionality, servers, and even IT in a pinch. It made a lot of sense when Space Jam was the peak we all aspired to climb.
Design and development saw the first big split as browsers improved to support things closer resembling “art.” CSS’s quirkiness helped shoehorn a front-end developer position in there soon after.
These days, every role is trying its hardest to hit an inflection point and split, like some kind of specialization mitosis. Yesterday’s web designer may now be a product designer, interface designer, experience designer, illustrator, animator, or some other subspecialty. The same is true for any other software-related title.
This loops into cost on the back of expectation. What used to pass for a project team may not carry the collective expertise to match the quality of competing companies. Everyone wants to be the next Slack, but Slack’s level is set by 1500 employees and contracts with a number of talented external firms.
Today’s good is tomorrow’s outdated, and what passes today in implementation won’t fly on the next project — each new specialty adds a decision (do we need it?) and an onboarding cost.
Performance, Responsiveness, Accessibility
Make things fast, make them work on all the devices you can, and make sure they’re accessible. The first two affect SEO and conversions; the last one has laws and fines attached.
It doesn’t take much evangelizing anymore to make these points a priority, but they’re all on a sliding scale. How accessible is accessible enough? How much has that changed in the last year? Does that require an additional team member with the requisite specialization?
In a field littered with moving implementation targets, these performance, responsiveness, and accessibility factors have seen the most change.
GDPR and Data Security
Like that building code discussion earlier, laws and regulations around the world create new project requirements. Billions of dollars have been spent on GDPR compliance efforts — when’s the last time you visited a site, no matter how niche, that didn’t have that cookie notification?
As laws catch up to the whole “move fast” thing the tech industry has been chanting for a decade, it’s inevitable that more line items will pop up on every estimate.
Keeping the Bigger Project Picture in Scope
Process problems cascade into the future (and not in the useful CSS way). It’s an expensive butterfly effect — seemingly innocuous decisions today can lead to flaming piles of money later.
Sprinting Until You’re Tired
Seeing a result to end a weekly or bi-weekly sprint is a huge improvement over a flippant “we’re still working on it” status update. It’s the best part of the agile movement.
Time and again, though, we see teams focus so intensely on the short-term cadence at the expense of the big picture. Several sprints and a few direction changes later, progress has been made — but not the right progress.
HIPAA compliance, internationalization support, event stores. The project roadmap may not need certain data governance bits now, but backfilling architecture in a future phase is substantially more difficult than starting with it in place.
It’s a hard call, but there are foundational decisions worth making early. Heated floors are a tougher ask when they require ripping out and replacing perfectly good tile.
Coming back to software requirements not surviving to launch, no interface survives first contact with users and real data. Wireframes, prototypes, designs — process can help get you to the right neighborhood, but polish and refinement of the underlying ideas is a necessary line item.
The sunk cost fallacy rears its head here often. It’s hard to let features go or simplify functionality that doesn’t test well after putting development time in.
Testing and Quality Assurance
No automated tests, a big change requested… and a 1–20 hour estimate.
Opting out of proper code testing and data sanitization only reaps benefits in the extreme short-term. Soon, every change will become a nightmare to find, fix, and verify. The larger the platform, the more painful the debt that accompanies wading into the unknown.
Embracing Change with Custom Software
Custom software is an exercise in change management. Change isn’t a bad thing, and it’s unavoidable — it’s that whole surprise thing that creates trouble. As entropy in the form of specializations, tools, and overall expectations increases, the underlying effort will always carry a danger of surprise.
Planning for hidden costs comes in two parts: finding the gas lines under the landscaping and assuming they’re there in the first place. Early detection, early prevention, and even avoidance altogether is possible, but it takes experience and expertise.