How the Not-Invented-Here Syndrome Manifests in the Age of Package Managers
Every feature, an ocean.
Investment capital, the processing power in our pockets, and a few generations of people steeped in technology have all made this software development thing more complicated. Even the dusty corners of large applications have hefty requirements, from accessibility and performance to responsiveness and delight.
Despite a ramp up in complexity, many teams adopt the same starting mindset: We’ll knock this feature out ourselves. This known psychological bias is usually referred to as not-invented-here syndrome.
Labeled or not, every long-term software project makes decisions with this not-invented-here bias in mind. Joel Spolsky’s classic article details a few benefits of keeping features in-house, especially as they relate to the Excel team at Microsoft.
At the same time, hefty feature requirements and the number of companies investing in developers has driven the open-source and software-as-a-service ecosystems forward. Like flat-packed furniture, these bits of prepackaged code may not be heirloom quality, but they’re usable in a hurry.
Taking advantage of third-party code has become easier than ever with the rise of package managers (NPM, Yarn, Hex, and RubyGems, to name a few). Through their automated caretaking of dependency versions and environments (and their sub-dependencies), it’s easy to forget that work from outside your organization is being relied upon.
So, with a feature request queued up, what’s the right approach?
Don’t “Roll Your Own”
It seems simple enough: your developers are great, so you’ll take advantage of in-house talent to build the feature! Not much to argue against there — until we start to list out to-dos:
- Build a team. Prioritize this effort and necessary funds over other potential tasks.
- Create a scope. What are the requirements for the first release and metrics for considering this effort complete? Who owns approval?
- Start the long-term roadmap. Since it’s an internal tool, how much time and money will be set aside for future changes and improvements?
- Build a maintenance team. You likely don’t want your specialized A-team focusing on this forever, so you’ll also need documentation for hand-offs as team members and objectives change.
That’s the short version, but it’s still reminiscent of If You Give a Mouse a Cookie. By opting to own the code, you also own the burden of planning and maintenance. For organizations with tapped development resources or no desire to strategically plan the future of a feature that isn’t critical, that responsibility is a nonstarter.
Instead, Use Open-Source Software
As it turns out, other people (and companies) also have some pretty impressive talent, and their open-source contributions amount to a bunch of ready-to-go code.
Take data visualization. If your web application needs a few interactive graphs, handling the trigonometry and SVG output is onerous; joining the masses and using D3.js is a good move. (Or one of the many libraries built on top of it, like Vega, Vega-Lite, and nivo.)
Crowdsourced bug reports, testing, security checks, and quality assurance are far more thorough than anything cooked up in-house. A high contributor count also increases the likelihood of (current) documentation, release notes, and a focus on regular updates. Often, an open-source library will even come with functionality beyond the internally-scoped need.
Open-source software also brings a pair of ease-of-use benefits along for the ride:
- New developers may already have familiarity via other projects. It’s easier to find a React developer than train a new hire on your in-house framework.
- Installation, version control, and sub-dependency management are all streamlined through the use of package managers. Long-term software project maintenance boils down to keeping these libraries upgraded.
Wait, Maybe Not Open Source Software
How confident are you in trusting critical business needs to volunteers? It’s not the norm, but even popular open-source libraries are routinely abandoned or fall off the pace of rapidly changing web standards.
Short of being a primary contributor, your company’s lack of a controlling financial stake is also a gamble. The software can change direction, goals, or features without room to protest.
You’re also beholden to dangers like:
- Direction by committee (or the full community).
- The coding standards of outside developers.
- Outdated, incomplete, or incorrect documentation.
- License choice. Can this be used commercially? Is attribution required?
There’s no shortage of horror stories in the past few years on the open-source + package management front: a copyright claim, a dependency of a dependency having malicious code, and even discord at the parent company.
Instead, Use Software as a Service
If you’re trying to integrate custom maps to your software, there’s a good chance you’ll become a customer of Mapbox. Some things, like satellite tiles and vector mapping, are best left to a third-party provider.
Once you’ve committed to an outside company, they have a financial incentive to keep you satisfied with their service. In trying to stay out in front of open-source options, they often offer a killer feature, pristine documentation, or readily available support. In certain situations, your patronage may even entitle you to input on the future or custom development.
With a company-backed service, many potential open-source shortcomings become entitlements: regular updates, roadmaps, code quality, and communication are assumed to be part of your subscription fee.
Wait, Maybe Not Software as a Service
Nothing underlines the danger of third-party service reliance quite like Killed by Google.
Even if the dependency is an industry leader, there’s nothing guaranteeing it’ll survive long term. Without a look at their books, you’ll never know if they’re stable or a startup floating on venture capital and an unsustainable business model. Acquisitions, acqui-hires, and pivots — these services are often backed by young companies still finding their footing, and there’s danger in how easily digital products can be transferred or turned off.
If the service sticks around, you’re still reliant on an outside provider for a critical business need. What sort of price increase will you tolerate? Two costs loom in a replacement effort (pulling it out and creating a new one), and that gives the service a ton of room to adjust cost without losing customers.
Roll Your Own Software Features
Control, direction, cost — myriad drawbacks await on the open-source and software as a service side of the fence, so we really should create everything ourselves. The benefits speak for themselves:
- Uptime? You control where it’s hosted and any failover systems.
- Priorities and roadmap? Whatever you want, paired with your product’s future.
- License? By you, for you. You can even explore releasing it ourselves as open source for recognition.
- Maintenance and upgrades? As soon as you need them.
By creating and maintaining the code you need, all of the necessary features live on for as long as they’re required. Seems like a no-brainer.
We Just Rolled Some Circular Logic
Those playing along at home probably noticed the loop back to the beginning. Like everything else in application development, the right path is it depends. Accepting that reality comes with an understanding of hidden costs in custom software builds.
There’s no perfect answer: Most projects make use of all-of-the-above.
Determining What to Build
Not-invented-here isn’t the enemy it’s often made out to be, but it also shouldn’t be the driver for your dependency approach. Software features are a true balancing act, hinging on everything from:
- Future plans
- Developer availability
- Future plans
And probably future plans, too. From inception to maintenance, every addition to your codebase will need to be weighed and measured.
The best development partners? Strong communicators. Because you need effective communication when it comes to technical hurdles and production delays.