At a previous company I know about through the industry grapevine, a team of five engineers spent three months migrating a perfectly functional monolith to a Kubernetes-orchestrated microservices architecture. The product had a few hundred active users. The infrastructure team consisted of one junior DevOps engineer who had learned Kubernetes on the job during the migration.
The migration succeeded technically. The system was deployed. Six weeks later, a certificate rotation issue brought down production for four hours. Nobody on the team understood the failure mode. The incident response involved frantic Googling of Kubernetes certificate management documentation. By the time it was resolved, the company had spent significant engineering time on a problem that would not have existed on the previous system.
This is hype-driven development. It's more common than the industry acknowledges. And its costs are real.
The Hype Cycle in Software Engineering
Technology hype is not new. The enthusiasm for microservices in 2016, containers in 2015, blockchain in 2017, serverless in 2019, Kubernetes in 2020, and now AI for everything — each wave has produced legitimate adoption where the problems matched the technology, and cargo-cult adoption where they didn't.
The cargo-cult pattern follows a predictable path:
- A technology solves a real problem at significant scale (Netflix decomposing a monolith, Spotify building event streaming infrastructure, Airbnb containerising for deployment consistency)
- Conference talks disseminate the pattern to the broader engineering community
- Blogs and YouTube tutorials make the implementation accessible
- Engineering teams adopt the pattern because it's what "modern" systems use
- Teams operate the pattern at 1% of the scale that justified it, paying full operational cost
The signal/noise problem is that step 1 is real. Netflix's microservices architecture makes sense for Netflix. Spotify's event streaming infrastructure makes sense for Spotify. The lesson that gets lost in the hype cycle is that these architectures were adopted to solve specific problems at specific scale — not because they're universally superior.
Resume-Driven Development
There's an uncomfortable truth behind some technology adoption decisions: they're driven by what engineers want on their resumes, not by what the product needs.
Engineers who want to deepen their Kubernetes expertise will find reasons to introduce Kubernetes. Engineers who are excited about a new database technology will find problems it solves. This isn't malicious — it's entirely human. Learning valuable skills is a legitimate professional interest.
The problem is when individual career interests shape technology decisions that affect an entire team's operational burden and a company's infrastructure costs. A technology you learned on the company's time, on the company's production system, that you then leave with — while the maintenance cost stays behind.
I'm not immune to this. Early in my career I introduced abstractions and patterns that I found intellectually interesting without adequately weighing the cost of those choices on the team. I've been on the receiving end of the same — inheriting systems built by engineers who had moved on, where the architecture served no business purpose I could identify beyond "the person who built it found this interesting."
The discipline is: your technology decisions belong to the team and the product, not to you. The right question is always "is this the right choice for this system?" not "is this interesting to me?" Those questions sometimes have the same answer. They shouldn't be confused.
The Operational Cost Nobody Prices In
Every technology decision carries operational cost that is systematically underestimated at adoption time. The reasons are structural: the engineers making the adoption decision are excited about the technology's capabilities, not its failure modes. The operational burden belongs to future-them, who is an abstraction.
Let me make this concrete. Adopting Kafka for an event streaming requirement is a legitimate choice in many contexts — I wrote about the cases where Kafka is genuinely the right tool in my RabbitMQ vs Kafka comparison. But Kafka requires:
- Zookeeper or KRaft for cluster coordination
- Appropriate partition configuration for your throughput
- Consumer group management and rebalancing strategies
- Schema registry if you care about schema evolution
- Monitoring for consumer lag, partition health, broker performance
- Runbooks for the failure modes: broker failure, consumer group stalls, topic compaction issues
- Someone on the team who understands all of the above at 3am when production is down
That operational bill comes due regardless of whether your system is processing 10,000 events per day or 10 million. The question is whether your scale justifies the investment. For many teams, it doesn't — and a simple database-backed queue, or even RabbitMQ, would have handled the actual load with a fraction of the operational complexity.
The framework I use when evaluating a technology adoption:
| Consideration | Questions to Answer |
|---|---|
| Problem fit | Does this technology solve a problem we actually have? |
| Team capability | Can we operate this in production, including during incidents? |
| Operational overhead | What does this cost to run, monitor, and maintain? |
| Hiring implications | Does this narrow our hiring pool significantly? |
| Vendor/lock-in risk | How difficult is it to replace this if needed? |
| Adoption cost | What's the realistic time to become proficient operators? |
| Exit cost | If we need to migrate away, how expensive is it? |
Running this analysis before adoption — rather than after enthusiasm has already committed the team — changes decisions.
The Database Pluralism Trap
One of the most common overreaches in modern backend architecture is adopting multiple specialized databases without business justification for each.
The pattern: a team adds a relational database for transactional data, a document database for flexible schema requirements, a search database for full-text search, a time-series database for metrics, a graph database for relationship queries, and a cache layer. Six data storage systems. Six different operational models. Six potential sources of data consistency issues. Six different backup, restoration, and disaster recovery procedures.
This architecture makes sense for companies with significant scale in each data access pattern and teams dedicated to operating each system. For most teams, it's an operational nightmare in pursuit of theoretical optimality.
PostgreSQL handles relational data, JSON documents, full-text search (with pg_trgm or tsvector), and time-series queries reasonably well. Redis handles caching, session storage, and pub/sub. That's two systems — two operational models, two backup strategies, two sets of failure modes to understand. That's a manageable operational surface.
The right question isn't "what's the best database for this access pattern in isolation?" It's "what's the best database for this access pattern given our team's operational capacity, the rest of our stack, and our actual scale requirements?"
The specialized database earns its place when you've hit a concrete limit: PostgreSQL full-text search isn't fast enough for your search volume, and you need Elasticsearch. Your relational queries have become unmaintainably complex because the data is genuinely graph-shaped, and you need Neo4j. You have time-series data at a scale that PostgreSQL can't handle, and you need InfluxDB.
Not before. Not because Elasticsearch is a respectable technology. Because you need it.
Frontend Complexity Has the Same Problem
Hype-driven technology decisions aren't only a backend problem. Frontend stacks have an equally strong tendency toward unjustified complexity.
The classic case: a marketing website, with mostly static content, five pages, and minimal user interaction, built with a full React application featuring client-side routing, a state management library, a component library, a CSS-in-JS solution, and TypeScript — because that's the "modern" way to build web applications.
The result: a 300KB JavaScript bundle that must be parsed and executed before the user sees content, a first contentful paint three times slower than an equivalent server-rendered page, an SEO challenge from client-rendered HTML, and a development environment that takes sixty seconds to start.
The same website as a server-rendered HTML page, with CSS, and minimal JavaScript for interactive elements: instantaneous first paint, excellent SEO, maintainable by any engineer who understands HTML and CSS, deployed as static files with no Node.js runtime.
The React application is not wrong. It's wrong for this problem. A rich single-page application with complex state, real-time updates, and extensive user interactions is a legitimate use case for React. A marketing website is not.
The right tool for the problem. Always that question.
The Next Engineer Test
One of the most useful filters I apply to technology decisions is what I call the "next engineer test": if I left the company tomorrow, could the next engineer hired for this role operate this system?
Not an engineer with your specific expertise. An engineer of the general caliber you'd expect for this role, with two to five years of experience, who is competent but not specialised in every technology in your stack.
If the honest answer is no — if the system requires deep expertise in a technology that's not broadly understood, without documentation, runbooks, or a team with the expertise to support the next engineer — that's a signal that the technology choice has created a single point of human failure.
This test has stopped me from adopting technologies that were genuinely interesting but would have been unmaintainable for anyone but me. It's made me more thoughtful about documentation. It's also made me realize that the boring technologies — PostgreSQL, Redis, NestJS, standard deployment patterns — are boring because they're so well understood, documented, and supported that almost any competent engineer can operate them.
Boring is underrated.
The Right Criteria
After years of watching technology decisions go well and badly, the criteria that correlate with good outcomes are consistent:
Good technology decisions are:
- Made after a clear problem definition
- Evaluated against concrete alternatives
- Assessed for operational cost, not just capability
- Made by people who will operate the technology in production
- Documented with the reasoning, so future maintainers understand why
- Revisited when the original conditions change
Bad technology decisions are:
- Driven by enthusiasm, trend-chasing, or resume concerns
- Made by people who won't maintain the system
- Undocumented, leaving future engineers to reverse-engineer the intent
- Made based on what successful companies use, not what you need
- Irreversible because the adoption happened too deep, too fast
A Note on Bangladesh Context
Working as an engineer in Bangladesh has sharpened my perspective on this. Operational costs matter more when infrastructure budget is constrained. Running six database systems in production on a startup's budget, in a region where cloud costs are felt differently, isn't a theoretical concern — it's a real constraint that forces clarity about what you actually need.
Engineers who've operated systems in resource-constrained environments develop a healthy skepticism about complexity for its own sake. You become very good at asking "do we actually need this?" when the cost of the answer is concrete. That instinct — the pragmatic engineer's instinct — is one I try to carry into every architecture discussion regardless of the budget available.
The right technology isn't the one generating the most excitement on social media. It's the one that solves the problem with the lowest total cost of ownership, operated by a team that understands it well enough to maintain it at 3am. That's the bar.
Key Takeaways
Every technology choice is a long-term commitment that extends far beyond the initial implementation. The engineers excited to adopt it may not be there for the third year of maintenance, the hiring challenge it creates, or the production incident that reveals its failure modes.
Choose technologies for the right reasons: they solve a real problem you have, your team can operate them, and the total ownership cost is justified by the value they provide. Be skeptical of choices driven by excitement, trend-following, or individual career interests.
The best engineering teams I've seen are intentionally boring in their infrastructure choices. Their systems work. Their on-call is manageable. Their new engineers ramp up quickly. Their deploys are predictable.
That's what good technology selection looks like. Not impressive architecture diagrams. Not cutting-edge stacks. Just systems that work, operated by people who understand them.
Comments
No comments yet — be the first!