Scaling Software: From 100 Users to 1 Million
Every great product starts small — a handful of users, a single server, and a bold idea. But as your user base grows from a hundred to a million and beyond, the architecture that got you here won't get you there. This presentation charts the engineering journey of a scaling system, stage by stage, decision by decision.
The Prototype Phase: 100 Users
At this stage, the goal is not scale. The goal is learning. Architecture should maximize speed of iteration and minimize operational complexity.
The Startup Runway
Monolithic Architecture
- Single deployment unit
- Shared memory architecture
- Simple local development
- Minimal infrastructure overhead
Distributed Complexity
- Microservices
- Service Meshes
- Complex Queues
- Premature Sharding
Why the Monolith Wins
At 10,000 users, the cracks in a monolith begin to show. This is the phase for targeted, high-leverage interventions that relieve the specific bottlenecks causing pain.
Don’t rewrite the architecture first. Add the reliever that maps to the bottleneck: replicas for read pressure, load balancing for request distribution, and a CDN for static delivery.
Removing Friction: 10,000 Users
Practical Scaling Rule
Reaching one million users is a fundamental inflection point. The strategies that carried you to 10,000 users — read replicas, load balancers, a CDN — begin to buckle under true scale. This is where centralized simplicity must evolve into deliberate distribution.
The monolith decomposes into independently deployable services — auth, payments, notifications, search. Teams deploy, scale, and fail independently. A spike in search traffic no longer threatens checkout. Use Domain-Driven Design (DDD) to identify service boundaries, starting with high-load services like auth and media.
Horizontal partitioning distributes data across nodes by shard key (user ID, region). Each shard holds a subset of rows, reducing per-node load. Complexity arises with cross-shard queries and rebalancing. Managed solutions like Vitess (YouTube) or CockroachDB help avoid building custom sharding layers.
Asynchronous queues (Kafka, AWS SQS, RabbitMQ) decouple background tasks from the request cycle. Email delivery, report generation, image resizing, and fraud analysis move off the critical path. Queues provide natural backpressure — buffering safely instead of crashing when consumers lag.
Migration is rarely clean-cut. Most teams run hybrids for 12–24 months, incrementally extracting services while keeping the core intact. The strangler fig pattern routes new traffic to new services while legacy code is progressively replaced — balancing risk and resilience at scale.
The Million-User Shift: Embracing Distribution
Microservices Migration
Database Sharding
Message Queues
Evolution Path
Single deployable unit
Decomposed services + queues
Sharded data & async consumers
At global scale, architecture is no longer just about handling traffic. It becomes a discipline of latency reduction, fault tolerance, observability, and worldwide user experience.
Multi-region active-active deployments eliminate geographic bottlenecks and regional single points of failure.
Metrics, logs, traces, and intelligent alerting become essential for diagnosing issues across distributed systems.
Each workload must balance consistency, availability, and partition tolerance according to business risk.
Social feeds can tolerate eventual consistency. Financial balances often cannot. The correct consistency model depends on the business impact of stale data.
Success at global scale depends on active-active infrastructure, world-class observability, intelligent failure handling, and carefully chosen consistency guarantees. The system must continue operating smoothly even when entire regions, services, or networks fail.
Engineering for Global Reach: 10M+ Users
Global Operations Command Center
ACTIVEGlobal Infrastructure
Observability
Consistency
Observability Stack
Every Global System Must Choose Trade-offs
At 10M+ Users, Reliability Becomes a Product Feature
Scaling is not a destination — it is a discipline. The teams that build systems that endure are the ones that make simplicity, measurement, and deliberate evolution part of the culture.
The architecture that got you here won’t get you there — and that is not a failure. It is the system working exactly as intended, evolving under real-world pressure.
The Architecture of Tomorrow
Final Principle
What's Your Reaction?
