[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"glossary-sharding::en":3,"gloss-cluster-sharding::en":20,"gloss-next-sharding::en":9},{"slug":4,"category":5,"name":6,"definition":7,"meta_desc":8,"faq":9,"schema_markup":9,"related":10},"sharding","data-infra","Sharding","Sharding is a database scaling technique that splits a large dataset horizontally across multiple servers (\"shards\"), where each shard holds a subset of the total rows rather than every server holding a full copy. It is the primary strategy for scaling write throughput and total data volume beyond what a single database server can handle, once vertical scaling (buying a bigger server) hits diminishing returns or a hard ceiling. Why it matters for AI\u002FSaaS builders: most SaaS products never need sharding — a well-indexed Postgres instance handles surprisingly large workloads — but AI products that ingest high-volume data (embeddings for millions of documents across thousands of tenants, event logs from every AI interaction, vector indexes that must stay in memory) hit the ceiling faster than typical CRUD apps, because vector and event data both tend to grow much faster than traditional relational data. Understanding sharding matters even for teams who aren't there yet, because the sharding key chosen early (often, in multi-tenant SaaS, the customer\u002Ftenant ID) has to be baked into the data model from day one — retrofitting it onto an unsharded schema years later is a brutal migration. How it works: a shard key (or partition key) determines which shard a given row lives on — commonly a hash of a customer ID, a user ID, or a geographic region. Range-based sharding puts contiguous ranges of the key on each shard (simple, but risks hot spots if activity clusters in one range); hash-based sharding distributes keys more evenly across shards but makes range queries (\"all records from March\") harder since they now span every shard. Distributed vector databases (Pinecone, Qdrant, Milvus) shard automatically under the hood as an index grows, splitting the vector space across nodes so queries fan out to relevant shards in parallel rather than one node bearing the whole index. The trade-off sharding always introduces: cross-shard queries (joins or aggregations spanning multiple shards) become significantly more complex and slower than they were on a single unsharded database, so schema and query design should minimize how often cross-shard operations are needed. Worked example: a multi-tenant AI analytics SaaS shards its Postgres event-logging database by `customer_id % 16`, giving 16 shards. A large enterprise customer with 500M events\u002Fmonth never causes the whole system to slow down for other customers, because their events physically live on one or two shards, not competing for the same disk I\u002FO and connection pool as everyone else's data — while the application's shard-router layer transparently directs each query to the correct shard based on the `customer_id` in the request.","Sharding splits a large database horizontally across multiple servers, distributing data so no single machine holds or serves the entire dataset.",null,[11,14,17],{"slug":12,"name":13},"data-warehouse","Data Warehouse",{"slug":15,"name":16},"postgresql","PostgreSQL",{"slug":18,"name":19},"replication","Replication",[21,25,28,31,34,38,41,44,47,50,54,57],{"slug":22,"category":5,"name":23,"updated_at":24},"acid","ACID","2026-08-24T02:46:37+00:00",{"slug":26,"category":5,"name":27,"updated_at":24},"ann-search","ANN Search",{"slug":29,"category":5,"name":30,"updated_at":24},"backpressure","Backpressure",{"slug":32,"category":5,"name":33,"updated_at":24},"batch-processing","Batch Processing",{"slug":35,"category":5,"name":36,"updated_at":37},"bm25","BM25","2026-08-24T02:46:38+00:00",{"slug":39,"category":5,"name":40,"updated_at":24},"cache","Cache",{"slug":42,"category":5,"name":43,"updated_at":24},"cap-theorem","CAP Theorem",{"slug":45,"category":5,"name":46,"updated_at":24},"change-data-capture","Change Data Capture (CDC)",{"slug":48,"category":5,"name":49,"updated_at":24},"chroma","Chroma",{"slug":51,"category":5,"name":52,"updated_at":53},"chunk-overlap","Chunk Overlap","2026-08-24T03:30:02+00:00",{"slug":55,"category":5,"name":56,"updated_at":24},"columnar-storage","Columnar Storage",{"slug":58,"category":5,"name":59,"updated_at":24},"connection-pooling","Connection Pooling"]