[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"glossary-data-lake::en":3,"gloss-cluster-data-lake::en":20,"gloss-next-data-lake::en":9},{"slug":4,"category":5,"name":6,"definition":7,"meta_desc":8,"faq":9,"schema_markup":9,"related":10},"data-lake","data-infra","Data Lake","A data lake is a centralized storage repository that holds raw data in its native format — structured (database exports), semi-structured (JSON, CSV logs), and unstructured (images, PDFs, audio, raw text) — at massive scale and low cost, without requiring the data to be structured into a predefined schema before it's stored. This is the key distinction from a data warehouse: a warehouse enforces \"schema-on-write\" (data must be cleaned and structured before it goes in), while a data lake practices \"schema-on-read\" (data goes in as-is; structure and meaning are applied later, at query or processing time, by whatever tool reads it). Why it matters for AI\u002FSaaS builders: data lakes have become especially relevant in the AI era because so much of the raw material for AI features — documents, chat transcripts, uploaded files, model outputs, eval logs — doesn't fit neatly into relational tables, and forcing a rigid schema onto it before storage would mean deciding upfront exactly how it'll be used, which is often unknown when the data first lands. Storing it raw in a data lake first (commonly just object storage like S3 with a defined folder\u002Fpartition convention) preserves optionality: a new AI feature six months later can reprocess the same raw historical data in a way nobody anticipated when it was first collected. How it works: data lakes are typically built directly on object storage (S3, Google Cloud Storage, Azure Blob Storage) using open file formats (Parquet, Avro, JSON) organized into a partitioning scheme (e.g., `s3:\u002F\u002Flake\u002Fevents\u002Fyear=2026\u002Fmonth=07\u002Fday=02\u002F`) that lets query engines scan only relevant partitions instead of the whole dataset. Query engines like Apache Spark, Presto\u002FTrino, or AWS Athena can run SQL directly against files sitting in the lake without a separate load step, blurring the line between a \"data lake\" and a \"data warehouse\" — a hybrid pattern often called a \"lakehouse\" (popularized by Databricks) that adds warehouse-like transactional guarantees and schema enforcement on top of lake-style cheap, flexible raw storage. Worked example: an AI SaaS logs every raw LLM request\u002Fresponse pair (prompt, model, tokens, latency, output) as JSON files into an S3-based data lake, partitioned by date, long before deciding exactly what analysis they'll want to run on it. Months later, when the team wants to build a prompt-regression eval system comparing model behavior across versions, they run Athena queries directly against the historical raw JSON in the lake — no ETL pipeline had to be pre-built for a use case nobody had defined yet at collection time.","A data lake stores raw data of any format — structured, semi-structured, unstructured — at scale and cheaply, deferring schema decisions until read time.",null,[11,14,17],{"slug":12,"name":13},"data-pipeline","Data Pipeline",{"slug":15,"name":16},"data-warehouse","Data Warehouse",{"slug":18,"name":19},"object-storage","Object Storage",[21,25,28,31,34,38,41,44,47,50,54,57],{"slug":22,"category":5,"name":23,"updated_at":24},"acid","ACID","2026-08-24T02:46:37+00:00",{"slug":26,"category":5,"name":27,"updated_at":24},"ann-search","ANN Search",{"slug":29,"category":5,"name":30,"updated_at":24},"backpressure","Backpressure",{"slug":32,"category":5,"name":33,"updated_at":24},"batch-processing","Batch Processing",{"slug":35,"category":5,"name":36,"updated_at":37},"bm25","BM25","2026-08-24T02:46:38+00:00",{"slug":39,"category":5,"name":40,"updated_at":24},"cache","Cache",{"slug":42,"category":5,"name":43,"updated_at":24},"cap-theorem","CAP Theorem",{"slug":45,"category":5,"name":46,"updated_at":24},"change-data-capture","Change Data Capture (CDC)",{"slug":48,"category":5,"name":49,"updated_at":24},"chroma","Chroma",{"slug":51,"category":5,"name":52,"updated_at":53},"chunk-overlap","Chunk Overlap","2026-08-24T03:30:02+00:00",{"slug":55,"category":5,"name":56,"updated_at":24},"columnar-storage","Columnar Storage",{"slug":58,"category":5,"name":59,"updated_at":24},"connection-pooling","Connection Pooling"]