Optical Character Recognition (OCR)

Optical character recognition (OCR) is the technology that converts images containing text — scanned documents, photographed receipts, screenshots, PDFs — into machine-readable, editable, and searchable text. Classical OCR engines (Tesseract) used pattern-matching against character templates and worked reasonably well on clean, high-contrast printed text but struggled with handwriting, skewed/rotated scans, unusual fonts, and complex layouts (tables, multi-column text). Modern AI-driven OCR increasingly uses vision-language models (multimodal LLMs like GPT-4o, Claude, and Gemini, or dedicated document-AI models like Google Document AI and AWS Textract) that don't just recognize individual characters but understand document structure and semantics — correctly parsing a table into rows and columns, distinguishing a header from a footer, and extracting specific fields (invoice number, total amount, vendor name) directly as structured JSON rather than returning a flat, unstructured text blob the application would then have to parse with brittle regular expressions. Why it matters for SaaS builders: OCR is foundational infrastructure for expense-management and invoicing tools (auto-extracting line items from a photographed receipt), document-digitization platforms (turning paper archives searchable), ID-verification/KYC flows (extracting name/DOB from a passport photo), and form-processing automation. Modern "OCR + LLM" pipelines have largely replaced brittle regex/template-based field extraction, since an LLM can generalize across document formats it's never seen before. A concrete worked example — an expense-tracking SaaS auto-filling receipts: (1) user photographs a paper receipt with their phone, often at an angle, under fluorescent lighting, slightly crumpled; (2) the image is sent to a multimodal LLM with a structured-extraction prompt: "Extract the following fields from this receipt image as JSON: merchant_name, date, total_amount, currency, line_items[]. If a field is illegible, return null for that field rather than guessing. Image: {image}"; (3) the model returns `{"merchant_name": "Blue Bottle Coffee", "date": "2026-06-28", "total_amount": 14.50, "currency": "USD", "line_items": [...]}`; (4) the app pre-fills the expense form with these fields, letting the user confirm and submit in seconds rather than manually typing every field, and highlights any field the model returned as `null` so the user knows to fill it in manually; (5) the receipt image itself is retained and linked to the expense record as an audit attachment, since most expense-policy and tax-compliance rules require the original document, not just the extracted data. Accuracy on handwriting, faded thermal-paper receipts, and low-quality or skewed images remains the main failure mode, so production flows should always surface the extracted fields for user confirmation before saving rather than silently trusting automated extraction for financial records.