Ingesting Clinical Documents

Clinical text often arrives as scanned faxes, photographed notes, CSV exports, or C-CDA XML — not plain text. openmed.multimodal converts these into a normalized ExtractedDocument (clean text + character-offset → source-location spans) so you can run de-identification and NER. It runs on-device: OCR backends are local, no document leaves the machine.

When to use

• You have images / scanned faxes of clinical notes and need text out (OCR).
• You have CSV/TSV patient exports that need column-aware handling.
• You have C-CDA XML to flatten into text.
• You are building the intake stage that feeds openmed.deidentify and openmed.analyze_text.

This is the first stage. After intake, hand off to deidentifying-clinical-text then extracting-clinical-entities.

What is supported today

redact_document dispatches by file extension. Live handlers:

Input	Extensions	Path
Images / scans	.png .jpg .jpeg .tif .tiff .bmp .gif .webp	OCR (ocr() / image handler)
Tables	.csv .tsv	column-aware tabular redaction
C-CDA	.xml (detected as CDA)	stdlib CDA adapter

PDF and DOCX have no live handler yet — redact_document("x.pdf") raises UnsupportedDocumentError. Convert PDFs to page images first (or to text with your own tool) and feed the images through OCR. See references/multimodal-ingest.md for the full contract, engines, and the tabular pipeline.

Install

pip install "openmed[multimodal]"      # document intake contract + image deps
pip install "openmed[ocr-paddle]"      # add the PaddleOCR engine
# Tesseract engine also needs the system binary, e.g.:  brew install tesseract

Quick start: OCR an image, then de-identify

The clean two-step intake path. ocr() lives in the submodule (it is intentionally not re-exported from openmed.multimodal):

from openmed.multimodal.ocr import ocr
import openmed

# 1) OCR a scanned/faxed note -> OcrResult -> ExtractedDocument -> plain text
result = ocr("fax_page.png", engine=None)   # None = auto-select an installed engine
doc    = result.to_document()                # ExtractedDocument
text   = doc.text                            # clean text for downstream OpenMed

# 2) De-identify, then run NER (privacy-first order)
deid = openmed.deidentify(text, method="mask", policy="hipaa_safe_harbor")
ner  = openmed.analyze_text(deid.deidentified_text, output_format="dict")

for ent in ner.entities:
    print(ent.label, ent.text, ent.confidence)

engine may be None (auto-select), "tesseract", "paddleocr", or an OcrEngine instance. OcrResult exposes .text and per-word boxes via .words (each OcrWord has text, bbox, confidence, page).

One-step intake + redaction with redact_document

For images, CSV/TSV, and CDA, redact_document performs intake and de-identification in a single, format-aware call, returning an already-redacted ExtractedDocument:

from openmed.multimodal import redact_document

# Image scan: OCR + redact in one call
doc = redact_document("fax_page.png")
print(doc.text)        # redacted text
print(doc.spans[:3])   # SourceSpan offsets -> page / bbox in the original scan

# CSV export: per-column classification (direct id / quasi-id / safe) + redaction
table_doc = redact_document("patients.csv")
print(table_doc.text)

Use redact_document when you want OpenMed to own intake and redaction (especially for tables, where redaction is column-scoped, not free-text NER). Use the ocr() → to_document() → deidentify path when you want to control the de-identification method, policy, or mapping yourself.

Tabular CSV/TSV redaction

CSV columns get classified before any cell is touched, so a free-text NER pass is not run blindly over structured data:

from openmed.multimodal import read_table, redact_table

view = read_table("patients.csv")            # TableView with column decisions
for col in view.columns:
    print(col.name, "->", col.assigned_class, col.action, col.canonical_label)

redacted = redact_table("patients.csv", keep_year=True)
print(redacted.text)            # redacted CSV
for entry in redacted.manifest: # PHI-SAFE audit: counts/actions per column, no raw values
    print(entry)

redact_table(...) returns a RedactedTable with .text, .headers, .rows, .columns, and a PHI-safe .manifest (no raw cell values). See references/multimodal-ingest.md for column classes and actions.

Preserve layout / reading order and map back to the source

Every ExtractedDocument keeps character offset → source location. After detecting PHI on doc.text, project a span's offset back to its page and bounding box:

from openmed.multimodal.ocr import ocr
import openmed

doc  = ocr("fax_page.png").to_document()
deid = openmed.deidentify(doc.text, method="mask")

for ent in deid.pii_entities:
    loc = doc.location_at(ent.start)   # SourceSpan or None
    if loc is not None:
        print(ent.label, "page", loc.page, "bbox", loc.bbox)

This lets you redact pixels on the original scan, not just the extracted text.

Hand-off to / from OpenMed

• To deidentifying-clinical-text: pass doc.text to openmed.deidentify(...) with a policy profile; this is the required next stage for PHI.
• To extracting-clinical-entities: run openmed.analyze_text on the redacted text, not raw OCR output.
• From file conversion (out-of-process): for PDFs/DOCX, render to page images with your own tool, then OCR those images through this skill.

Edge cases & gotchas

• ocr() is imported from the submodule: from openmed.multimodal.ocr import ocr. It is deliberately not re-exported from openmed.multimodal.
• No PDF/DOCX handler yet: redact_document raises UnsupportedDocumentError for them. Rasterize to images first.
• OCR needs a backend: install [ocr-paddle] for PaddleOCR, or the system Tesseract binary for pytesseract. Missing backends raise MissingDependencyError with an install hint.
• OCR is noisy: misreads lower downstream recall. Prefer higher-DPI scans; inspect OcrWord.confidence to flag low-quality pages.
• Tables are not free text: redact_table redacts per column classification — don't run whole-table NER and expect structured columns to be handled correctly.
• No raw PHI in artifacts: the table manifest and any logs record counts/actions/labels, never raw values. Keep OCR intermediates on-device and out of logs.
• Local-first: OCR engines run locally; do not send scans to a cloud OCR API in a PHI workflow.

Standards & references

• HIPAA de-identification: https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/
• Tesseract OCR: https://github.com/tesseract-ocr/tesseract
• PaddleOCR: https://github.com/PaddlePaddle/PaddleOCR
• HL7 C-CDA: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=492
• Full intake contract, engines, and table pipeline: references/multimodal-ingest.md