Extraction API Reference

The extraction library in arbitex-core provides a unified interface for extracting text from binary content. All format handlers implement the same protocol and are looked up by MIME type through a lazy-loading registry.

Package: arbitex_core.extraction

---

Public API

extract()

The single entry point for all extraction. Looks up the appropriate handler by MIME type and returns an ExtractedText result.

from arbitex_core.extraction import extract

result = extract(content, content_type, filename=None)

Parameters:

Parameter	Type	Description
content	bytes	Raw file content
content_type	str	MIME type string (e.g., "application/pdf")
filename	str | None	Optional filename hint for extractors

Returns: ExtractedText

If no extractor is registered for the MIME type, returns an ExtractedText with empty text and extraction_failed: True in metadata.

---

Core Types

ExtractedText

Frozen dataclass returned by all extractors. Defined in arbitex_core.extraction.base.

@dataclass(frozen=True, slots=True)
class ExtractedText:
    text: str
    extraction_method: str
    metadata: dict = field(default_factory=dict)

Field	Type	Description
text	str	Extracted text content
extraction_method	str	Handler identifier: pdf, docx, xlsx, pptx, csv, rtf, txt, image/paddleocr, image/tesseract, direct, unknown
metadata	dict	Format-specific metadata (see per-handler sections below)

Error convention: When extraction fails, text is empty and metadata contains {"error": "...", "extraction_failed": True}.

ExtractorProtocol

Runtime-checkable protocol that all format handlers must implement. Defined in arbitex_core.extraction.base.

@runtime_checkable
class ExtractorProtocol(Protocol):
    def extract(self, content: bytes, content_type: str, filename: str | None = None) -> ExtractedText: ...

---

ExtractorRegistry

Flat MIME-to-extractor mapping with lazy imports and instance caching. Defined in arbitex_core.extraction.registry.

Methods

get(content_type: str) -> ExtractorProtocol | None

Look up an extractor by MIME type. Returns None for unsupported types. The extractor is instantiated on first access and cached — subsequent lookups for the same MIME type (or alias MIME types pointing to the same handler) return the cached instance.

supported_types() -> list[str]

Returns all registered MIME type strings.

MIME type mapping

MIME Type	Handler Class	Module
text/plain	TxtExtractor	arbitex_core.extraction.txt
text/csv	CsvExtractor	arbitex_core.extraction.csv_ext
text/rtf	RtfExtractor	arbitex_core.extraction.rtf
application/rtf	RtfExtractor	arbitex_core.extraction.rtf
application/pdf	PdfExtractor	arbitex_core.extraction.pdf
application/vnd.openxmlformats-officedocument.wordprocessingml.document	DocxExtractor	arbitex_core.extraction.docx
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet	XlsxExtractor	arbitex_core.extraction.xlsx
application/vnd.openxmlformats-officedocument.presentationml.presentation	PptxExtractor	arbitex_core.extraction.pptx
image/png	ImageExtractor	arbitex_core.extraction.image
image/jpeg	ImageExtractor	arbitex_core.extraction.image
image/tiff	ImageExtractor	arbitex_core.extraction.image
image/bmp	ImageExtractor	arbitex_core.extraction.image
image/webp	ImageExtractor	arbitex_core.extraction.image

All image MIME types share a single ImageExtractor instance.

---

Per-Format Handler Details

PdfExtractor

Library: pdfplumber | Install: pip install arbitex-core[pdf]

Extracts text from all pages using pdfplumber.open(). Pages are joined with newline separators.

Metadata fields:

Field	Type	Description
page_count	int	Number of pages in the PDF
word_count	int	Total word count (only if text is non-empty)

DocxExtractor

Library: python-docx | Install: pip install arbitex-core[docx]

Extracts text from all paragraphs using docx.Document(). Paragraphs are joined with newline separators.

Metadata fields:

Field	Type	Description
paragraph_count	int	Number of paragraphs
word_count	int	Total word count (only if text is non-empty)

XlsxExtractor

Library: openpyxl | Install: pip install arbitex-core[xlsx]

Opens workbooks in read-only mode with data_only=True (reads cached formula values). Iterates all worksheets, joining non-null cell values per row with spaces and rows with newlines.

Metadata fields:

Field	Type	Description
sheet_count	int	Number of worksheets
row_count	int	Total non-empty rows across all sheets
word_count	int	Total word count (only if text is non-empty)

PptxExtractor

Library: python-pptx | Install: pip install arbitex-core[pptx]

Iterates all slides and extracts text from shapes with text frames. Paragraphs within a slide are joined with newlines; slides are separated by double newlines.

Metadata fields:

Field	Type	Description
slide_count	int	Number of slides
word_count	int	Total word count (only if text is non-empty)

CsvExtractor

Library: stdlib csv | Install: (included)

Decodes content as UTF-8 (with errors="replace"), parses via csv.reader, and joins cells per row with spaces.

Metadata fields:

Field	Type	Description
row_count	int	Number of CSV rows
word_count	int	Total word count

RtfExtractor

Library: striprtf | Install: pip install arbitex-core[rtf]

Decodes content as UTF-8 and strips RTF formatting using rtf_to_text().

Metadata fields:

Field	Type	Description
word_count	int	Total word count (only if text is non-empty)

TxtExtractor

Library: stdlib | Install: (included)

Decodes content as UTF-8 with errors="replace". No transformation applied.

Metadata fields:

Field	Type	Description
word_count	int	Total word count (only if text is non-empty)

ImageExtractor

Library: PaddleOCR or pytesseract + Pillow | Install: pip install arbitex-core[ocr-paddle] or pip install arbitex-core[ocr-tesseract]

Delegates to a pluggable OCR backend selected by the OCR_BACKEND environment variable (default: paddleocr). The image is validated via Pillow before OCR processing.

OCR backends:

Backend	Class	Protocol Method	Description
PaddleOCR	PaddleOCRBackend	ocr(image_bytes) -> str	GPU inference, CJK/multilingual, handwriting. Models baked at build time.
Tesseract	TesseractBackend	ocr(image_bytes) -> str	CPU-only fallback.

Both implement the OCRBackend protocol:

@runtime_checkable
class OCRBackend(Protocol):
    def ocr(self, image_bytes: bytes) -> str: ...

Metadata fields:

Field	Type	Description
ocr_backend	str	Backend used: paddleocr or tesseract
image_format	str	Detected image format (e.g., PNG, JPEG)
width	int	Image width in pixels
height	int	Image height in pixels
word_count	int	OCR word count

Error handling: If the OCR backend is not installed, the ImageExtractor captures the import error at initialization and returns extraction_failed: True for all subsequent calls without raising.

---

Adding a New Format Handler

To add support for a new content type:

1. Create a handler module in arbitex_core/extraction/ (e.g., eml.py):

from arbitex_core.extraction.base import ExtractedText

class EmlExtractor:
    def extract(self, content: bytes, content_type: str, filename: str | None = None) -> ExtractedText:
        # Parse the content and extract text
        text = parse_eml(content)
        return ExtractedText(
            text=text,
            extraction_method="eml",
            metadata={"word_count": len(text.split())},
        )

2. Register the MIME type in arbitex_core/extraction/registry.py:

_MIME_MAP: dict[str, tuple[str, str]] = {
    # ... existing entries ...
    "message/rfc822": ("arbitex_core.extraction.eml", "EmlExtractor"),
}

3. Add optional dependencies in pyproject.toml if the handler requires third-party libraries:

[project.optional-dependencies]
eml = ["email-parser>=1.0"]

The handler is lazy-loaded on first access — the library is only imported when content of that MIME type arrives.

---

See also

• ScanEngine Architecture — How extraction fits into the full scan pipeline
• DLP Pipeline Configuration — Configuring the DLP tiers that process extracted text