CS491/492 · Bilkent University 2025–26

Semantic search for
Digital Pathology
at Scale

Query your pathology archive using clinical language or image regions. PatchMatch leverages state-of-the-art foundation models like CONCH and TITAN to retrieve diagnostic-grade tissue matches across 70,000+ indexed patches in milliseconds.

See how it works

BRACS retrieval results — query: IC region · CONCH v1 encoder

Hover patches to reveal tissue class · simulated from real benchmark data

Core capabilities

Built for how
pathologists think

Every search mode maps to a real clinical workflow. PatchMatch connects the way pathologists describe tissue to the way AI encodes it.

Natural language search

Find slides using natural language queries. Retrieve cases by typing diagnostic terms, clinical details from reports, descriptive tissue appearances, or even filter by cellular metrics like TIL ratio.

Dense region search

Select any region on a slide to perform a patch-by-patch comparison across the entire library. Find visually and morphologically similar regions, ranked by match quality, to identify recurrent patterns.

Diagnostic class heatmap

Visualize the spatial distribution of diagnostic categories across the entire slide. A color-coded overlay highlights specific tissue types like Normal, Benign, Atypical, and Carcinoma to easily identify high-risk regions.

Slide-to-slide retrieval

Identify slides most similar to the current case based on overall tissue architecture and cellular patterns. Explore matching cases with optional similarity maps highlighting shared morphological characteristics.

Automated cellular metrics

Instantly detect and outline individual cells (Tumor, Immune, Connective). Automatically calculate key diagnostic indicators like Tumor Cellularity, TIL, and TSR, supported by spatial heatmaps for cell density and circularity.

AI-assisted region inspection

Click and inspect specific local regions to receive automated semantic descriptions of tissue morphology. Review predicted diagnostic class distributions, attach clinical notes, and export comprehensive PDF reports.

Architecture

From gigapixel slide to
ranked results in milliseconds

Modular ML Subsystem: fully decoupled online similarity search services powered by dedicated foundation models.

Text Search (CONCH v1)

Clinical text and reports are encoded using CONCH v1 (CoCa). The Search Engine executes FAISS Inner-Product retrieval, applying IDF-weighted keyword boosts and clinical entity extraction (e.g., ER/PR statuses).

Region-to-Slide Search (CONCH v1)

Dense ROI queries are executed using CONCH v1 image embeddings. The system performs batched k-NN per patch and aggregates results via max-pool-then-mean to generate slide-level rankings and spatial heatmaps.

Slide-to-Slide Search (CONCH v1.5)

Whole-slide nearest-neighbor search is powered by CONCH v1.5 (ViT-L). It provides high-fidelity global retrieval combined with dense cross-slide patch similarity heatmaps.

Cellular Metrics (CellViT)

Quantitative tissue metrics are generated by streaming pre-computed CellViT segmentations. It dynamically calculates Tumor Cellularity, TIL ratio, and TSR for precise clinical assessment.

Region Inspect

When inspecting a specific area, a BRACS-trained linear probe predicts diagnostic class probabilities (e.g., ADH, DCIS, IC) in real time, while a Quilt-1M semantic index retrieves contextual natural-language tissue descriptions.

Foundation Models

CONCH v1

Text & Region Search

CONCH v1.5

Slide-to-Slide Search

CellViT

Cellular Analytics

Storage & Indices

Text & Region Index (CONCH v1)

FAISS IndexFlatIP · Dedicated index for clinical text and dense ROI patch queries.

Slide Index (CONCH v1.5)

FAISS IndexFlatIP · High-fidelity index for global slide-level retrieval.

Cellular Analytics (CellViT)

msgpack / HDF5 / .npy storage for rapid spatial statistics.

Search modalities are independently evaluated and powered by separate foundation models. Quantitative tissue metrics are completely decoupled and driven by localized CellViT data.