File size: 4,471 Bytes
e407339 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 | ---
license: apache-2.0
library_name: transformers
tags:
- medical
- radiology
- chest-x-ray
- feature-extraction
- cxr-bert
- embeddings
base_model: microsoft/BiomedVLP-CXR-BERT-specialized
pipeline_tag: feature-extraction
---
# CXRFE — Chest X-ray Fact Encoder
CXRFE is a radiology **fact encoder** for chest X-ray report text. It embeds factual statements (and short report phrases) into a **128-dimensional** projected embedding space for retrieval, ranking, NLI-style comparison, and fact-level evaluation metrics.
It is part of the two-stage *Extracting and Encoding* framework from Findings of ACL 2024:
1. **Fact extraction** — [`pamessina/T5FactExtractor`](https://huggingface.co/pamessina/T5FactExtractor)
2. **Fact encoding** — this model (`pamessina/CXRFE`)
Paper: [*Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation*](https://aclanthology.org/2024.findings-acl.236/)
## Model details
| | |
|---|---|
| **Architecture** | CXR-BERT (`CXRBertModel`) with a projection head |
| **Initialized from** | [`microsoft/BiomedVLP-CXR-BERT-specialized`](https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized) |
| **Hidden size** | 768 |
| **Projected embedding size** | 128 (`projection_size`) |
| **Intended inputs** | Short radiology facts / sentences (typically after fact extraction) |
| **License** | Apache 2.0 |
> **Note:** This public checkpoint is trained with slightly more NLI data than the single best CXRFE variant reported in the paper. Additional paper-matched variants may be released later.
## How to use
Requires `trust_remote_code=True` (custom CXR-BERT code from the BioViL / CXR-BERT family).
### Projected embeddings (recommended)
This is the representation used by [CXRFEScore](https://github.com/PabloMessina/CXRFEScore) (`get_projected_text_embeddings`):
```python
import torch
from transformers import AutoModel, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
model_id = "pamessina/CXRFE"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device)
model.eval()
texts = [
"small right pleural effusion",
"normal heart size",
]
inputs = tokenizer(
texts,
add_special_tokens=True,
padding="longest",
return_tensors="pt",
)
input_ids = inputs["input_ids"].to(device)
attention_mask = inputs["attention_mask"].to(device)
with torch.no_grad():
embeddings = model.get_projected_text_embeddings(
input_ids=input_ids,
attention_mask=attention_mask,
)
print(embeddings.shape) # (batch_size, 128)
```
### Easiest path: CXRFEScore
If you want fact extraction + encoding + report-pair scoring in one API:
```bash
pip install cxrfescore
# optional heatmaps:
pip install "cxrfescore[viz]"
```
```python
from cxrfescore import CXRFEScore
metric = CXRFEScore(device="cuda") # default encoder: pamessina/CXRFE
result = metric(
["There is a small right pleural effusion. The heart size is normal."],
["Small right pleural effusion. Normal heart size."],
)
print(result["mean_similarity"])
```
Demo notebook: [CXR-Fact-Encoder / notebooks/cxrfescore_demo.ipynb](https://github.com/PabloMessina/CXR-Fact-Encoder/blob/main/notebooks/cxrfescore_demo.ipynb)
## Related resources
- Paper hub: https://github.com/PabloMessina/CXR-Fact-Encoder
- Metric package: https://github.com/PabloMessina/CXRFEScore · [PyPI](https://pypi.org/project/cxrfescore/)
- Companion fact extractor: https://huggingface.co/pamessina/T5FactExtractor
- ACL Anthology: https://aclanthology.org/2024.findings-acl.236/
- arXiv: https://arxiv.org/abs/2407.01948
## Citation
If you use CXRFE, please cite:
```bibtex
@inproceedings{messina-etal-2024-extracting,
title = "Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation",
author = "Messina, Pablo and
Vidal, Rene and
Parra, Denis and
Soto, Alvaro and
Araujo, Vladimir",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-acl.236/",
doi = "10.18653/v1/2024.findings-acl.236",
pages = "3955--3986"
}
```
|