This is the accompanying repository for the Paper Is open-source there yet? A comparative study on commercial and open-source LLMs in their ability to label Chest X-Ray reports.
The repository contains the code that was used to perform the labeling of chest x-ray reports using GPT-4 and different open-source Large Language Models hosted that were locally hosted through vLLM.
Follow these steps to perform the local LLM inference:
-
Follow the Quickstart Instructions of VLLM to locally deploy a Open-AI compatible server. This server will be queried by the labeling script in this repository through the Open-AI Python API.
-
Setup the CSV file that contains the free-text reports that should be labeled. Three columns: 'Patient MRN', 'Accession Number' and 'Report Text' are expected by default. The prompt.py can be adjusted for individual use cases.
-
Run the create_labels_vllm.py script.
TBD