Skip to content

Commit

Permalink
Add GPU support instructions
Browse files Browse the repository at this point in the history
- Validated on an EC2 GPU instance to work.
- Closes #8

Signed-off-by: Brent Salisbury <[email protected]>
  • Loading branch information
nerdalert committed Sep 25, 2024
1 parent eae3859 commit af7ae3e
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,24 @@ curl -X 'POST' \
}
}'
```

### Cuda GPU Support

For GPU support try the following:

```sh
# Create a virtual env
python3 -m venv venv

# Activate the venv
source venv/bin/active

# Install torch with the special index
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install the package
pip install -e .

# Run the server
poetry run uvicorn docling_serve.app:app --reload
```

Check failure on line 56 in README.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Files should end with a single newline character

README.md:56:3 MD047/single-trailing-newline Files should end with a single newline character https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md047.md

0 comments on commit af7ae3e

Please sign in to comment.