Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do not merge] sccache test #170

Closed
wants to merge 77 commits into from
Closed

[do not merge] sccache test #170

wants to merge 77 commits into from

Commits on Sep 13, 2024

  1. chore: add fork OWNERS

    @z103cb @dtrifiro
    z103cb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    dbf639e View commit details
    Browse the repository at this point in the history

  2. add ubi Dockerfile

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a24f03d View commit details
    Browse the repository at this point in the history

  3. Dockerfile.ubi: remove references to grpc/protos

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    753f948 View commit details
    Browse the repository at this point in the history

  4. Dockerfile.ubi: use vllm-tgis-adapter

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    84eb826 View commit details
    Browse the repository at this point in the history

  5. gha: add sync workflow

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    577cb43 View commit details
    Browse the repository at this point in the history

  6. Dockerfile.ubi: use distributed-executor-backend=mp as default

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    e3be40e View commit details
    Browse the repository at this point in the history

  7. Dockerfile.ubi: remove vllm-nccl workaround 

    Fixed upstream in vllm-project#5091
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    6844cb9 View commit details
    Browse the repository at this point in the history

  8. Dockerfile.ubi: add missing requirements-*.txt bind mounts

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    cbffbf1 View commit details
    Browse the repository at this point in the history

  9. add triton CustomCacheManger 

    fixes RHOAIENG-8043

Co-authored-by: Chih-Chieh-Yang <[email protected]>
Signed-off-by: Thomas Parnell <[email protected]>
    @tdoublep @cyang49 @dtrifiro
    2 people authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ae4ac8d View commit details
    Browse the repository at this point in the history

  10. gha: sync-with-upstream workflow create PRs as draft

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    bd5ea86 View commit details
    Browse the repository at this point in the history

  11. add smoke/unit tests scripts

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    57b2cbf View commit details
    Browse the repository at this point in the history

  12. extras: exit unit tests on err

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    5b1ada5 View commit details
    Browse the repository at this point in the history

  13. Dockerfile.ubi: misc improvements 

    - get rid cuda-devel stage, use cuda 12.4
- add build flags
- remove useless installs
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    3a3dae7 View commit details
    Browse the repository at this point in the history

  14. update OWNERS

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f8e70cd View commit details
    Browse the repository at this point in the history

  15. Dockerfile.ubi: use tensorizer (#64) 

    add libsodium for tensorizer encryption

---------

Signed-off-by: Prashant Gupta <[email protected]>
Co-authored-by: Daniele <[email protected]>
    @prashantgupta24 @dtrifiro
    prashantgupta24 and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    57fdb00 View commit details
    Browse the repository at this point in the history

  16. Dockerfile.ubi: pin vllm-tgis-adapter to 0.1.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    586c30c View commit details
    Browse the repository at this point in the history

  17. gha: fix fetch step in upstream sync workflow

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f0dc391 View commit details
    Browse the repository at this point in the history

  18. gha: always update sync workflow PR body/title

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    cfded1b View commit details
    Browse the repository at this point in the history

  19. Dockerfile.ubi: bump vllm-tgis-adapter to 0.1.3

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    5493671 View commit details
    Browse the repository at this point in the history

  20. Dockerfile.ubi: get rid of --distributed-executor-backend=mp 

    this is the default when `--worker-use-ray` is not provided and
world-size > 1
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    eb82bd4 View commit details
    Browse the repository at this point in the history

  21. Dockerfile.ubi: add flashinfer

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a4bb48b View commit details
    Browse the repository at this point in the history

  22. pin adapter to 2.0.0

    @prashantgupta24 @dtrifiro
    prashantgupta24 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    8dcca5c View commit details
    Browse the repository at this point in the history

  23. deps: bump flashinfer to 0.0.9

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    48aa285 View commit details
    Browse the repository at this point in the history

  24. Update OWNERS with IBM folks

    @heyselbi @dtrifiro
    heyselbi authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    8955a3b View commit details
    Browse the repository at this point in the history

  25. Dockerfile.ubi: bind mount .git dir to allow inclusion of git commit … 

    …hash
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c66756a View commit details
    Browse the repository at this point in the history

  26. gha: remove reminder_comment

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    9d29a26 View commit details
    Browse the repository at this point in the history

  27. Dockerfile: bump vllm-tgis-adapter to 0.2.1

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    24a9763 View commit details
    Browse the repository at this point in the history

  28. fix: update setup.py to differentiate between fork and upstream 

    Signed-off-by: Nathan Weinberg <[email protected]>
    @nathan-weinberg @dtrifiro
    nathan-weinberg authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2e83b96 View commit details
    Browse the repository at this point in the history

  29. Dockerfile.ubi: properly mount .git dir

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    6cb0961 View commit details
    Browse the repository at this point in the history

  30. Revert "[CI/Build] fix: update setup.py to differentiate between fork… 

    … and upstream"
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a6ee52d View commit details
    Browse the repository at this point in the history

  31. Dockerfile.ubi: bump vllm-tgis-adapter to 0.2.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    6ada55d View commit details
    Browse the repository at this point in the history

  32. gha: remove unused upstream workflows

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    155d16f View commit details
    Browse the repository at this point in the history

  33. deps: bump vllm-tgis-adapter to 0.2.3

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    0835e4d View commit details
    Browse the repository at this point in the history

  34. Dockerfile.ubi: get rid of custom cache manager 

    fixed in vllm-project#6140

fixes https://issues.redhat.com/browse/RHOAIENG-8043
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d40e335 View commit details
    Browse the repository at this point in the history

  35. Dockerfile.ubi: add missing dependency

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2f92fe8 View commit details
    Browse the repository at this point in the history

  36. deps: bump vllm-tgis-adapter to 0.3.0

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    e061bb7 View commit details
    Browse the repository at this point in the history

  37. Dockerfile.ubi: force using python-installed cuda runtime libraries

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1f786f1 View commit details
    Browse the repository at this point in the history

  38. Dockerfile: use uv pip everywhere (it's faster)

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7b71e0c View commit details
    Browse the repository at this point in the history

  39. Dockerfile.ubi: bump flashinfer to 0.1.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ecb1c9d View commit details
    Browse the repository at this point in the history

  40. feat: allow long max seq length 

    Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415 @dtrifiro
    tjohnson31415 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a5c68a0 View commit details
    Browse the repository at this point in the history

  41. smoke test: kill server on timeout

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    367809a View commit details
    Browse the repository at this point in the history

  42. Dockerfile.ubi: set vllm_tgis_adapter unicorn log level to warning

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    fcd3419 View commit details
    Browse the repository at this point in the history

  43. fix: enable logprobs during spec decoding by default 

    Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415 @dtrifiro
    tjohnson31415 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    35a9167 View commit details
    Browse the repository at this point in the history

  44. deps: bump vllm-tgis-adapter to 0.4.0 (#132) 

    [changelog for
0.4.0](https://github.com/opendatahub-io/vllm-tgis-adapter/releases/tag/0.4.0)


https://issues.redhat.com/browse/RHOAIENG-11591
    @vaibhavjainwiz @dtrifiro
    vaibhavjainwiz authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    0df042d View commit details
    Browse the repository at this point in the history

  45. Disable usage tracking 

    This turns off tracking by default. If someone wants to, they
can simply override this in yaml.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    5a14601 View commit details
    Browse the repository at this point in the history

  46. Start by updating the image 

    A review showed that nowhere in the Dockerfile.ubi do we do a
microdnf -y update to pickup any known CVE and bugfixes. This patch
adds this to the build process.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    91520a7 View commit details
    Browse the repository at this point in the history

  47. Update ROCm build for UBI

    @Xaenalt @dtrifiro
    Xaenalt authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7c76447 View commit details
    Browse the repository at this point in the history

  48. Add sample chat template into vLLM container

    @vaibhavjainwiz @dtrifiro
    vaibhavjainwiz authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    05dd581 View commit details
    Browse the repository at this point in the history

  49. Harden build of libsodium 

    Libsodium is being built with default CFLAGS. This adds optimization on par
with cmake release builds. It also adds security hardening flags suggested
for RHEL 9 to protect against various issues.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c540965 View commit details
    Browse the repository at this point in the history

  50. Update Dockerfile.ubi 

    Remove debug code.
    @RH-steve-grubb @dtrifiro
    RH-steve-grubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d02a789 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Update OWNERS file

    @vaibhavjainwiz
    vaibhavjainwiz committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    1491b9e View commit details
    Browse the repository at this point in the history

  2. Merge pull request #160 from vaibhavjainwiz/update_OWNER 

    Update OWNERS file
    @vaibhavjainwiz
    vaibhavjainwiz authored Sep 16, 2024
    Configuration menu
    Copy the full SHA
    fed9b25 View commit details
    Browse the repository at this point in the history

  3. [CI/Build] Enable InternVL2 PP test only on single node (vllm-project… 

    …#8437)
    @Isotr0py @dtrifiro
    Isotr0py authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    e30866f View commit details
    Browse the repository at this point in the history

  4. [doc] recommend pip instead of conda (vllm-project#8446)

    @youkaichao @dtrifiro
    youkaichao authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    616f5ad View commit details
    Browse the repository at this point in the history

  5. [Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (vllm-project#8442 

    )
    @jeejeelee @dtrifiro
    jeejeelee authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8b1f881 View commit details
    Browse the repository at this point in the history

  6. [misc][ci] fix quant test (vllm-project#8449)

    @youkaichao @dtrifiro
    youkaichao authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    07afc6d View commit details
    Browse the repository at this point in the history

  7. [Installation] Gate FastAPI version for Python 3.8 (vllm-project#8456)

    @DarkLight1337 @dtrifiro
    DarkLight1337 authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    2c657ec View commit details
    Browse the repository at this point in the history

  8. [plugin][torch.compile] allow to add custom compile backend (vllm-pro… 

    …ject#8445)
    @youkaichao @dtrifiro
    youkaichao authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f8d2bf0 View commit details
    Browse the repository at this point in the history

  9. [CI/Build] Reorganize models tests (vllm-project#7820)

    @DarkLight1337 @dtrifiro
    DarkLight1337 authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    754dc0f View commit details
    Browse the repository at this point in the history

  10. [Doc] Add oneDNN installation to CPU backend documentation (vllm-proj… 

    …ect#8467)
    @Isotr0py @dtrifiro
    Isotr0py authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    bf7e710 View commit details
    Browse the repository at this point in the history

  11. [HotFix] Fix final output truncation with stop string + streaming (vl… 

    …lm-project#8468)
    @njhill @dtrifiro
    njhill authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8d32eaf View commit details
    Browse the repository at this point in the history

  12. bump version to v0.6.1.post2 (vllm-project#8473)

    @simon-mo @dtrifiro
    simon-mo authored and dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    1ed711a View commit details
    Browse the repository at this point in the history

  13. Dockerfile.rocm.ubi: cleanup 

    - get rid of non-essential dependencies
- consolidate package installs
- do not copy wheels in final stage
- fix ccache usage
- use flashattention with triton backend by default:
    - clone main_perf branch
    - build rocm target
    - set up triton rocm env var
- configure numba, outlines and triton cache directory
    @dtrifiro
    dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    93c04f3 View commit details
    Browse the repository at this point in the history

  14. add vllm-tgis-adapter layer

    @dtrifiro
    dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    66984d4 View commit details
    Browse the repository at this point in the history

  15. Dockerfile.ubi: bump python to 3.12

    @dtrifiro
    dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f5387d0 View commit details
    Browse the repository at this point in the history

  16. Dockerfile.ubi: bump flashinfer to 0.1.6

    @dtrifiro
    dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b3abd3a View commit details
    Browse the repository at this point in the history

  17. Dockerfile.rocm.ubi: do not use nightly pytorch_triton

    @dtrifiro
    dtrifiro committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    9f85dae View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Dockerfile.ubi: fix PYTHON_VERSION arg usage 

    fixup to f5387d0 (openshift/release#56770)
    @dtrifiro
    dtrifiro committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    083f0d5 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. Dockerfile.rocm.ubi: move microdnf update in base stage

    @dtrifiro
    dtrifiro committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    c29c9f4 View commit details
    Browse the repository at this point in the history

  2. Dockerfile.rocm.ubi: bump torch version to 2.5.0.dev20240912+rocm6.1

    @dtrifiro
    dtrifiro committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    69ac6c1 View commit details
    Browse the repository at this point in the history

  3. Dockerfile.rocm.ubi: get rid of build triton stage 

    this is a torch dependency when installed from the pytorch/rocm6.1
index: https://download.pytorch.org/whl/nightly/rocm6.1
    @dtrifiro
    dtrifiro committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    399c114 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #167 from dtrifiro/fix-amd-build 

    Dockerfile.ubi.rocm: fix build
    @RH-steve-grubb
    RH-steve-grubb authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    9ebf28d View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Sync with upstream @ v0.6.2

    @dtrifiro
    dtrifiro committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    18d7da7 View commit details
    Browse the repository at this point in the history

  2. Dockerfile.rocm.ubi: add setuptools-scm build dependency

    @dtrifiro
    dtrifiro committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    56fdd53 View commit details
    Browse the repository at this point in the history

  3. Dockerfile.ubi: add VLLM_FA_CMAKE_GPU_ARCHES

    @dtrifiro
    dtrifiro committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    d151278 View commit details
    Browse the repository at this point in the history

  4. ⚗️ try public sccache 

    Signed-off-by: Joe Runde <[email protected]>
    @joerunde
    joerunde committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    42a5a5b View commit details
    Browse the repository at this point in the history

  5. 🔥 remove sudo 

    Signed-off-by: Joe Runde <[email protected]>
    @joerunde
    joerunde committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8abfa38 View commit details
    Browse the repository at this point in the history