Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync with 0.6.1.post1 #159

Closed
wants to merge 50 commits into from
Closed

sync with 0.6.1.post1 #159

wants to merge 50 commits into from

Commits on Sep 13, 2024

  1. chore: add fork OWNERS

    @z103cb @dtrifiro
    z103cb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    fa61f56 View commit details
    Browse the repository at this point in the history

  2. add ubi Dockerfile

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c594de3 View commit details
    Browse the repository at this point in the history

  3. Dockerfile.ubi: remove references to grpc/protos

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    81472ad View commit details
    Browse the repository at this point in the history

  4. Dockerfile.ubi: use vllm-tgis-adapter

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    05e3eb2 View commit details
    Browse the repository at this point in the history

  5. gha: add sync workflow

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c40a107 View commit details
    Browse the repository at this point in the history

  6. Dockerfile.ubi: use distributed-executor-backend=mp as default

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f7adf58 View commit details
    Browse the repository at this point in the history

  7. Dockerfile.ubi: remove vllm-nccl workaround 

    Fixed upstream in vllm-project#5091
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c26c2c2 View commit details
    Browse the repository at this point in the history

  8. Dockerfile.ubi: add missing requirements-*.txt bind mounts

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2e659d8 View commit details
    Browse the repository at this point in the history

  9. add triton CustomCacheManger 

    fixes RHOAIENG-8043

Co-authored-by: Chih-Chieh-Yang <[email protected]>
Signed-off-by: Thomas Parnell <[email protected]>
    @tdoublep @cyang49 @dtrifiro
    2 people authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    9bc016c View commit details
    Browse the repository at this point in the history

  10. gha: sync-with-upstream workflow create PRs as draft

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    66f7b78 View commit details
    Browse the repository at this point in the history

  11. add smoke/unit tests scripts

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    3c23baa View commit details
    Browse the repository at this point in the history

  12. extras: exit unit tests on err

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    87b91df View commit details
    Browse the repository at this point in the history

  13. Dockerfile.ubi: misc improvements 

    - get rid cuda-devel stage, use cuda 12.4
- add build flags
- remove useless installs
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d0b5860 View commit details
    Browse the repository at this point in the history

  14. update OWNERS

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c1dd90f View commit details
    Browse the repository at this point in the history

  15. Dockerfile.ubi: use tensorizer (vllm-project#64) 

    add libsodium for tensorizer encryption

---------

Signed-off-by: Prashant Gupta <[email protected]>
Co-authored-by: Daniele <[email protected]>
    @prashantgupta24 @dtrifiro
    prashantgupta24 and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    3a84da1 View commit details
    Browse the repository at this point in the history

  16. Dockerfile.ubi: pin vllm-tgis-adapter to 0.1.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    bcd8a81 View commit details
    Browse the repository at this point in the history

  17. gha: fix fetch step in upstream sync workflow

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1dbc8d2 View commit details
    Browse the repository at this point in the history

  18. gha: always update sync workflow PR body/title

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    4efe90d View commit details
    Browse the repository at this point in the history

  19. Dockerfile.ubi: bump vllm-tgis-adapter to 0.1.3

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    e7a09af View commit details
    Browse the repository at this point in the history

  20. Dockerfile.ubi: get rid of --distributed-executor-backend=mp 

    this is the default when `--worker-use-ray` is not provided and
world-size > 1
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    b395ab6 View commit details
    Browse the repository at this point in the history

  21. Dockerfile.ubi: add flashinfer

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f17d214 View commit details
    Browse the repository at this point in the history

  22. pin adapter to 2.0.0

    @prashantgupta24 @dtrifiro
    prashantgupta24 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1218f2a View commit details
    Browse the repository at this point in the history

  23. deps: bump flashinfer to 0.0.9

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    3b85222 View commit details
    Browse the repository at this point in the history

  24. Update OWNERS with IBM folks

    @heyselbi @dtrifiro
    heyselbi authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d319f2c View commit details
    Browse the repository at this point in the history

  25. Dockerfile.ubi: bind mount .git dir to allow inclusion of git commit … 

    …hash
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    496021c View commit details
    Browse the repository at this point in the history

  26. gha: remove reminder_comment

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f3dc750 View commit details
    Browse the repository at this point in the history

  27. Dockerfile: bump vllm-tgis-adapter to 0.2.1

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    aedf0e0 View commit details
    Browse the repository at this point in the history

  28. fix: update setup.py to differentiate between fork and upstream 

    Signed-off-by: Nathan Weinberg <[email protected]>
    @nathan-weinberg @dtrifiro
    nathan-weinberg authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    89418c5 View commit details
    Browse the repository at this point in the history

  29. Dockerfile.ubi: properly mount .git dir

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    0514c06 View commit details
    Browse the repository at this point in the history

  30. Revert "[CI/Build] fix: update setup.py to differentiate between fork… 

    … and upstream"
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    cb030df View commit details
    Browse the repository at this point in the history

  31. Dockerfile.ubi: bump vllm-tgis-adapter to 0.2.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ccdaeab View commit details
    Browse the repository at this point in the history

  32. gha: remove unused upstream workflows

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a9cc1d8 View commit details
    Browse the repository at this point in the history

  33. deps: bump vllm-tgis-adapter to 0.2.3

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    88bd920 View commit details
    Browse the repository at this point in the history

  34. Dockerfile.ubi: get rid of custom cache manager 

    fixed in vllm-project#6140

fixes https://issues.redhat.com/browse/RHOAIENG-8043
    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    bf46368 View commit details
    Browse the repository at this point in the history

  35. Dockerfile.ubi: add missing dependency

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d08e827 View commit details
    Browse the repository at this point in the history

  36. deps: bump vllm-tgis-adapter to 0.3.0

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d3298d1 View commit details
    Browse the repository at this point in the history

  37. Dockerfile.ubi: force using python-installed cuda runtime libraries

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    003bd23 View commit details
    Browse the repository at this point in the history

  38. Dockerfile: use uv pip everywhere (it's faster)

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    bbd8828 View commit details
    Browse the repository at this point in the history

  39. Dockerfile.ubi: bump flashinfer to 0.1.2

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a22bef9 View commit details
    Browse the repository at this point in the history

  40. feat: allow long max seq length 

    Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415 @dtrifiro
    tjohnson31415 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    4832f81 View commit details
    Browse the repository at this point in the history

  41. smoke test: kill server on timeout

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    13c9e01 View commit details
    Browse the repository at this point in the history

  42. Dockerfile.ubi: set vllm_tgis_adapter unicorn log level to warning

    @dtrifiro
    dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    61586fb View commit details
    Browse the repository at this point in the history

  43. fix: enable logprobs during spec decoding by default 

    Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415 @dtrifiro
    tjohnson31415 authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    52c1c44 View commit details
    Browse the repository at this point in the history

  44. deps: bump vllm-tgis-adapter to 0.4.0 (vllm-project#132) 

    [changelog for
0.4.0](https://github.com/opendatahub-io/vllm-tgis-adapter/releases/tag/0.4.0)


https://issues.redhat.com/browse/RHOAIENG-11591
    @vaibhavjainwiz @dtrifiro
    vaibhavjainwiz authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    6771321 View commit details
    Browse the repository at this point in the history

  45. Disable usage tracking 

    This turns off tracking by default. If someone wants to, they
can simply override this in yaml.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7b53dfa View commit details
    Browse the repository at this point in the history

  46. Start by updating the image 

    A review showed that nowhere in the Dockerfile.ubi do we do a
microdnf -y update to pickup any known CVE and bugfixes. This patch
adds this to the build process.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ca92dbf View commit details
    Browse the repository at this point in the history

  47. Update ROCm build for UBI

    @Xaenalt @dtrifiro
    Xaenalt authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    226d986 View commit details
    Browse the repository at this point in the history

  48. Add sample chat template into vLLM container

    @vaibhavjainwiz @dtrifiro
    vaibhavjainwiz authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    9811c0d View commit details
    Browse the repository at this point in the history

  49. Harden build of libsodium 

    Libsodium is being built with default CFLAGS. This adds optimization on par
with cmake release builds. It also adds security hardening flags suggested
for RHEL 9 to protect against various issues.
    @stevegrubb @dtrifiro
    stevegrubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d39d0c1 View commit details
    Browse the repository at this point in the history

  50. Update Dockerfile.ubi 

    Remove debug code.
    @RH-steve-grubb @dtrifiro
    RH-steve-grubb authored and dtrifiro committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    32cacb1 View commit details
    Browse the repository at this point in the history