Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

habana_main rebase v6 #182

Closed
wants to merge 181 commits into from
Closed

habana_main rebase v6 #182

wants to merge 181 commits into from

Commits on Jul 30, 2024

  1. [Doc] Super tiny fix doc typo (vllm-project#6949)

    @fzyzcjy
    fzyzcjy authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    f058403 View commit details
    Browse the repository at this point in the history

  2. [BugFix] Fix use of per-request seed with pipeline parallel (vllm-pro… 

    …ject#6698)
    @njhill
    njhill authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    5cf9254 View commit details
    Browse the repository at this point in the history

  3. [Kernel] Squash a few more warnings (vllm-project#6914)

    @tlrmchlsmth
    tlrmchlsmth authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    cbbc904 View commit details
    Browse the repository at this point in the history

  4. [OpenVINO] Updated OpenVINO requirements and build docs (vllm-project… 

    …#6948)
    @ilya-lavrenov
    ilya-lavrenov authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    5895b24 View commit details
    Browse the repository at this point in the history

  5. [Bugfix] Fix tensorizer memory profiling bug during testing (vllm-pro… 

    …ject#6881)
    @sangstar
    sangstar authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    052b6f8 View commit details
    Browse the repository at this point in the history

  6. [Kernel] Remove scaled_fp8_quant kernel padding footgun (vllm-project… 

    …#6842)
    @tlrmchlsmth
    tlrmchlsmth authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    d7a299e View commit details
    Browse the repository at this point in the history

  7. [core][misc] improve free_finished_seq_groups (vllm-project#6865) 

    Co-authored-by: Woosuk Kwon <[email protected]>
    @youkaichao @WoosukKwon
    youkaichao and WoosukKwon authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    6ca8031 View commit details
    Browse the repository at this point in the history

  8. [Build] Temporarily Disable Kernels and LoRA tests (vllm-project#6961)

    @simon-mo
    simon-mo authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    40c27a7 View commit details
    Browse the repository at this point in the history

  9. [Nightly benchmarking suite] Remove pkill python from run benchmark s… 

    …uite (vllm-project#6965)
    @cadedaniel
    cadedaniel authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    79319ce View commit details
    Browse the repository at this point in the history

  10. [CI] [nightly benchmark] Do not re-download sharegpt dataset if exists ( 

    vllm-project#6706)
    @cadedaniel
    cadedaniel authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    fb4f530 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. [Speculative decoding] Add serving benchmark for llama3 70b + specula… 

    …tive decoding (vllm-project#6964)
    @cadedaniel
    cadedaniel authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c32ab8b View commit details
    Browse the repository at this point in the history

  2. [mypy] Enable following imports for some directories (vllm-project#6681)

    @DarkLight1337
    DarkLight1337 authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    da1f7cc View commit details
    Browse the repository at this point in the history

  3. [Bugfix] Fix broadcasting logic for multi_modal_kwargs (vllm-projec… 

    …t#6836)
    @DarkLight1337
    DarkLight1337 authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    f230cc2 View commit details
    Browse the repository at this point in the history

  4. [CI/Build] Fix mypy errors (vllm-project#6968)

    @DarkLight1337
    DarkLight1337 authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    9f0e69b View commit details
    Browse the repository at this point in the history

  5. [Bugfix][TPU] Set readonly=True for non-root devices (vllm-project#6980)

    @WoosukKwon
    WoosukKwon authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    533d193 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] fix logit processor excceed vocab size issue (vllm-project#6927 

    )
    @FeiDeng
    FeiDeng authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c0644cf View commit details
    Browse the repository at this point in the history

  7. Support W4A8 quantization for vllm (vllm-project#5218)

    @HandH1998
    HandH1998 authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    6512937 View commit details
    Browse the repository at this point in the history

  8. [Bugfix] Clean up MiniCPM-V (vllm-project#6939) 

    Co-authored-by: hezhihui <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
    @HwwwwwwwH @DarkLight1337
    3 people authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    2f4e108 View commit details
    Browse the repository at this point in the history

  9. [Bugfix] Fix feature size calculation for LLaVA-NeXT (vllm-project#6982)

    @DarkLight1337
    DarkLight1337 authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    daed30c View commit details
    Browse the repository at this point in the history

  10. [Model] use FusedMoE layer in Jamba (vllm-project#6935)

    @avshalomman
    avshalomman authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    2ee8d3b View commit details
    Browse the repository at this point in the history

  11. [MISC] Introduce pipeline parallelism partition strategies (vllm-proj… 

    …ect#6920)

Co-authored-by: youkaichao <[email protected]>
    @comaniac @youkaichao
    comaniac and youkaichao authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    bd70013 View commit details
    Browse the repository at this point in the history

  12. [Bugfix] Support cpu offloading with fp8 quantization (vllm-project#6960 

    )
    @mgoin
    mgoin authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    460c188 View commit details
    Browse the repository at this point in the history

  13. [Kernel] Enable FP8 Cutlass for Ada Lovelace (vllm-project#6950) 

    Co-authored-by: Varun Sundar Rabindranath <[email protected]>
    @varun-sundar-rabindranath
    varun-sundar-rabindranath and Varun Sundar Rabindranath authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    93548eb View commit details
    Browse the repository at this point in the history

  14. [Kernel] Tuned int8 Cutlass Kernels for SM75 (T4) (vllm-project#6996) 

    Co-authored-by: Varun Sundar Rabindranath <[email protected]>
    @varun-sundar-rabindranath
    varun-sundar-rabindranath and Varun Sundar Rabindranath authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    35e9c12 View commit details
    Browse the repository at this point in the history

  15. [Misc] Add compressed-tensors to optimized quant list (vllm-project#7006 

    )
    @mgoin
    mgoin authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    a0dce93 View commit details
    Browse the repository at this point in the history

  16. Revert "[Frontend] Factor out code for running uvicorn" (vllm-project… 

    …#7012)

Co-authored-by: Robert Shaw <[email protected]>
    @simon-mo @robertgshaw2-neuralmagic
    simon-mo and robertgshaw2-neuralmagic authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    7eb0cb4 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. [Kernel][RFC] Refactor the punica kernel based on Triton (vllm-projec… 

    …t#5036)
    @jeejeelee
    jeejeelee authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    7ecee34 View commit details
    Browse the repository at this point in the history

  2. [Model] Pipeline parallel support for Qwen2 (vllm-project#6924)

    @xuyi
    xuyi authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    1d2e7fb View commit details
    Browse the repository at this point in the history

  3. [Bugfix][TPU] Do not use torch.Generator for TPUs (vllm-project#6981)

    @WoosukKwon
    WoosukKwon authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    23993a7 View commit details
    Browse the repository at this point in the history

  4. [Bugfix][Model] Skip loading lm_head weights if using tie_word_embedd… 

    …ings (vllm-project#6758)

Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415
    tjohnson31415 authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    630dd9e View commit details
    Browse the repository at this point in the history

  5. PP comm optimization: replace send with partial send + allgather (vll… 

    …m-project#6695)

Co-authored-by: Aurick Qiao <[email protected]>
    @aurickq @sfc-gh-aqiao
    aurickq and sfc-gh-aqiao authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    0437492 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not pro… 

    …vided by user (vllm-project#6954)
    @zifeitong
    zifeitong authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    3c10591 View commit details
    Browse the repository at this point in the history

  7. [core][scheduler] simplify and improve scheduler (vllm-project#6867)

    @youkaichao
    youkaichao authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    c8a7e93 View commit details
    Browse the repository at this point in the history

  8. [Build/CI] Fixing Docker Hub quota issue. (vllm-project#7043)

    @Alexei-V-Ivanov-AMD
    Alexei-V-Ivanov-AMD authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    a72a424 View commit details
    Browse the repository at this point in the history

  9. [CI/Build] Update PyTorch to 2.4.0 (vllm-project#6951) 

    Co-authored-by: Michael Goin <[email protected]>
    @SageMoore @mgoin
    SageMoore and mgoin authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    7e0861b View commit details
    Browse the repository at this point in the history

  10. [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (vll… 

    …m-project#6992)
    @Isotr0py
    Isotr0py authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    2dd3437 View commit details
    Browse the repository at this point in the history

  11. [CI/Build] Remove sparseml requirement from testing (vllm-project#7037)

    @mgoin
    mgoin authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    fb3db61 View commit details
    Browse the repository at this point in the history

  12. [Bugfix] Lower gemma's unloaded_params exception to warning (vllm-pro… 

    …ject#7002)
    @mgoin
    mgoin authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f4fd390 View commit details
    Browse the repository at this point in the history

  13. [Models] Support Qwen model with PP (vllm-project#6974) 

    Signed-off-by: Muralidhar Andoorveedu <[email protected]>
    @andoorve
    andoorve authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    fc912e0 View commit details
    Browse the repository at this point in the history

  14. Update run-amd-test.sh (vllm-project#7044)

    @okakarpa
    okakarpa authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    562e580 View commit details
    Browse the repository at this point in the history

  15. [Misc] Support attention logits soft-capping with flash-attn (vllm-pr… 

    …oject#7022)
    @WoosukKwon
    WoosukKwon authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    805a8a7 View commit details
    Browse the repository at this point in the history

  16. [CI/Build][Bugfix] Fix CUTLASS header-only line (vllm-project#7034)

    @tlrmchlsmth
    tlrmchlsmth authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    6a11fdf View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. [Performance] Optimize get_seqs (vllm-project#7051)

    @WoosukKwon
    WoosukKwon authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    6ce01f3 View commit details
    Browse the repository at this point in the history

  2. [Kernel] Fix input for flashinfer prefill wrapper. (vllm-project#7008)

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    954f730 View commit details
    Browse the repository at this point in the history

  3. [mypy] Speed up mypy checking (vllm-project#7056)

    @DarkLight1337
    DarkLight1337 authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    3bb4b1e View commit details
    Browse the repository at this point in the history

  4. [ci][distributed] try to fix pp test (vllm-project#7054)

    @youkaichao
    youkaichao authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    2523577 View commit details
    Browse the repository at this point in the history

  5. Fix tracing.py (vllm-project#7065)

    @bong-furiosa
    bong-furiosa authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    cf2a1a4 View commit details
    Browse the repository at this point in the history

  6. [cuda][misc] remove error_on_invalid_device_count_status (vllm-projec… 

    …t#7069)
    @youkaichao
    youkaichao authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    660dea1 View commit details
    Browse the repository at this point in the history

  7. [Core] Comment out unused code in sampler (vllm-project#7023)

    @peng1999
    peng1999 authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    db35186 View commit details
    Browse the repository at this point in the history

  8. [Hardware][Intel CPU] Update torch 2.4.0 for CPU backend (vllm-projec… 

    …t#6931)
    @DamonFool
    DamonFool authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    c16eaac View commit details
    Browse the repository at this point in the history

  9. [ci] set timeout for test_oot_registration.py (vllm-project#7082)

    @youkaichao
    youkaichao authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    8069495 View commit details
    Browse the repository at this point in the history

  10. [CI/Build] Add support for Python 3.12 (vllm-project#7035)

    @mgoin
    mgoin authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    b482b9a View commit details
    Browse the repository at this point in the history

  11. [Misc] Disambiguate quantized types via a new ScalarType (vllm-projec… 

    …t#6396)
    @LucasWilkinson
    LucasWilkinson authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    a8d604c View commit details
    Browse the repository at this point in the history

  12. [Core] Pipeline parallel with Ray ADAG (vllm-project#6837) 

    Support pipeline-parallelism with Ray accelerated DAG.

Signed-off-by: Rui Qiao <[email protected]>
    @ruisearch42
    ruisearch42 authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    0530889 View commit details
    Browse the repository at this point in the history

  13. [Misc] Revive to use loopback address for driver IP (vllm-project#7091) 

    Signed-off-by: Rui Qiao <[email protected]>
    @ruisearch42
    ruisearch42 authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    22e718f View commit details
    Browse the repository at this point in the history

  14. [misc] add a flag to enable compile (vllm-project#7092)

    @youkaichao
    youkaichao authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    7089893 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. [ Frontend ] Multiprocessing for OpenAI Server with zeromq (vllm-pr… 

    …oject#6883)

Signed-off-by: Joe Runde <[email protected]>
Co-authored-by: Joe Runde <[email protected]>
Co-authored-by: Joe Runde <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
Co-authored-by: Simon Mo <[email protected]>
    @robertgshaw2-neuralmagic @joerunde
    @njhill @simon-mo
    5 people authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    ed812a7 View commit details
    Browse the repository at this point in the history

  2. [ci][distributed] shorten wait time if server hangs (vllm-project#7098)

    @youkaichao
    youkaichao authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    69ea15e View commit details
    Browse the repository at this point in the history

  3. [Frontend] Factor out chat message parsing (vllm-project#7055)

    @DarkLight1337
    DarkLight1337 authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    8c025fa View commit details
    Browse the repository at this point in the history

  4. [ci][distributed] merge distributed test commands (vllm-project#7097) 

    Co-authored-by: Cyrus Leung <[email protected]>
    @youkaichao @DarkLight1337
    youkaichao and DarkLight1337 authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    04e5583 View commit details
    Browse the repository at this point in the history

  5. [ci][distributed] disable ray dag tests (vllm-project#7099)

    @youkaichao
    youkaichao authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    a0d1645 View commit details
    Browse the repository at this point in the history

  6. [Model] Refactor and decouple weight loading logic for InternVL2 model ( 

    vllm-project#7067)
    @Isotr0py
    Isotr0py authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    0c25435 View commit details
    Browse the repository at this point in the history

  7. [Bugfix] Fix block table for seqs that have prefix cache hits (vllm-p… 

    …roject#7018)
    @zachzzc
    zachzzc authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    fb2c1c8 View commit details
    Browse the repository at this point in the history

  8. [LoRA] ReplicatedLinear support LoRA (vllm-project#7081)

    @jeejeelee
    jeejeelee authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    99d7cab View commit details
    Browse the repository at this point in the history

  9. [CI] Temporarily turn off H100 performance benchmark (vllm-project#7104)

    @KuntaiDu
    KuntaiDu authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    67d745c View commit details
    Browse the repository at this point in the history

  10. [ci][test] finalize fork_new_process_for_each_test (vllm-project#7114)

    @youkaichao
    youkaichao authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    44dcb52 View commit details
    Browse the repository at this point in the history

  11. [Frontend] Warn if user max_model_len is greater than derived `max_… 

    …model_len` (vllm-project#7080)

Signed-off-by: Jefferson Fialho <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
    @fialhocoelho @njhill
    fialhocoelho and njhill authored Aug 3, 2024
    Configuration menu
    Copy the full SHA
    825b044 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Support for guided decoding for offline LLM (vllm-project#6878) 

    Co-authored-by: Cyrus Leung <[email protected]>
    @kevinbu233 @DarkLight1337
    kevinbu233 and DarkLight1337 authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    654bc5c View commit details
    Browse the repository at this point in the history

  2. [misc] add zmq in collect env (vllm-project#7119)

    @youkaichao
    youkaichao authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    9fadc7b View commit details
    Browse the repository at this point in the history

  3. [core][misc] simply output processing with shortcut code path (vllm-p… 

    …roject#7117)
    @youkaichao
    youkaichao authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    83c644f View commit details
    Browse the repository at this point in the history

  4. [Model]Refactor MiniCPMV (vllm-project#7020) 

    Co-authored-by: Cyrus Leung <[email protected]>
    @jeejeelee @DarkLight1337
    jeejeelee and DarkLight1337 authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    179a6a3 View commit details
    Browse the repository at this point in the history

  5. [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size … 

    …to 1 when using MLPSpeculator (vllm-project#7105)

Signed-off-by: Thomas Parnell <[email protected]>
    @tdoublep
    tdoublep authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    b1c9aa3 View commit details
    Browse the repository at this point in the history

  6. [misc][distributed] improve libcudart.so finding (vllm-project#7127)

    @youkaichao
    youkaichao authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    16a1cc9 View commit details
    Browse the repository at this point in the history

  7. Clean up remaining Punica C information (vllm-project#7027)

    @jeejeelee
    jeejeelee authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    f80ab35 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. [Model] Add multi-image support for minicpmv (vllm-project#7122) 

    Co-authored-by: hezhihui <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
    @HwwwwwwwH @DarkLight1337
    3 people authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    7b86e7c View commit details
    Browse the repository at this point in the history

  2. [Frontend] Reapply "Factor out code for running uvicorn" (vllm-projec… 

    …t#7095)
    @DarkLight1337
    DarkLight1337 authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    cc08fc7 View commit details
    Browse the repository at this point in the history

  3. [Model] SiglipVisionModel ported from transformers (vllm-project#6942) 

    Co-authored-by: Roger Wang <[email protected]>
    @ChristopherCho @ywang96
    ChristopherCho and ywang96 authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    c0d8f16 View commit details
    Browse the repository at this point in the history

  4. [Speculative decoding] Add periodic log with time spent in proposal/s… 

    …coring/verification (vllm-project#6963)
    @cadedaniel
    cadedaniel authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    82a1b1a View commit details
    Browse the repository at this point in the history

  5. [SpecDecode] Support FlashInfer in DraftModelRunner (vllm-project#6926)

    @bong-furiosa
    bong-furiosa authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    e963045 View commit details
    Browse the repository at this point in the history

  6. [BugFix] Use IP4 localhost form for zmq bind (vllm-project#7163)

    @njhill
    njhill authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    003f8ee View commit details
    Browse the repository at this point in the history

  7. [BugFix] Use args.trust_remote_code (vllm-project#7121)

    @VastoLorde95
    VastoLorde95 authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    57f560a View commit details
    Browse the repository at this point in the history

  8. [Misc] Fix typo in GroupCoordinator.recv() (vllm-project#7167) 

    Signed-off-by: Rui Qiao <[email protected]>
    @ruisearch42
    ruisearch42 authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    997cf78 View commit details
    Browse the repository at this point in the history

  9. [Kernel] Update CUTLASS to 3.5.1 (vllm-project#7085)

    @tlrmchlsmth
    tlrmchlsmth authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    8571ac4 View commit details
    Browse the repository at this point in the history

  10. [CI/Build] Suppress divide-by-zero and missing return statement warni… 

    …ngs (vllm-project#7001)
    @tlrmchlsmth
    tlrmchlsmth authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    6e4852c View commit details
    Browse the repository at this point in the history

  11. [Bugfix][CI/Build] Fix CUTLASS FetchContent (vllm-project#7171)

    @tlrmchlsmth
    tlrmchlsmth authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    4cf1dc3 View commit details
    Browse the repository at this point in the history

  12. bump version to v0.5.4 (vllm-project#7139)

    @simon-mo
    simon-mo authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    4db5176 View commit details
    Browse the repository at this point in the history

  13. [ci][frontend] deduplicate tests (vllm-project#7101)

    @youkaichao
    youkaichao authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    dfb1a15 View commit details
    Browse the repository at this point in the history

  14. [Doc] [SpecDecode] Update MLPSpeculator documentation (vllm-project#7100 

    )

Signed-off-by: Thomas Parnell <[email protected]>
    @tdoublep
    tdoublep authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    789937a View commit details
    Browse the repository at this point in the history

  15. [Bugfix] Specify device when loading LoRA and embedding tensors (vllm… 

    …-project#7129)

Co-authored-by: Jacob Schein <[email protected]>
    @jischein
    jischein and Jacob Schein authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    89b8db6 View commit details
    Browse the repository at this point in the history

  16. [MISC] Use non-blocking transfer in prepare_input (vllm-project#7172)

    @comaniac
    comaniac authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ef527be View commit details
    Browse the repository at this point in the history

  17. [Core] Support loading GGUF model (vllm-project#5191) 

    Co-authored-by: Michael Goin <[email protected]>
    @Isotr0py @mgoin
    Isotr0py and mgoin authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    360bd67 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. [Build] Add initial conditional testing spec (vllm-project#6841)

    @simon-mo
    simon-mo authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    e3c664b View commit details
    Browse the repository at this point in the history

  2. [LoRA] Relax LoRA condition (vllm-project#7146)

    @jeejeelee
    jeejeelee authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    9118217 View commit details
    Browse the repository at this point in the history

  3. [Model] Support SigLIP encoder and alternative decoders for LLaVA mod… 

    …els (vllm-project#7153)

Co-authored-by: Roger Wang <[email protected]>
    @DarkLight1337 @ywang96
    DarkLight1337 and ywang96 authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    1f26efb View commit details
    Browse the repository at this point in the history

  4. [BugFix] Fix DeepSeek remote code (vllm-project#7178)

    @dsikka
    dsikka authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    a3bbbfa View commit details
    Browse the repository at this point in the history

  5. [ BugFix ] Fix ZMQ when VLLM_PORT is set (vllm-project#7205)

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    541c185 View commit details
    Browse the repository at this point in the history

  6. [Bugfix] add gguf dependency (vllm-project#7198) 

    Co-authored-by: katarzyna.papis <[email protected]>
    @kpapis
    kpapis and katarzyna.papis authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    00afc78 View commit details
    Browse the repository at this point in the history

  7. [SpecDecode] [Minor] Fix spec decode sampler tests (vllm-project#7183)

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    5c60c8c View commit details
    Browse the repository at this point in the history

  8. [Kernel] Add per-tensor and per-token AZP epilogues (vllm-project#5941) 

    Co-authored-by: Tyler Michael Smith <[email protected]>
    @ProExpertProg @tlrmchlsmth
    ProExpertProg and tlrmchlsmth authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    8d59dbb View commit details
    Browse the repository at this point in the history

  9. [Core] Optimize evictor-v2 performance (vllm-project#7193)

    @xiaobochen123
    xiaobochen123 authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    660470e View commit details
    Browse the repository at this point in the history

  10. [Core] Subclass ModelRunner to support cross-attention & encoder sequ… 

    …ences (towards eventual encoder/decoder model support) (vllm-project#4942)

Co-authored-by: Andrew Feldman <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
    @afeldman-nm @abf149 @njhill
    3 people authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    fd95e02 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (vllm-project#7225)

    @mgoin
    mgoin authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    f9a5600 View commit details
    Browse the repository at this point in the history

  2. [BugFix] Overhaul async request cancellation (vllm-project#7111)

    @njhill
    njhill authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    9a3f49a View commit details
    Browse the repository at this point in the history

  3. [Doc] Mock new dependencies for documentation (vllm-project#7245)

    @ywang96
    ywang96 authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    2385c8f View commit details
    Browse the repository at this point in the history

  4. [BUGFIX]: top_k is expected to be an integer. (vllm-project#7227)

    @Atllkks10
    Atllkks10 authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    7b26109 View commit details
    Browse the repository at this point in the history

  5. [Frontend] Gracefully handle missing chat template and fix CI failure (… 

    …vllm-project#7238)

Co-authored-by: Roger Wang <[email protected]>
    @DarkLight1337 @ywang96
    DarkLight1337 and ywang96 authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    66d617e View commit details
    Browse the repository at this point in the history

  6. [distributed][misc] add specialized method for cuda platform (vllm-pr… 

    …oject#7249)
    @youkaichao
    youkaichao authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    639159b View commit details
    Browse the repository at this point in the history

  7. [Misc] Refactor linear layer weight loading; introduce `BasevLLMParam… 

    …eter` and `weight_loader_v2` (vllm-project#5874)
    @dsikka
    dsikka authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    0f7052b View commit details
    Browse the repository at this point in the history

  8. [ BugFix ] Move zmq frontend to IPC instead of TCP (vllm-project#7222)

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5649857 View commit details
    Browse the repository at this point in the history

  9. Fixes typo in function name (vllm-project#7275) 

    Signed-off-by: Rafael Vasquez <[email protected]>
    @rafvasq
    rafvasq authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    ab0f5e2 View commit details
    Browse the repository at this point in the history

  10. [Bugfix] Fix input processor for InternVL2 model (vllm-project#7164) 

    Co-authored-by: Cyrus Leung <[email protected]>
    @Isotr0py @DarkLight1337
    Isotr0py and DarkLight1337 authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    b764547 View commit details
    Browse the repository at this point in the history

  11. [OpenVINO] migrate to latest dependencies versions (vllm-project#7251)

    @ilya-lavrenov
    ilya-lavrenov authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    80cbe10 View commit details
    Browse the repository at this point in the history

  12. [Doc] add online speculative decoding example (vllm-project#7243)

    @stas00
    stas00 authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    0e12cd6 View commit details
    Browse the repository at this point in the history

  13. [BugFix] Fix frontend multiprocessing hang (vllm-project#7217) 

    Signed-off-by: Max de Bayser <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
    @maxdebayser @robertgshaw2-neuralmagic
    maxdebayser and robertgshaw2-neuralmagic authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    fde47d3 View commit details
    Browse the repository at this point in the history

  14. [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (vllm-project#7219)

    @mgoin
    mgoin authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    5223199 View commit details
    Browse the repository at this point in the history

  15. [ci] Make building wheels per commit optional (vllm-project#7278) 

    Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    469b3bc View commit details
    Browse the repository at this point in the history

  16. [Bugfix] Fix gptq failure on T4s (vllm-project#7264)

    @LucasWilkinson
    LucasWilkinson authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    311f743 View commit details
    Browse the repository at this point in the history

  17. [FrontEnd] Make merge_async_iterators is_cancelled arg optional (v… 

    …llm-project#7282)
    @njhill
    njhill authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    fc1493a View commit details
    Browse the repository at this point in the history

  18. [Doc] Update supported_hardware.rst (vllm-project#7276)

    @mgoin
    mgoin authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    6d94420 View commit details
    Browse the repository at this point in the history

  19. [Kernel] Fix Flashinfer Correctness (vllm-project#7284)

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    e53dfd3 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. [Misc] Fix typos in scheduler.py (vllm-project#7285) 

    Signed-off-by: Rui Qiao <[email protected]>
    @ruisearch42
    ruisearch42 authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    7467096 View commit details
    Browse the repository at this point in the history

  2. [Frontend] remove max_num_batched_tokens limit for lora (vllm-project… 

    …#7288)
    @NiuBlibing
    NiuBlibing authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    48abee9 View commit details
    Browse the repository at this point in the history

  3. [Bugfix] Fix LoRA with PP (vllm-project#7292)

    @andoorve
    andoorve authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    6dffa4b View commit details
    Browse the repository at this point in the history

  4. [Model] Rename MiniCPMVQwen2 to MiniCPMV2.6 (vllm-project#7273)

    @jeejeelee
    jeejeelee authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    757ac70 View commit details
    Browse the repository at this point in the history

  5. [Bugfix][Kernel] Increased atol to fix failing tests (vllm-project#7305)

    @ProExpertProg
    ProExpertProg authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    5fb4a3f View commit details
    Browse the repository at this point in the history

  6. [Frontend] Kill the server on engine death (vllm-project#6594) 

    Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
    @joerunde
    joerunde authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    21b9c49 View commit details
    Browse the repository at this point in the history

  7. [Bugfix][fast] Fix the get_num_blocks_touched logic (vllm-project#6849)

    @zachzzc
    zachzzc authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    782e53a View commit details
    Browse the repository at this point in the history

  8. [Doc] Put collect_env issue output in a <detail> block (vllm-project#… 

    …7310)
    @mgoin
    mgoin authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e14fb22 View commit details
    Browse the repository at this point in the history

  9. [CI/Build] Dockerfile.cpu improvements (vllm-project#7298)

    @dtrifiro
    dtrifiro authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e904576 View commit details
    Browse the repository at this point in the history

  10. [Bugfix] Fix new Llama3.1 GGUF model loading (vllm-project#7269)

    @Isotr0py
    Isotr0py authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8334c39 View commit details
    Browse the repository at this point in the history

  11. [Misc] Temporarily resolve the error of BitAndBytes (vllm-project#7308)

    @jeejeelee
    jeejeelee authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    a049b10 View commit details
    Browse the repository at this point in the history

  12. Add Skywork AI as Sponsor (vllm-project#7314)

    @simon-mo
    simon-mo authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    5923532 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. [TPU] Add Load-time W8A16 quantization for TPU Backend (vllm-project#… 

    …7005)
    @lsy323
    lsy323 authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    0fa1490 View commit details
    Browse the repository at this point in the history

  2. [Core] Support serving encoder/decoder models (vllm-project#7258)

    @DarkLight1337
    DarkLight1337 authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7eb4a51 View commit details
    Browse the repository at this point in the history

  3. [TPU] Fix dockerfile.tpu (vllm-project#7331)

    @WoosukKwon
    WoosukKwon authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    73388c0 View commit details
    Browse the repository at this point in the history

  4. [Performance] Optimize e2e overheads: Reduce python allocations (vllm… 

    …-project#7162)
    @alexm-neuralmagic
    alexm-neuralmagic authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    e02ac55 View commit details
    Browse the repository at this point in the history

  5. [Bugfix] Fix speculative decoding with MLPSpeculator with padded voca… 

    …bulary (vllm-project#7218)

Signed-off-by: Travis Johnson <[email protected]>
    @tjohnson31415
    tjohnson31415 authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    99b4cf5 View commit details
    Browse the repository at this point in the history

  6. [Speculative decoding] [Multi-Step] decouple should_modify_greedy_pro… 

    …bs_inplace (vllm-project#6971)
    @SolitaryThinker
    SolitaryThinker authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    57b7be0 View commit details
    Browse the repository at this point in the history

  7. [Core] Streamline stream termination in AsyncLLMEngine (vllm-projec… 

    …t#7336)
    @njhill
    njhill authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b4e9528 View commit details
    Browse the repository at this point in the history

  8. [Model][Jamba] Mamba cache single buffer (vllm-project#6739) 

    Co-authored-by: Mor Zusman <[email protected]>
    @mzusman
    mzusman and Mor Zusman authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    07ab160 View commit details
    Browse the repository at this point in the history

  9. [VLM][Doc] Add stop_token_ids to InternVL example (vllm-project#7354)

    @Isotr0py
    Isotr0py authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    67abdbb View commit details
    Browse the repository at this point in the history

  10. [Performance] e2e overheads reduction: Small followup diff (vllm-proj… 

    …ect#7364)
    @alexm-neuralmagic
    alexm-neuralmagic authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    fc7b8d1 View commit details
    Browse the repository at this point in the history

  11. [Bugfix] Fix reinit procedure in ModelInputForGPUBuilder (vllm-projec… 

    …t#7360)
    @alexm-neuralmagic
    alexm-neuralmagic authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    74af2bb View commit details
    Browse the repository at this point in the history

  12. [Frontend] Support embeddings in the run_batch API (vllm-project#7132) 

    Co-authored-by: Simon Mo <[email protected]>
    @pooyadavoodi @simon-mo
    pooyadavoodi and simon-mo authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    249b882 View commit details
    Browse the repository at this point in the history

  13. [Bugfix] Fix ITL recording in serving benchmark (vllm-project#7372)

    @ywang96
    ywang96 authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    70d268a View commit details
    Browse the repository at this point in the history

  14. [Core] Add span metrics for model_forward, scheduler and sampler time (… 

    …vllm-project#7089)
    @sfc-gh-mkeralapura
    sfc-gh-mkeralapura authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    933790c View commit details
    Browse the repository at this point in the history

  15. [Bugfix] Fix PerTensorScaleParameter weight loading for fused models ( 

    vllm-project#7376)
    @dsikka
    dsikka authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    5c6c54d View commit details
    Browse the repository at this point in the history

  16. [Misc] Add numpy implementation of compute_slot_mapping (vllm-proje… 

    …ct#7377)
    @Yard1
    Yard1 authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    999ef0b View commit details
    Browse the repository at this point in the history

  17. [Core] Fix edge case in chunked prefill + block manager v2 (vllm-proj… 

    …ect#7380)
    @cadedaniel
    cadedaniel authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    baa2402 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. [Bugfix] Fix phi3v batch inference when images have different aspect … 

    …ratio (vllm-project#7392)
    @Isotr0py
    Isotr0py authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    4c5d8e8 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. [TPU] Use mark_dynamic to reduce compilation time (vllm-project#7340)

    @WoosukKwon
    WoosukKwon authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    90bab18 View commit details
    Browse the repository at this point in the history

  2. Updating LM Format Enforcer version to v0.10.6 (vllm-project#7189)

    @noamgat
    noamgat authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    4fb7b52 View commit details
    Browse the repository at this point in the history

  3. [core] [2/N] refactor worker_base input preparation for multi-step (v… 

    …llm-project#7387)
    @SolitaryThinker
    SolitaryThinker authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    c08e2b3 View commit details
    Browse the repository at this point in the history

  4. [CI/Build] build on empty device for better dev experience (vllm-proj… 

    …ect#4773)
    @tomeras91
    tomeras91 authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    3860879 View commit details
    Browse the repository at this point in the history

  5. [Doc] building vLLM with VLLM_TARGET_DEVICE=empty (vllm-project#7403)

    @tomeras91
    tomeras91 authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    02b1988 View commit details
    Browse the repository at this point in the history

  6. [misc] add commit id in collect env (vllm-project#7405)

    @youkaichao
    youkaichao authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    6c8e595 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. [Docs] Update readme (vllm-project#7316)

    @simon-mo
    simon-mo authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    f020a62 View commit details
    Browse the repository at this point in the history

  2. [CI/Build] Minor refactoring for vLLM assets (vllm-project#7407)

    @ywang96
    ywang96 authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    86ab567 View commit details
    Browse the repository at this point in the history

  3. [Kernel] Flashinfer correctness fix for v0.1.3 (vllm-project#7319)

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    ec2affa View commit details
    Browse the repository at this point in the history

  4. [Core][VLM] Support image embeddings as input (vllm-project#6613)

    @ywang96
    ywang96 authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    e6e42e4 View commit details
    Browse the repository at this point in the history

  5. [Frontend] Disallow passing model as both argument and option (vllm… 

    …-project#7347)
    @DarkLight1337
    DarkLight1337 authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    24154f8 View commit details
    Browse the repository at this point in the history

  6. [CI/Build] bump Dockerfile.neuron image base, use public ECR (vllm-pr… 

    …oject#6832)
    @dtrifiro
    dtrifiro authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    d2bc451 View commit details
    Browse the repository at this point in the history

  7. [Bugfix] Fix logit soft cap in flash-attn backend (vllm-project#7425)

    @WoosukKwon
    WoosukKwon authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    cfba4de View commit details
    Browse the repository at this point in the history

  8. [ci] Entrypoints run upon changes in vllm/ (vllm-project#7423) 

    Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    65950e8 View commit details
    Browse the repository at this point in the history

  9. [ci] Cancel fastcheck run when PR is marked ready (vllm-project#7427) 

    Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    9b3e2ed View commit details
    Browse the repository at this point in the history

  10. [ci] Cancel fastcheck when PR is ready (vllm-project#7433) 

    Signed-off-by: kevin <[email protected]>
    @khluu
    khluu authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    1137f34 View commit details
    Browse the repository at this point in the history

  11. [Misc] Use scalar type to dispatch to different gptq_marlin kernels (… 

    …vllm-project#7323)
    @LucasWilkinson
    LucasWilkinson authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    6aa33cb View commit details
    Browse the repository at this point in the history

  12. [Core] Consolidate GB constant and enable float GB arguments (vllm-… 

    …project#7416)
    @DarkLight1337
    DarkLight1337 authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    4ddc474 View commit details
    Browse the repository at this point in the history

  13. [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefi… 

    …ll Triton Kernel (vllm-project#7208)

Co-authored-by: Cody Yu <[email protected]>
    @jon-chuang @comaniac
    jon-chuang and comaniac authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    a046f86 View commit details
    Browse the repository at this point in the history

  14. [Bugfix] Handle PackageNotFoundError when checking for xpu version (v… 

    …llm-project#7398)
    @sasha0552
    sasha0552 authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    91294d5 View commit details
    Browse the repository at this point in the history

  15. [CI/Build] bump minimum cmake version (vllm-project#6999)

    @dtrifiro
    dtrifiro authored Aug 12, 2024
    Configuration menu
    Copy the full SHA
    774cd1d View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. [Core] Shut down aDAG workers with clean async llm engine exit (vllm-… 

    …project#7224)

Signed-off-by: Rui Qiao <[email protected]>
    @ruisearch42
    ruisearch42 authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    198d6a2 View commit details
    Browse the repository at this point in the history

  2. [mypy] Misc. typing improvements (vllm-project#7417)

    @DarkLight1337
    DarkLight1337 authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    9ba85bc View commit details
    Browse the repository at this point in the history

  3. [Misc] improve logits processors logging message (vllm-project#7435)

    @aw632
    aw632 authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    97a6be9 View commit details
    Browse the repository at this point in the history

  4. [ci] Remove fast check cancel workflow (vllm-project#7455)

    @khluu
    khluu authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    5469146 View commit details
    Browse the repository at this point in the history

  5. [Bugfix] Fix weight loading for Chameleon when TP>1 (vllm-project#7410)

    @DarkLight1337
    DarkLight1337 authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    7025b11 View commit details
    Browse the repository at this point in the history

  6. [hardware] unify usage of is_tpu to current_platform.is_tpu() (vllm-p… 

    …roject#7102)
    @youkaichao
    youkaichao authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4d2dc50 View commit details
    Browse the repository at this point in the history

  7. [TPU] Suppress import custom_ops warning (vllm-project#7458)

    @WoosukKwon
    WoosukKwon authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    d6e634f View commit details
    Browse the repository at this point in the history

  8. Revert "[Doc] Update supported_hardware.rst (vllm-project#7276)" (vll… 

    …m-project#7467)
    @WoosukKwon
    WoosukKwon authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    e20233d View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'upstream/main' into HEAD

    @kzawora-intel
    kzawora-intel committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    f328349 View commit details
    Browse the repository at this point in the history

  10. format.sh

    @kzawora-intel
    kzawora-intel committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    212e87e View commit details
    Browse the repository at this point in the history