v0.0.7

github-actions released this 28 Jun 10:15

· 0 commits to 95f507fdbab2ab1d73918ca95d8cbbca67e96ec8 since this release

0.0.7 (2024-06-28)

Breaking Changes

batch_decode_with_padded_kv_cache was removed, we encourage user to use BatchDecodeWithPagedKVCacheWrapper instead. (#343)

Bugfix

fix the forward_return_lse function in BatchPrefillWithRaggedKVCache class (#337)
fix the scheduler behavior of large page size (#333)

Features

customize logits_soft_cap value (#339) (a2498f5)

Performance Improvements

change minimal kv_chunk_size back to 128 (#329) (f237f5f)
more options for kv tile size (#336) (bf2a6c7)

Assets 27