Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

Add "context swap" functions to session and add "decoded_tokens" to snapshot read/write #424

Merged
merged 3 commits into from
Nov 12, 2023

Conversation

Andreybest
Copy link

Good day! Thank you for this awesome lib.
I've added feature for "context swap" similar on how is one made in llama.cpp cli. And a small fix to include "decoded_tokens" in snapshots
Im not sure if my code can go with all rust coder standards (coding for more than a month in rust). Please let me know if something needs a change.

@philpax philpax changed the base branch from main to develop November 12, 2023 20:58
@philpax philpax merged commit 2c127df into rustformers:develop Nov 12, 2023
10 of 14 checks passed
@philpax
Copy link
Collaborator

philpax commented Nov 12, 2023

Hi! Sorry for the very late response - I'm catching up on pending PRs by merging them into a develop branch so that they can be unified without impacting main.

Great work - the only changes I'd make are to the interface, but I figured I'd merge your code as-is because I need to redesign the overall interface anyway. Much appreciated 🙂

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants