Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

Commit

Permalink
Merge pull request #432 from Sanctum-AI/fix-rewind
Browse files Browse the repository at this point in the history
Fix rewind crash
  • Loading branch information
philpax authored Oct 31, 2023
2 parents db31c2d + cd97c9d commit fb48749
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion crates/llm-base/src/inference_session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ impl InferenceSession {
}

// Remove the tokens from self.tokens.
let token_start = self.n_past - num;
let token_start = self.tokens.len() - num;
let deleted_tokens: Vec<_> = self.tokens.drain(token_start..).collect();

// Remove the corresponding chars from decoded
Expand Down

0 comments on commit fb48749

Please sign in to comment.