Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Implement LayerSkip #2

Draft
wants to merge 19 commits into
base: paper_default
Choose a base branch
from
Draft

[WIP] Implement LayerSkip #2

wants to merge 19 commits into from

Commits on Jul 8, 2024

  1. set probabilities to be a list

    @mostafaelhoushi
    mostafaelhoushi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    68e83ea View commit details
    Browse the repository at this point in the history

  2. use setter wrapper for drop_p

    @mostafaelhoushi
    mostafaelhoushi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    671779d View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. implement layer wise schedule for dropout

    @mostafaelhoushi
    mostafaelhoushi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    f3f3591 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. ensure layer drop condition works

    @mostafaelhoushi
    mostafaelhoushi committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    1c77e33 View commit details
    Browse the repository at this point in the history

  2. start early_exit_loss.py

    @mostafaelhoushi
    mostafaelhoushi committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    1900596 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. get early exit loss to work with seamless

    @mostafaelhoushi
    mostafaelhoushi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    3d806c4 View commit details
    Browse the repository at this point in the history

  2. fix exit hooks

    @mostafaelhoushi
    mostafaelhoushi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    e138eae View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. add profiling code for SamplingGenerator

    @mostafaelhoushi
    mostafaelhoushi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    3e8cd67 View commit details
    Browse the repository at this point in the history

  2. add other loss scaling types and make uniform default

    @mostafaelhoushi
    mostafaelhoushi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    baebda3 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. start speculative decoding

    @mostafaelhoushi
    mostafaelhoushi committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    dcaf66e View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2024

  1. end-to-end speculative decoding but not correct

    @mostafaelhoushi
    mostafaelhoushi committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    9c4f52b View commit details
    Browse the repository at this point in the history

  2. create functions for prefill for draft model

    @mostafaelhoushi
    mostafaelhoushi committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    3721aca View commit details
    Browse the repository at this point in the history

  3. cleanup and get draft model to actually decode

    @mostafaelhoushi
    mostafaelhoushi committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    5a2e021 View commit details
    Browse the repository at this point in the history

  4. verify k_speculate tokens

    @mostafaelhoushi
    mostafaelhoushi committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    8aa1a91 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2024

  1. remove scores calculation for draft prefill

    @mostafaelhoushi
    mostafaelhoushi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    5480af5 View commit details
    Browse the repository at this point in the history

  2. make verification step return vocab

    @mostafaelhoushi
    mostafaelhoushi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    8331df2 View commit details
    Browse the repository at this point in the history

  3. calculate q p as per gpt-fast speculative decoding algorithm

    @mostafaelhoushi
    mostafaelhoushi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    a844e99 View commit details
    Browse the repository at this point in the history

  4. fix shapes of tensors

    @mostafaelhoushi
    mostafaelhoushi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    4ac712b View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. add more changes

    @mostafaelhoushi
    mostafaelhoushi committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e6ce2e7 View commit details
    Browse the repository at this point in the history