Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add online trainers #204

Merged
merged 74 commits into from
Sep 13, 2024
Merged

Add online trainers #204

merged 74 commits into from
Sep 13, 2024

Commits on Jul 24, 2024

  1. Add online trainers

    @vwxyzjn
    vwxyzjn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    e352062 View commit details
    Browse the repository at this point in the history

  2. black / isort

    @vwxyzjn
    vwxyzjn committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    84645bb View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. push online trainers

    @vwxyzjn
    vwxyzjn committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3e492ff View commit details
    Browse the repository at this point in the history

  2. black

    @vwxyzjn
    vwxyzjn committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    8e49eb6 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. add reward trainer

    @vwxyzjn
    vwxyzjn committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    74ae736 View commit details
    Browse the repository at this point in the history

  2. push changes

    @vwxyzjn
    vwxyzjn committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    5e32e2f View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. format

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    7a5b5d8 View commit details
    Browse the repository at this point in the history

  2. bug fix

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    07192cc View commit details
    Browse the repository at this point in the history

  3. only do visualize in main process

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    51f72b0 View commit details
    Browse the repository at this point in the history

  4. quick fix

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    3d1e6b9 View commit details
    Browse the repository at this point in the history

  5. black

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    84cdd98 View commit details
    Browse the repository at this point in the history

  6. update docs even more

    @vwxyzjn
    vwxyzjn committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    00af870 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. formatting

    @vwxyzjn
    vwxyzjn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    8e2228d View commit details
    Browse the repository at this point in the history

  2. fix eval logging

    @vwxyzjn
    vwxyzjn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    72c5bf0 View commit details
    Browse the repository at this point in the history

  3. add ds config

    @vwxyzjn
    vwxyzjn committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    4175535 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Merge branch 'main' into reward-trainer

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    eb999ea View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into reward-trainer

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3d2d0ae View commit details
    Browse the repository at this point in the history

  3. maybe use ai2 wandb entity

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8c624b8 View commit details
    Browse the repository at this point in the history

  4. quick change

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ca9cbc0 View commit details
    Browse the repository at this point in the history

  5. remove unused print

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    3f14f24 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9f0c8c2 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'reward-trainer' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e69d5ec View commit details
    Browse the repository at this point in the history

  8. remove online sft trainer; doesn't work

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ed27422 View commit details
    Browse the repository at this point in the history

  9. make the dataset column configurable; remove eos from prompt token;

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    10a617b View commit details
    Browse the repository at this point in the history

  10. Merge branch 'reward-trainer' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    fa8b63c View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. more self-contained online DPO

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    df2d3f9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into reward-trainer

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    d479c59 View commit details
    Browse the repository at this point in the history

  3. add a generate collator

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f58c4d9 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'reward-trainer' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9e81514 View commit details
    Browse the repository at this point in the history

  5. fix bugs with deepspeed

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    41799ee View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    885bf90 View commit details
    Browse the repository at this point in the history

  7. format

    @vwxyzjn
    vwxyzjn committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    a7aa521 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. push changes

    @vwxyzjn
    vwxyzjn committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    d7b443d View commit details
    Browse the repository at this point in the history

  2. remove unused files

    @vwxyzjn
    vwxyzjn committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    73fb54a View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. push changes

    @vwxyzjn
    vwxyzjn committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    bc7cd7e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    7b9404a View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. push

    @vwxyzjn
    vwxyzjn committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    3f36f65 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2024

  1. Refactor using VLLMGenerator

    @vwxyzjn
    vwxyzjn committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    b907712 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. push changes

    @vwxyzjn
    vwxyzjn committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e5e017c View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Update reward modeling docs

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    8b06aaf View commit details
    Browse the repository at this point in the history

  2. mix reward model dataset

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    67afc01 View commit details
    Browse the repository at this point in the history

  3. push

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    4526f2a View commit details
    Browse the repository at this point in the history

  4. quick change

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    5fa9294 View commit details
    Browse the repository at this point in the history

  5. support gradient checkpointing

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    bfb9aed View commit details
    Browse the repository at this point in the history

  6. test

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    46fb998 View commit details
    Browse the repository at this point in the history

  7. quick change

    @vwxyzjn
    vwxyzjn committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    6d8b6f7 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. push changes

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6daffd1 View commit details
    Browse the repository at this point in the history

  2. push changes

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    c554283 View commit details
    Browse the repository at this point in the history

  3. push

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    5641385 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    3bf39cd View commit details
    Browse the repository at this point in the history

  5. remove unused

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e016043 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'update-docs' into online-trainers

    @vwxyzjn
    vwxyzjn committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    381393b View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. push latest code

    @vwxyzjn
    vwxyzjn committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    9aa30e1 View commit details
    Browse the repository at this point in the history

  2. Add online DPO docs

    @vwxyzjn
    vwxyzjn committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    eadfd96 View commit details
    Browse the repository at this point in the history

  3. style and formt

    @vwxyzjn
    vwxyzjn committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    1b4c2bd View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. update docs

    @vwxyzjn
    vwxyzjn committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    e36535a View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. handle preemption

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    6ae5642 View commit details
    Browse the repository at this point in the history

  2. quick change

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f09b85f View commit details
    Browse the repository at this point in the history

  3. push

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    e459a3d View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1a70317 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into online-trainers

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    eb07d34 View commit details
    Browse the repository at this point in the history

  6. remove changes

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    070883d View commit details
    Browse the repository at this point in the history

  7. quick change

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    7e77b30 View commit details
    Browse the repository at this point in the history

  8. quick change

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    423877d View commit details
    Browse the repository at this point in the history

  9. update mason

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    a03d565 View commit details
    Browse the repository at this point in the history

  10. quick change

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    db171f0 View commit details
    Browse the repository at this point in the history

  11. push changes

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    9c3e38e View commit details
    Browse the repository at this point in the history

  12. quick change

    @vwxyzjn
    vwxyzjn committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f8b2898 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. push changes

    @vwxyzjn
    vwxyzjn committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    d104cda View commit details
    Browse the repository at this point in the history

  2. quick change

    @vwxyzjn
    vwxyzjn committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    368e402 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. quick change

    @vwxyzjn
    vwxyzjn committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    34d5bbc View commit details
    Browse the repository at this point in the history

  2. style quality

    @vwxyzjn
    vwxyzjn committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    6f3e02c View commit details
    Browse the repository at this point in the history

  3. add comment

    @vwxyzjn
    vwxyzjn committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7e448c0 View commit details
    Browse the repository at this point in the history

  4. quick change

    @vwxyzjn
    vwxyzjn committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c42a2b1 View commit details
    Browse the repository at this point in the history