Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add more options to ViT #184

Draft
wants to merge 53 commits into
base: master
Choose a base branch
from
Draft

Add more options to ViT #184

wants to merge 53 commits into from

Commits on Jun 27, 2022

  1. Add DropBlock

    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    cd0edef View commit details
    Browse the repository at this point in the history

  2. Initial commit for new ResNet API

    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    271b430 View commit details
    Browse the repository at this point in the history

  3. Cleanup

    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    866dbcc View commit details
    Browse the repository at this point in the history

  4. Get some stuff to work 

    1. Some docs
2. Basic tests for ResNet and ResNeXt now pass
    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    a038ff8 View commit details
    Browse the repository at this point in the history

  5. Tweaks - I

    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    de079bc View commit details
    Browse the repository at this point in the history

  6. Make pretrain condition explicit

    @theabhirath
    theabhirath committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    4fa28d4 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2022

  1. More declarative interface for ResNet 

    1. Less keywords for the user to worry about
2. Delete `ResNeXt` just for now
    @theabhirath
    theabhirath committed Jun 28, 2022
    Configuration menu
    Copy the full SHA
    7846f8b View commit details
    Browse the repository at this point in the history

  2. Make DropBlock really work

    @theabhirath
    theabhirath committed Jun 28, 2022
    Configuration menu
    Copy the full SHA
    a1d5ddc View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. Construct the stem outside and pass it into resnet 

    `downsample_args` is actually redundant
    @theabhirath
    theabhirath committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    3be1d81 View commit details
    Browse the repository at this point in the history

  2. Add ResNeXt back 

    Also add tests. A lot of tests
    @theabhirath
    theabhirath committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    16cbcd0 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2022

  1. Enable CI for Windows

    @theabhirath
    theabhirath committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    e5294ec View commit details
    Browse the repository at this point in the history

  2. Add more general implementation of SE layer 

    Also

1. Tweaks - II : Formatting + some docs
2. Groundwork for abstracting out the classifier
    @theabhirath
    theabhirath committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    a439bdf View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2022

  1. Tweaks III + Some more docs 

    1. Reorganise layer imports for easy access
2. Get pooling to work
    @theabhirath
    theabhirath committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    441ade8 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2022

  1. Fix DropBlock on the GPU

    @theabhirath
    theabhirath committed Jul 3, 2022
    Configuration menu
    Copy the full SHA
    5d059f5 View commit details
    Browse the repository at this point in the history

  2. Add SEResNet and SEResNeXt 

    So much GC, might as well have a function for it
    @theabhirath
    theabhirath committed Jul 3, 2022
    Configuration menu
    Copy the full SHA
    226e96a View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2022

  1. More docs, more tweaks

    @theabhirath
    theabhirath committed Jul 4, 2022
    Configuration menu
    Copy the full SHA
    3a4ffbf View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. More aggressive GC 

    Co-authored-by: Brian Chen <[email protected]>
    @theabhirath @ToucheSir
    theabhirath and ToucheSir authored Jul 8, 2022
    Configuration menu
    Copy the full SHA
    2f755cf View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2022

  1. Tweaks don't stop 

    Neither does formatting, unfortunately. Also refactor `classifier` to separate out FC-layer creation and pooling
    @theabhirath
    theabhirath committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    5ba4b84 View commit details
    Browse the repository at this point in the history

  2. Reorganisation and formatting 

    It really does never stop

Co-Authored-By: Kyle Daruwalla <[email protected]>
    @theabhirath @darsnack
    theabhirath and darsnack committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    aaf2abb View commit details
    Browse the repository at this point in the history

  3. Refactor shortcut connections

    @theabhirath
    theabhirath committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    326f36c View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2022

  1. Generalise resnet further

    @theabhirath
    theabhirath committed Jul 10, 2022
    Configuration menu
    Copy the full SHA
    4e01443 View commit details
    Browse the repository at this point in the history

  2. Documentation 

    And make `downsample_opts` a smidge easier to work with. Also, a wee bit o' formatting and cleanup.
    @theabhirath
    theabhirath committed Jul 10, 2022
    Configuration menu
    Copy the full SHA
    e8d3488 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. Add classifier and backbone methods

    @theabhirath
    theabhirath committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    92ed4fa View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2022

  1. Refactor of resnet core

    @theabhirath
    theabhirath committed Jul 17, 2022
    Configuration menu
    Copy the full SHA
    96a7d31 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2022

  1. Add DropBlock

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    9540299 View commit details
    Browse the repository at this point in the history

  2. Initial commit for new ResNet API

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    588d703 View commit details
    Browse the repository at this point in the history

  3. Cleanup

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    2a5d0cc View commit details
    Browse the repository at this point in the history

  4. Get some stuff to work 

    1. Some docs
2. Basic tests for ResNet and ResNeXt now pass
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    07c1e95 View commit details
    Browse the repository at this point in the history

  5. Tweaks - I

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    2e88201 View commit details
    Browse the repository at this point in the history

  6. Make pretrain condition explicit

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    01eaa8b View commit details
    Browse the repository at this point in the history

  7. More declarative interface for ResNet 

    1. Less keywords for the user to worry about
2. Delete `ResNeXt` just for now
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    546b131 View commit details
    Browse the repository at this point in the history

  8. Make DropBlock really work

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    3f45f27 View commit details
    Browse the repository at this point in the history

  9. Construct the stem outside and pass it into resnet 

    `downsample_args` is actually redundant
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    f373f45 View commit details
    Browse the repository at this point in the history

  10. Add ResNeXt back 

    Also add tests. A lot of tests
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    51d0757 View commit details
    Browse the repository at this point in the history

  11. Add more general implementation of SE layer 

    Also

1. Tweaks - II : Formatting + some docs
2. Groundwork for abstracting out the classifier
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    106f260 View commit details
    Browse the repository at this point in the history

  12. Tweaks III + Some more docs 

    1. Reorganise layer imports for easy access
2. Get pooling to work
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    7147309 View commit details
    Browse the repository at this point in the history

  13. Fix DropBlock on the GPU

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    7ed20d4 View commit details
    Browse the repository at this point in the history

  14. Add SEResNet and SEResNeXt 

    So much GC, might as well have a function for it
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    f0051b7 View commit details
    Browse the repository at this point in the history

  15. More docs, more tweaks

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    e5d2295 View commit details
    Browse the repository at this point in the history

  16. More aggressive GC 

    Co-authored-by: Brian Chen <[email protected]>
    @theabhirath @ToucheSir
    theabhirath and ToucheSir committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    4a91fc4 View commit details
    Browse the repository at this point in the history

  17. Tweaks don't stop 

    Neither does formatting, unfortunately. Also refactor `classifier` to separate out FC-layer creation and pooling
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    cf538bb View commit details
    Browse the repository at this point in the history

  18. Reorganisation and formatting 

    It really does never stop

Co-Authored-By: Kyle Daruwalla <[email protected]>
    @theabhirath @darsnack
    theabhirath and darsnack committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    5be45ef View commit details
    Browse the repository at this point in the history

  19. Refactor shortcut connections

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    1e509df View commit details
    Browse the repository at this point in the history

  20. Generalise resnet further

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    e4930f1 View commit details
    Browse the repository at this point in the history

  21. Documentation 

    And make `downsample_opts` a smidge easier to work with. Also, a wee bit o' formatting and cleanup.
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    80bdcde View commit details
    Browse the repository at this point in the history

  22. Add classifier and backbone methods

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    ab37901 View commit details
    Browse the repository at this point in the history

  23. Refactor of resnet core

    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    68abbb7 View commit details
    Browse the repository at this point in the history

  24. Refactor of resnet core II 

    Closures is the name of the game
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    7ad362b View commit details
    Browse the repository at this point in the history

  25. Merge branch 'resnet-plus' of https://github.com/theabhirath/Metalhea… 

    …d.jl into resnet-plus
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    93fb500 View commit details
    Browse the repository at this point in the history

  26. Allow prenorm 

    Simplify `conv_bn` to `conv_norm` and use it
    @theabhirath
    theabhirath committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    13ed5ac View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2022

  1. Cleanup

    @theabhirath
    theabhirath committed Jul 23, 2022
    Configuration menu
    Copy the full SHA
    6c005d3 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2022

  1. Reorganisation 

    And some more formatting
    @theabhirath
    theabhirath committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    bd443f1 View commit details
    Browse the repository at this point in the history

  2. Add prenorm/postnorm option to ViT 

    Also
1. Make class tokens optional
2. Allow class tokens to be before positional embedding as in DeIT-III
    @theabhirath
    theabhirath committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    6a91485 View commit details
    Browse the repository at this point in the history