BetterTransformers Support For DeBERTaV3

[serdarildercaglar]

Feature request

Hello, I often use libraries developed by Huggingface in my projects.
I follow your innovations and I want to implement them in my projects. BetterTransformers intrigued me and I want to try it with Debertav3. When I tried BetterTransformer with DeBERTaV3, It gave NotImplementedError: The Better Transformers implementation for the model DebertaV2ForTokenClassification has not been implemented yet. Please open an issue requesting the addition of this model with its BetterTransformer implementation.
When can you provide BetterTransformer support for Debertav3?

You are doing great work, thank you

Motivation

I look forward to trying Better Transformer and seeing results.

Your contribution

Thanks for all

[younesbelkada]

Hi @forrestfaraday
Thank you very much for your message and your interest in BetterTransformer integration! 🎉
Unfortunately DeBERTa uses a special type of attention mechanism, which is not currently supported :/ Check out the specific conditions on the documentation here. Is there any other architecture you might be interested in? We are also planning to open "Good first issues" for contributing in BetterTransformer in case you are interested

cc @michaelbenayoun @fxmarty

#488

[serdarildercaglar]

Thank you for your attention to my issue. Recently I have focused on BetterTransformer and optimum[ONNX] specifically to speed up deberta models.
I sincerely support your work.
Your success makes it easy for us to do our job.
Best wishes

[younesbelkada]

Thank you very much for your words @forrestfaraday !