New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问目前支持qwen2吗？ #936

Open

Zheng-Jay opened this issue Sep 24, 2024 · 3 comments

Zheng-Jay commented Sep 24, 2024

我看文档里只写支持到qwen1.5，但是issue里不少人有用在qwen2上？

Author

Zheng-Jay commented Sep 24, 2024

我想在qwen2上用序列并行训长文本

shiningliang commented Oct 6, 2024

Sequence parallel needs transformers <4.43. Same issue in #935

Author

Zheng-Jay commented Oct 14, 2024

Sequence parallel needs transformers <4.43. Same issue in #935

训了一版，不过loss看着不太正常，性能也没提升

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment