Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

relu6实现有误 #166

Open
MARD1NO opened this issue Dec 23, 2020 · 3 comments
Open

relu6实现有误 #166

MARD1NO opened this issue Dec 23, 2020 · 3 comments

Comments

@MARD1NO
Copy link
Contributor

MARD1NO commented Dec 23, 2020

使用 flow.clip(x, 0, 6) 数值表现是跟 Relu6一致的
但是在求梯度的时候行为不一致

clip在边界值，如上例的0， 6 时候，梯度为1

而 Relu6在边界值的表现，梯度为0

现在正在开发 Relu6算子
@daquexian
Copy link
Contributor

输入会恰好等于 0 或者 6 吗

@MARD1NO
Copy link
Contributor Author

MARD1NO commented Dec 23, 2020

输入会恰好等于 0 或者 6 吗

是的，如果恰好等于这两个边界值，梯度的表现是不一样的

@daquexian
Copy link
Contributor

是的，如果恰好等于这两个边界值，梯度的表现是不一样的

什么情况会恰好等于边界值呢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@daquexian @MARD1NO