You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We (@zml) found that tf.aliasing support seemed to be not working as expected, with the model producing garbage when used. In our case Llama 3.1 8B.
This is problematic for transformer models because we leverage donations for the KvCache.
For now we're not emitting those attributes when on neuron, but we're not sure what to do as we feel that if the SDK doesn't support them, it should just ignore them right ?
Hi,
We (@zml) found that
tf.aliasing
support seemed to be not working as expected, with the model producing garbage when used. In our case Llama 3.1 8B.This is problematic for transformer models because we leverage donations for the KvCache.
For now we're not emitting those attributes when on neuron, but we're not sure what to do as we feel that if the SDK doesn't support them, it should just ignore them right ?
The llama implementation is attached.
Thank you !
llama.aliasing.mlir.txt
The text was updated successfully, but these errors were encountered: