Why List<Double> instead of float[] for embeddings ? #632
michaelsembwever
started this conversation in
General
Replies: 1 comment
-
This was addressed in version M2, where the type of embeddings has been changed from |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Curious to why
List<Double>
was used, considering the performance benefits a primitive array gives, particularly with SIMD operations newer Java versions can do.A number of the vector stores have already a
toFloatArray(List<Double>)
to workaround this.Bonus question :)
What about using the JVector library and its
VectorFloat
class …?It's heavily optimised against the Panama Vector API(SIMD), and detects and takes advantage of JDK20+ when available.
While spring-ai might not be doing much with the embeddings, it's quite likely consumers of the framework will be, e.g. re-ranking large similarity result sets.
Beta Was this translation helpful? Give feedback.
All reactions