Similary for Vectors

Overview of similarity metrics

• Euclidean (L2): Stright line (distance) between vectors. Useful for exact matching.
• Cosine similarity: Angle between vectors, thus ignores the distance of the vectors.
• Dot product: The overlap in space. It is similar to cosine, yet it takes into account the length of the vectors. dot product + normalization is similar to Cosine similarity, as normalized vectors are the same in length, thus, the projection from vec A onto vec B works similar to the angle between both

Important: Cosine similary = Inner product, if the vectors are normalized to unit length.

These higly effect Retrieval augumented generation (RAG) as:

• L2 focuses on raw distance, not semantic meaning
• As Cosine takes the direction into account, meaning is better reflected. Cosine is often the better choice for Retrieval augumented generation (RAG)