Which measurement should drive the similarity threshold in a semantic cache?
- A A fixed 0.5 for all workloads
- B An empirically tuned threshold validated against labelled prompt pairs (correct answer)
- C The vector dimension
- D The cache size
Shortcut: Thresholds are embedding-model and domain specific. Too loose returns wrong answers; too tight destroys the hit rate. It must be calibrated on representative data.