Description

Summary of mailing list thread...

https://lists.apache.org/[email protected]

• Using UnifiedHighlighter w/ DefaultPassageFormatter
• Highlighting fields that use setStoreTermVectors(true) + setStoreTermVectorOffsets(true)
  • but do NOT use setStoreTermVectorPositions(true)
• IndexOutOfBoundsException can occur in DefaultPassageFormatter -> StringBuilder when query and field includes multiple terms
• Problem caused by TermVectorOffsetStrategy producing Passage instances where matches are not in order (by start offset)
  • Not clear from Passage API if this is allowed
  • DefaultPassageFormatter does not expect this (only the possibility that end/start will overlap)
• Problem started happening "by default" in 9.0 due to LUCENE-9431
  • LUCENE-9431: UnifiedHighlighter WEIGHT_MATCHES is now true by default #362
• Known workarounds:
  • Index Time Option: Add setStoreTermVectorPositions(true) to fields you wish to highlight that already use setStoreTermVectors(true)
  • Alternative Query Time Option: Subclass UnifiedHighlighter to override getFlags(String) and remove HighlightFlag.WEIGHT_MATCHES from the set returned by super.getFlags(field)

Test patch demonstrating problem in above linked mailing list thread

Version and environment details

No response