indexers.hooks.default_hooks.postprocessing.DeduplicationHook

indexers.hooks.default_hooks.postprocessing.DeduplicationHook(
    score_aggregation_method='max',
)

A post-processing hook to remove duplicate knowledgebase entries.

Designed to operate on a VectorStoreSearchOutput, i.e. the output of the VectorStore.search() method. Deduplicates entries that share the same label within each query’s results, aggregating their scores using the specified method.

Attributes

Name	Type	Description
score_aggregation_method	str	The name of the score aggregation method in use. Either “max” or “mean”.
score_aggregator	Callable	The callable used to aggregate scores for deduplicated entries.