Mitigating Pooling Bias in E-commerce Search via False Negative Estimation

Xiaochen Wang; Xiao Xiao; Ruhan Zhang; Xuan Zhang; Taesik Na; Tejaswi Tenneti; Haixun Wang; Fenglong Ma

doi:10.1145/3637528.3671630

Mitigating Pooling Bias in E-commerce Search via False Negative Estimation

Xiaochen Wang, Xiao Xiao, Ruhan Zhang, Xuan Zhang, Taesik Na, Tejaswi Tenneti, Haixun Wang, Fenglong Ma

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Efficient and accurate product relevance assessment is critical for user experiences and business success. Training a proficient relevance assessment model requires high-quality query-product pairs, often obtained through negative sampling strategies. Unfortunately, current methods introduce pooling bias by mistakenly sampling false negatives, diminishing performance and business impact. To address this, we present Bias-mitigating Hard Negative Sampling (BHNS), a novel negative sampling strategy tailored to identify and adjust for false negatives, building upon our original False Negative Estimation algorithm. Our experiments in the Instacart search setting confirm BHNS as effective for practical e-commerce use. Furthermore, comparative analyses on public dataset showcase its domain-agnostic potential for diverse applications.

Original language	English (US)
Title of host publication	KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Publisher	Association for Computing Machinery
Pages	5917-5925
Number of pages	9
ISBN (Electronic)	9798400704901
DOIs	https://doi.org/10.1145/3637528.3671630
State	Published - Aug 24 2024
Event	30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024 - Barcelona, Spain Duration: Aug 25 2024 → Aug 29 2024

Publication series

Name	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
ISSN (Print)	2154-817X

Conference

Conference	30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024
Country/Territory	Spain
City	Barcelona
Period	8/25/24 → 8/29/24

All Science Journal Classification (ASJC) codes

Software
Information Systems

Access to Document

10.1145/3637528.3671630

Cite this

Wang, X., Xiao, X., Zhang, R., Zhang, X., Na, T., Tenneti, T., Wang, H., & Ma, F. (2024). Mitigating Pooling Bias in E-commerce Search via False Negative Estimation. In KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 5917-5925). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). Association for Computing Machinery. https://doi.org/10.1145/3637528.3671630

Wang, Xiaochen ; Xiao, Xiao ; Zhang, Ruhan et al. / Mitigating Pooling Bias in E-commerce Search via False Negative Estimation. KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2024. pp. 5917-5925 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

@inproceedings{bd94a7ffbd2447ff867f1a962005572b,

title = "Mitigating Pooling Bias in E-commerce Search via False Negative Estimation",

abstract = "Efficient and accurate product relevance assessment is critical for user experiences and business success. Training a proficient relevance assessment model requires high-quality query-product pairs, often obtained through negative sampling strategies. Unfortunately, current methods introduce pooling bias by mistakenly sampling false negatives, diminishing performance and business impact. To address this, we present Bias-mitigating Hard Negative Sampling (BHNS), a novel negative sampling strategy tailored to identify and adjust for false negatives, building upon our original False Negative Estimation algorithm. Our experiments in the Instacart search setting confirm BHNS as effective for practical e-commerce use. Furthermore, comparative analyses on public dataset showcase its domain-agnostic potential for diverse applications.",

author = "Xiaochen Wang and Xiao Xiao and Ruhan Zhang and Xuan Zhang and Taesik Na and Tejaswi Tenneti and Haixun Wang and Fenglong Ma",

note = "Publisher Copyright: {\textcopyright} 2024 Copyright held by the owner/author(s).; 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024 ; Conference date: 25-08-2024 Through 29-08-2024",

year = "2024",

month = aug,

day = "24",

doi = "10.1145/3637528.3671630",

language = "English (US)",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

publisher = "Association for Computing Machinery",

pages = "5917--5925",

booktitle = "KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining",

}

Wang, X, Xiao, X, Zhang, R, Zhang, X, Na, T, Tenneti, T, Wang, H & Ma, F 2024, Mitigating Pooling Bias in E-commerce Search via False Negative Estimation. in KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, pp. 5917-5925, 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, 8/25/24. https://doi.org/10.1145/3637528.3671630

Mitigating Pooling Bias in E-commerce Search via False Negative Estimation. / Wang, Xiaochen; Xiao, Xiao; Zhang, Ruhan et al.
KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2024. p. 5917-5925 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Mitigating Pooling Bias in E-commerce Search via False Negative Estimation

AU - Wang, Xiaochen

AU - Xiao, Xiao

AU - Zhang, Ruhan

AU - Zhang, Xuan

AU - Na, Taesik

AU - Tenneti, Tejaswi

AU - Wang, Haixun

AU - Ma, Fenglong

PY - 2024/8/24

Y1 - 2024/8/24

N2 - Efficient and accurate product relevance assessment is critical for user experiences and business success. Training a proficient relevance assessment model requires high-quality query-product pairs, often obtained through negative sampling strategies. Unfortunately, current methods introduce pooling bias by mistakenly sampling false negatives, diminishing performance and business impact. To address this, we present Bias-mitigating Hard Negative Sampling (BHNS), a novel negative sampling strategy tailored to identify and adjust for false negatives, building upon our original False Negative Estimation algorithm. Our experiments in the Instacart search setting confirm BHNS as effective for practical e-commerce use. Furthermore, comparative analyses on public dataset showcase its domain-agnostic potential for diverse applications.

AB - Efficient and accurate product relevance assessment is critical for user experiences and business success. Training a proficient relevance assessment model requires high-quality query-product pairs, often obtained through negative sampling strategies. Unfortunately, current methods introduce pooling bias by mistakenly sampling false negatives, diminishing performance and business impact. To address this, we present Bias-mitigating Hard Negative Sampling (BHNS), a novel negative sampling strategy tailored to identify and adjust for false negatives, building upon our original False Negative Estimation algorithm. Our experiments in the Instacart search setting confirm BHNS as effective for practical e-commerce use. Furthermore, comparative analyses on public dataset showcase its domain-agnostic potential for diverse applications.

UR - http://www.scopus.com/inward/record.url?scp=85203681117&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85203681117&partnerID=8YFLogxK

U2 - 10.1145/3637528.3671630

DO - 10.1145/3637528.3671630

M3 - Conference contribution

AN - SCOPUS:85203681117

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 5917

EP - 5925

BT - KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

PB - Association for Computing Machinery

T2 - 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024

Y2 - 25 August 2024 through 29 August 2024

ER -

Wang X, Xiao X, Zhang R, Zhang X, Na T, Tenneti T et al. Mitigating Pooling Bias in E-commerce Search via False Negative Estimation. In KDD 2024 - Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery. 2024. p. 5917-5925. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/3637528.3671630

Mitigating Pooling Bias in E-commerce Search via False Negative Estimation

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this