Fairness in e-commerce Search

On the show today, we are joined by Abhisek Dash and Saptarshi Ghosh. Abhisek and Saptarshi are researchers at the Indian Institute of Technology, Kharagpur, and share similar research interests around fairness and bias in Machine Learning. They discuss their recently published research paper, titled Alexa, in you, I trust! Fairness and Interpretability Issues in E-commerce Search through Smart Speakers.

Abhisek started by giving some background on what fairness in machine learning is. He also explained what it means to audit machine learning systems. He referred to this research by Christian Sandvig on auditing algorithms. In 2016, we had an interview with Christian on auditing algorithms. The discussion in that episode is very much relevant in today’s talk.

Going forward, Saptarshi discussed how the application of machine learning models became mainstream in making business decisions.

Abhisek discussed the role of machine learning engineers in identifying biases. He also explained the two kinds of audits: code audits and scrapping audits. Abhisek spoke about an interesting discovery he made for products with Amazon’s Choice badge, further cementing the hypothesis that Amazon promotes its products.

He referred to some antitrust hearings and articles (links in the resources section) where Amazon has been questioned on promoting its products in search results. He also referenced a research paper that investigated how Amazon intrinsically promotes its private label product beyond organic recommendations.

Saptarshi then delved into the experimental setup for auditing the bias in Amazon smart speakers. He extensively discussed how he accounted for noise in the data gathering process and scenarios that were not taken into account. They both discussed some concerning discrepancies in search results between Amazon smart speakers and the desktop website.

Abhisek discussed the methods used to quantify the level of biases in such machine learning systems. Concluding, he discussed how Amazon has been striving to create more transparency in its systems. However, he suggests more convincing results in their search engines. You can follow  Abhisek on Twitter @adash0193.