AI-enabled technology ensures customers receive products in the condition they expect and helps Amazon further its sustainability efforts.
Inside Amazon fulfillment centers across North America, millions of products, from dog food and phone cases to T-shirts and books, pass through imaging tunnels daily.
An artificial intelligence (AI) model named “Project P.I.,” which stands for “private investigator,” uses advanced tools to scan items for defects. The goal is to ensure customers are delighted by every order they receive.
Using a combination of generative AI and computer vision technologies, Project P.I. can detect defects like damaged products or issues such as wrong color or size before products reach customers.
Additionally, Project P.I. can help identify the root cause of issues, enabling preventative measures upstream to prevent them from recurring.
At the sites where the system is available, it has proven adept at sorting through millions of items that pass through the tunnels each month and accurately identifying product issues.
How Amazon’s ‘Private Investigator’ Works
Before an item ships to a customer, it travels through an imaging tunnel where Project P.I. uses computer vision to scan the product and evaluate the images for defects, such as a bent book cover.
If a defect is found, Amazon isolates the product so it is not shipped to a customer and investigates further to determine if there is a wider issue with similar items.
Amazon associates, who review the items flagged by Project P.I., decide whether the item is eligible to be resold at a discounted price as part of Amazon’s Second Chance site, donated, or used in another way.
The model serves as an extra pair of eyes for Amazon associates and is already enhancing manual inspections at several fulfillment centers in North America. The technology is expected to expand to additional sites throughout 2024.
“We want to get the experience right for customers every time they shop in our store,” said Dharmesh Mehta, vice president of Worldwide Selling Partner Services at Amazon.
“By leveraging AI and product imaging within our operations facilities, we are able to efficiently detect potentially damaged products and address more of those issues before they ever reach a customer, which is a win for the customer, our selling partners, and the environment.”
Ensuring a More Sustainable Customer Experience
Project P.I.’s work is not only part of Amazon’s customer-obsessed culture but also a key component of the company’s commitment to addressing climate change. Preventing damaged or defective items from reaching customers is critical to a positive customer experience and is also important for the planet.
Accidentally shipping imperfect items can result in unwanted returns, leading to wasted packaging and unnecessary carbon emissions from additional transportation.
“Amazon is using AI to reach our sustainability commitments with the urgency that climate change demands, while also improving the customer experience,” said Kara Hurst, vice president of Worldwide Sustainability at Amazon.
“AI is helping Amazon ensure that we’re not just delighting customers with high-quality items, but we’re extending that customer obsession to our sustainability work by preventing less-than-perfect items from leaving our facilities and helping us avoid unnecessary carbon emissions due to transportation, packaging, and other steps in the returns process.”
What’s Next: Preventing Future Errors
Amazon teams are also leveraging a generative AI system that uses a Multi-Modal LLM (MLLM) to investigate the root cause of negative customer experiences.
When a defect is reported by a customer that was missed during inspection, the system uses that feedback to understand the cause and continuously improve. The system reviews customer feedback and then analyzes images taken from Project P.I. in fulfillment centers and other data sources to confirm what led to the problem.
For example, if a customer reports receiving king-size sheets instead of twin-size, the system cross-references that feedback with fulfillment center images and asks questions like, “Is the product label visible in the image?” and “Does the label read king or twin?”
This technology also aids Amazon’s selling partners by making data on defects more accessible. For example, if a selling partner mistakenly labels products with the wrong size, Amazon would communicate the issue to help prevent it from recurring.
More than 60% of sales in Amazon’s store are from independent sellers—most of which are small and medium-sized businesses—providing a vast selection of amazing products, competitive prices, and convenience for consumers.
By reducing the number of defective products shipped to customers, we’re also reducing the number of returns. Project P.I. exemplifies our focus on improving the customer and selling partner experience.
With ongoing advances in AI, Amazon continues to invent new ways to delight customers and selling partners while working to deliver on our sustainability commitments. Take a deeper look at the technology and teams powering Project P.I. Learn more about our sustainability work.
