See inside your Amazon box with Amazon's visual search camera. Customers can use the free Amazon shopping app and try Package X-Ray to see what’s in a box that’s been delivered, without having to hide unwrapped presents around the house or to open multiple packages. Simply scan the barcode using the camera icon within the Amazon shopping app and the items that are inside the box will display on screen. Customers can only see what they ordered and not the orders of others.
My role was to concept, lead and design the customer experience for the mobile feature.
The impact of this project was over 350,000 launches in the first 6 months and our computer vision team solving to recognize over 162 variations of shipping labels in the US. This feature has since expanded to recognize wedding and baby registries, giving customers a delightful surprise when scanning a gift they've received. As part of a separate project, I created the framework for our new Camera Search Modes, where the feature lives now.
Mobile Application Nov 2016
The Design Challenge was to provide an easy-to-use and "wow" experience as compared to physically opening the box. My goal was to bring the customer’s products to life using a live camera view married with interactive AR visual effects.
Research And Exploration came before I started designing. I researched the problem: what do customers with too many boxes do currently? I found some sources of truth on YouTube and shared them with my team.
Next, I began thinking about how customers could quickly accomplish the task of scanning a box and seeing it's contents as quickly and easily as possible. I knew a glanceable view would be necessary and with orders like Amazon Prime Pantry with 20+ items in a box, the solution would require interaction. While I explored the framework of "sticky" product bubbles, or a sticky AR box with a tap to open, neither solved the problem. A bubble carousel with product images was my next iteration. In iOS we used the animation kit to have the bubbles float within their grid at a slow pace for delightful interaction and later added this interaction on Android.
The biggest challenge was how to educate customers on landing in the feature. Persistent text and a large bounding box provided customers instructional copy and a target area so they could experience Package X-Ray quickly.
I created a "sticky" box that sticks to the last position of the package label upon a successful scan that automatically animates open to reveal the package contents. Our customers reactions to this small detail were rewarding to see as the designer.
Once the flow was created the next challenges were error handling and latency. Representing product that are "private" or gifts, no network connection, not a package, not an Amazon package were all created. The final designs were delivered within 3 weeks of project kick-off as our timeline around a holiday season launch required quick iteration. We continue to test with and observe customers following shipment.
Andrea Zehr, User Experience Designer
Peiqi Tang, User Experience Designer
Mark Waldo, Senior Design Manager
Adam Neustein, Sr. Technical Program Manager
Priyam Singhal, Software Development Engineer
Michael Lou, Tech Lead, Computer Vision
Ketan Deshpande, Sr Mgr. Software Development
Antony Rajiv, Software Development Engineer
Dheeraj Soti, Manager, Software Development