DealTracker - Project Genesis

Context#

In 2021, I was working at designstripe. I was in charge of finding machine learning solutions adaptable to the graphic design universe, and training models to add features to the product.

Of course, I always wanted to launch new projects on my personal time, and try new things. So I bought the holy grail RTX 3090, the crown jewel of the era. I managed the small feat of buying it for €1,500 on LDLC's website, while second-hand prices were around €3,000 during the Covid-19 pandemic. What joy! I was suddenly the owner of the best consumer GPU, with its 24GB of VRAM!

The idea for DealTracker came to me while I was watching a video about the second-hand price of an office chair I wanted to buy, the Herman Miller Aeron which goes for around €500 second-hand. It's this video: https://www.youtube.com/watch?v=x-aeDaHIEbY

At the end of this video, Ahnestly shares an interesting thought:

"If you want to find this chair cheaper, go through a dozen pages of 'office chair' sales on the second-hand market, every day. You'll eventually find someone selling this chair at a derisory price, because they don't know it's valuable".

And that's when I thought "hey, it would be nice to automate this task with Machine Learning". (to clarify: at that time, no second-hand platform had a feature for searching listings by image)

The First Training#

Without further ado, the project was launched. I cobbled together a few scripts to scrape second-hand listings currently selling office chairs, and here I was facing the crux of the problem: how to find the ones where the seller is selling an Aeron, without it being stated in the listing? The solution of course, is to go through the images.

On the surface, the problem is a very textbook binary classification task: you need to differentiate images showing an Aeron (class A), from images showing anything else (class B).

So I put together a small dataset. For class A, I put my best Aeron photos. For class B, I put diverse and varied images (cows, a sink, mustard...). And for the model, I chose a proven standard: ResNet-18.

[diagram]

The metrics were excellent, and I was thrilled to launch my first inferences with the model. Bad surprise: the model correctly detects Aerons, but I also get a lot of false positives. My model often tells me "Look, an Aeron!" when the images have nothing to do with it.

In fact, as I was about to learn, "detecting only Aerons" is a much harder challenge than just "detecting Aerons". My hastily assembled dataset was not representative of the images the model encountered during inference, and the model had simply drawn coarse boundaries to put my few Aeron images on one side, and the random images on the other. I'll talk about this in more detail on the next page.

Regardless, it didn't stop me. Although there were tons of false positives, the model still did an excellent filtering job, and already greatly facilitated my searches.

The First Results#

The First Find#

I'll always remember the first time my system detected an Aeron being sold as an "office chair". That day, I jumped up from my chair and exclaimed "I DID IT".

The listing was there, beautiful. It was exactly like all those other listings selling Aerons, with one slight difference: the price. The chair was sold for €50 instead of €500. Well, the only problem was that the seller was in Perpignan, while I live in Paris. No big deal. There are plenty of listings, and I'd surely detect others.

The Heist of the Century#

Patience pays off, and in this case, I didn't have to wait long. Because I ended up finding one of the best deals I've ever come across: an office liquidation.

It was a small business in the 16th arrondissement selling all its furniture in a single listing, with lots of photos. On the 7th photo: an Aeron, which wasn't mentioned in the listing. So I called the person managing the sale.

• Me: "Hello! I saw you had some nice office chairs!"
• The lady on the phone: "Yes exactly! How much are you offering?"
• Me: "€50?"
• The lady on the phone: "€50? What an incredible price, with pleasure."
• Me: "How many do you have, out of curiosity?"
• The lady on the phone: "6 I think..."
• Me: "I'll take them all!"

And there I went to pick up the batch of chairs. I remember the return trip with the hauler, during which I felt like laughing: thanks to my system, I had bought for €300 a batch of chairs worth a good €2,500. I had the feeling of having found a loophole in the system, which isn't really false.

The Evolution of the Project#

At this stage, I was convinced of the relevance of this approach that allowed finding hidden treasures around you. So I decided to push the project further and expand the search to more products, and more marketplaces.

By adding more products, I could no longer afford to manually define the "price where it becomes interesting", I needed to create a system based on market trend analysis.

For an Aeron for example, here's the usual price distribution on the second-hand market. Anything below a certain price can be considered a good deal:

[Graph]

Of course, for this to work, there's a major challenge to solve: eliminate all false positives in order to rely on reliable and clean data. And that, I absolutely did not anticipate the amount of time and energy it would take me.