eBay has started rolling out Best Match in earnest on the core eBay.com site, and boy is it getting noticed.
First, here is the original post on eBay that announced the test of Best Match as the default sort in five major categories, dated January 16th. Just a few days ago, really.
I caught this blog post over the weekend from Randy Smythe, and realized that I had a few things to say about the launch of this test.
The first of which is congratulations to the eBay Finding team. The launch of this test represents an inevitable step towards the future of a search engine on the eBay.com site optimized for the best possible buyer experience. For all the back-seat driving and Monday morning quarterbacking that they receive, very few people understand the complexity of the problems that the eBay Finding team has to tackle.
The second thing I have to say here is get ready to drink from the firehose. This move is bigger than anything I can think of in the history of the eBay buyer experience, and it’s going to test eBay and the eBay community in new ways. There is no playbook for this type of change, there is no simple pattern match. There is going to be a lot of churn, a lot to learn, and lot of quick action & analysis needed to make this successful.
It might not seem obvious to outsiders how big a change this really is. But believe me, it’s huge. There is a $60 Billion economy that is all predicated on the way that hundreds of millions of buyers search through and find billions of items for sale on eBay. That’s roughly the Gross Domestic Product of the country of Vietnam.
To explain why this change is so dramatic, let me explain a bit of the background behind this change. Let’s start with how eBay search works today.
eBay search has a history of being extremely literal and transparent. Until changes were made in the last few years, eBay search would literally do only the following:
- Look at the keywords entered by the buyer
- Look at the title keywords of every listing on the site
- Return only the listings that had 100% of the keywords entered by the buyer
- Sort the listings by “time remaining”
When I worked on the eBay Finding team, it was always surprising to me how many active eBay users I would talk to, both buyers and sellers, who assumed there was “something more” to the way eBay returned items. In fact, I would sometimes ask potential product managers, interviewing at eBay, to describe how they thought the eBay search engine worked. I would get the correct answer less than 10% of the time.
This system had some clear and obvious benefits. It’s simplicity meant that it was transparent to sellers and buyers, at least, in theory. Sellers would, in theory, experiment over time to find the right keywords to use in their listings. Buyers would also experiment. Over time, assuming that eBay was a fairly efficient market, sellers would provide listings with keywords to match the keywords that buyers would use. Supply would meet demand.
Sorting by time remaining had some natural benefits too. For an auction that ending soon, the differences between zero bids, one bid, and more than one bid are stark. One bid guarantees a sale, two bids puts you on a fast path to an efficient price. There was inherent benefit for sellers and for eBay to see auctions that were ending soon get exposure to a disproportionate number of buyers.
So, if it’s not broken, don’t fix it, right? Well, the good news is, the search system was good enough to grow eBay to the giant it is today. The bad news is that it had some fairly obvious shortcomings that became unsustainable over time.
There were a few obvious ones that almost anyone who used eBay ran into. Inexperienced sellers, just casually listing, had no idea what keywords to put in their titles. Pitty the poor seller, trying to sell their $1500 PowerMac G5, if they instead called it a “PowerMac G-5”. Inexperienced buyers also had no idea that searching for “Apple Macintosh” would bring back radically different results than “Apple Mac”. eBay didn’t know how to match keywords to categories. A search for “DVDs” wouldn’t just take the buyer to the DVD category – it would literally return all listings that had “DVDs” in the title. eBay didn’t even understand plurals! “DVD” would bring back very different results than “DVDs”.
eBay started addressing these issues in earnest about five years ago. They slowly rolled out improvements like transliteration (plurals), as well as some experiments with “generic keywords” like DVDs.
Why slowly? Well, the problem is, tinkering with a multi-billion dollar economy is, to lack a better word, scary. It’s scary because you have millions of sellers who have already adapted to the old search engine. You have billions of dollars of purchasing at stake, which means a 1% blip in finding efficiency can mean the difference of tens of millions of dollars in revenue for the company. And last, but not least, it’s scary because it’s hard to objectively find a measure of success that everyone can agree on.
How do you measure the success of search? When a buyer does more searches, is that a good thing or a bad thing? If a buyer views more items, does that mean you’ve done a good job showing them relevant items, or a bad job because they have to click through a lot of items to find one they want?
This gets even more complicated when you take into account the financial relationship between eBay and its sellers. eBay gets paid basically two ways: fees paid up-front when listing the item, and fees paid when an item sells. As a result, sellers pay eBay an up-front amount assuming a certain amount of visibility for their item. eBay does not guarantee impressions, clicks, or sales, but over time, sellers get used to the rough economics of their activities on eBay. They learn which keywords, which categories, which items get them enough clicks and sales to make their business works. That’s how they decide when and where to pay eBay it’s fees.
In any case, those changes merely affected the results that were returned by eBay’s search engine when the buyer performed a search (Step 3). It didn’t affect the sort order, which determines which items are on the first page of a buyer’s search results.
Unfortunately, changing the sort order was just a matter of time. “Time Remaining” is a very good sort for auction items, but it is almost meaningless for fixed price items. Over time, as eBay grew, more and more items on eBay were fixed price. In fact, if you include eBay Stores, eBay has had vastly more fixed price items than auction items for some time. What’s more, all the Step 3 changes mentioned above added more items to the search results, making it even less likely that you’d get good results on your first page.
If you are familiar with internet search, then you know sorting your items to provide the best possible results on the first page is incredibly important. And a meaningless sort for a majority of your listings is just not going to be sustainable without sacrificing a significant amount of your buyer experience and sales.
So, this rollout of Best Match is a big deal. Best Match does not change the results that are returned by eBay for a given keyword, but it does change what appears on that first page. It is a new way to sort items. And that, by itself, is huge.
Not surprisingly, sellers have noticed. Randy’s blog post quotes a seller who has purportedly seen a 40% drop in sales. It’s certainly possible. Best Match will alter the amount of time that listings will have at the top of results. Some sellers might see no change in their activity. Most will see small changes. But there will be a few who see huge swings from their existing metrics.
Most interestingly, it is practically impossible to predict what the outcome will be for any particular seller. To be sure, eBay will guide Best Match to increase overall sales for the site. That means, more items will receive bids and be bought. The economic pie will be bigger for the eBay selling community. But there is no known way to effectively simulate what the outcome will be for any particular seller with their existing listings.
This is a fundamental challenge for eBay. eBay has stated they will focus on improving the buyer experience. eBay will also continue to manage the marketplace to a greater number of sales. However, that won’t change the fact that some sellers will do better under this new system, and others will do worse.
Don’t be surprised to see sellers start to dissect public patent applications for clues on how eBay Best Match works. This is their lifeblood, as much as Google PageRank is the lifeblood for content websites. There is huge economic value in “cracking the code”, and one thing is for sure, the eBay community is full of entrepreneurs who will try to harvest some of that value. Like Google PageRank, Best Match is designed to be opaque. As a result, eBay will make no guarantees about how it functions, and they will actively change it over time to improve it and to prevent abuse.
Also, don’t be surprised when sellers are, in the aggregate, upset about this change. This adds uncertainty to their business, and even though every other site out there is based on relevance sort, they hold eBay to a different standard, and for good reason.
The version of Best Match that eBay is rolling out now has gone through more testing than any new piece of functionality that eBay has ever released. They have gone through numerous versions of the technology, numerous experiments with different factors and systems, and elaborate economic experiments to ensure that it results in higher sales for the marketplace and happier eBay buyers.
And now the real test begins. The eBay Finding team will need to listen, learn & react more in 2008 than they ever have before. It will not be easy, for anyone. But then again, the most important changes never are.
Update (1/24/2008): It looks like this post was picked up in the internal Weekly Gazette inside of eBay. I am, of course, flattered to be highlighted. Of course, I am not an unbiased source, since it was on eBay Express that we first discovered the need to move away from “ending soonest” and “lowest price” sorts, and launched the very first, crude version of Best Match.
Psychohistory 
I hope eBay already had ranking manipulation filters in place. The patent paper specifically mentions keyword search, click-through and adding to watch list as being part of the algorithm. All are valid measures to some degree. eBay needs to filter click fraud (click-through & add to watch list) and automated keyword searches that are not performed by genuine members of eBay. Otherwise, keyword search counts and even what is considered a popular search can be “gamed”. The only true measure of popularity is a keyword search that actually leads to a sale with confirmed payment.
Actually, it’s not that simple. I could write a whole post on the complexities of both gaming the system, and trying to detect gaming. But you can’t just use purchase events for ranking to produce high quality data – you can game purchases in some ways more easily that clicks or watches because purchases happen in far lower volumes than clicks or watches.
Like Google, eBay will have to invest significant resources in maintaining the quality of the ranking algorithm over time, to both adapt for changes in the market and for fraudulent abuse.
Adam
Hi Adam, you and I had a dialogue on the EE board quite a while ago. It was very informative for me and it, along with your other postings, displayed the positive nature of your ideals and character. I was, and still remain, truly sad that you left ebay. Their loss, is our loss. I value your opinion and ask you to comment, if you can, on the following.
I joined ebay in 1999; I do understand the obstacles that ebay faces in Relevant Finding (in my limited, user way). Yes, Fixed price is the du jour (we store sellers have been saying that for some time now!), but what about Auctions in all this Best Match manipulation? What about re-invigerating the Auction experience? Won’t showing the auctions out of time-ending sequence have the potential to cause fewer bids?
What about the unique, scarce and truly rare collectibles and antiques, as they are practically forced to use the auction format? As scarce items, any group of them would generally make up a Small Subset, to begin with, and should therefore be shown in a somewhat less-BM-scattered display. But, they could be vunerable to inclusion of non-relevant listings, thereby being hidden via a massive amount of unwanted search-result listings.
It is obvious the BM technology is geared towards manipulating the search results of a huge DB of similar / like items, which simply shouts NEW; as in Commodities. Is the entire Collectibles arena destined to be in that group that you write, “others will do worse”?
“…to ensure that it results in higher sales for the marketplace and happier eBay buyers.” That statement fragment (of yours) conveys the basic premise of BM. It is repeated throughout your post, in several paragraphs with various wording. I don’t doubt that that is its intended purpose, but will auctions suffer because of it?
If more auctions receive bids, then the goal is achieved. But, if MORE auctions receive fewer Multiple bids (than currently) because ending-times are scattered, the Marketplace will gain as a whole, while the individual sellers suffer. The wealth will be spread farther, the bidders (buyers) will get really great deals; ebay will collect on additional FVF fees due to increased STR; ebay’s metrics will increase. It’s a win-win for the buyer, for ebay and, possibly, the Commodity seller.
For the seller that starts a $300 Nippon vase at $9.95, it doesn’t look so good. The two arenas – Commodity and Collectible – are diametrically opposed when it comes to a search function. Your 4-step search result function is still valid for Collectibles, as long as a functioning category structure is also in place to allow for farther segmenting when a large volume is returned.
Is it time for ebay to finally make the Big Break and have two sites? You know, what EE was intended to be, but sellers refused to adopt.(wink) Perhaps you could do a whole posting about BM versus Colectibles… sorry, make that BM and Collectibles. Thanks Adam!
Tony
Hi Tony,
Wow! Quite a comment. I really appreciate the kind words. Although I love the vibrancy and excitement of the quickly growing LinkedIn community, I still miss those conversations on eBay, and of course, I’m still a fairly active eBay seller myself.
I think you are making a few assumption about Best Match that aren’t quite right – if done correctly, eBay will optimize Best Match on a very localized way – Best Match will be different across different queries, different categories, different countries – even potentially different buyers! It will depend on both long term and short term trends and behaviors. If done correctly, Best Match should always be optimizing itself to show the items that the buyer is most likely to buy.
Remember, the advanced buyer is going to come in, and choose other sorts – by time, by price, etc. They may even know how to do advanced searches with special keywords and controls (you should see my buying searches!)
Best Match will only be successful if it delivers more bids and more sales, in the aggregate, to the marketplace, period. I would expect it’s approach to doing this will vary significantly in collectibles vs. clothing vs. electronics vs. media.
I notice the comment on eBay Express… 🙂 Let me tell you, with eBay Express we spent our time on two things:
1) How to make it literally painless for as many sellers as possible to adopt. (For most sellers, they literally had to do nothing).
2) How to make the overall purchase experience as good as possible.
As JD & MW stated on the call today, a lot of the new technology, new thinking, and feature concepts from eBay Express are now forming the basis for great improvements to eBay.com. Of course, they will be modified to fit an auction environment.
But in my local world view, eBay Express was the first time that eBay said, whole heartedly, “How do we make the best buying experience possible?”
And regardless of the answer, that question is very right.
Adam
Hi Adam,
Thanks for the reply – I do appreciate it! Yeah, I remember that you wrote, back on the EE board, that you are a seller and I figured you still are. It is that POV you have that makes you the perfect Brain for picking. LOL. You certainly don’t have to depend upon your ebay sales for income (the techie field isn’t that bad, is it?), but I do assume you have more insight into ebay sales than many/most/all of the top Execs.
I really don’t have any assumptions about the ramifications of BM – even its direct impact upon (only) Collectibles. It is easy to see that it *can* be implemented into any collectible category with basically postive results. But, It’s like you wrote… “if done correctly, eBay will optimize Best Match on a very localized way”. It is that “if” word that has many of us worried.
“If done correctly, Best Match should always be optimizing itself…”. Again, that scary IF word. I have to say that most Collectible sellers doubt that the ebay personnel involved have the necessary POV needed for such items. This viewpoint stems from the perception (by the sellers) that BM is fundamentally wrong for the collectibles arena. You probably don’t agree with that statement, but do you understand from where it comes?
As far as Searching goes, I have faith in my buyers – both repeat and first-timers. They are a diligent bunch and they will wade through just about any roadblock encountered. Since I am incognito here, I can say it… I sell Old Junk. I love the stuff and so do my buyers. I have very little problem with scam buyers and other such troubles – if they buy it, they are truly interested in IT.
I have sold this Old Junk since the mid 70’s from a B&M store and did mail order by way of trade magazines. Later, this thing called the Internet allowed trade via Bulletin Boards, then to Prodigy ads and finally to ads on AOL with a puny link to a pix on our ‘personal page’. Oh yes, we thought we were really high-tech back then. I was a little late to ebay (1999), but I did/do have some experience with “unseen item trade”.
I know a few things about a collectibles buyer, because I am one and I have dealt with them for a while. I know they don’t want to be lead around, directing their attention to whatever it is that YOU want to show them. They want to look, browse, hunt, dig, scrounge while searching for a dozen things they have in mind. But wait! they just found something else and they just gotta have it. That is serendipity – not a leopard skin iPod cover!!
It would not matter how much data the BM machine has collected and collated from the entire ebay listing DB, nor for how long it has collected it, when a scarce / rare item comes along, it’s Relevancy and Desirability scores will be low, or quite possibly, into the negative. Now, I make no assumptions about how these factors *may* be overcome, or if this scenario is even considered ‘important’ by ebay. I am simple stating the general perception of some sellers. (or the forgone conclusion of the vast majority of collectibles sellers – but that would be presumptuous of me)
I realize (and others do also) that ebay can institute controls to affect the above example. But, that’s the problem, Adam. The fact that something has to be Done, to correct something else that was previously Done. And, whatever that ‘Done’ accomplished, will prolly need to have another Done implemented to correct yet another issue.
A river with a dam. A hole is drilled to allow water to flow. Downstream, another dam is erected to stem the flow. Yet another hole to allow water, and on and on and on… Just leave the dang river alone, to begin with. But that just ain’t gonna happen – the need (real or imaginary) to Showcase and also to Disadvantage is at hand, and it will be done. Period. So be it.
Then we are back to the point that BM is fundamentally wrong for the way that Collectibles Buyers shop. Period. I am but one voice, yet many others say the same thing on the boards and ebay chooses to ignore us. You have to know, that is taken as a signal that we are Correct and they have no answer. Maybe that isn’t factual, but it is the implication that many sellers believe. Their continued silence is our Proof. That’s how it works, in human nature.
Maybe they simply chose to not answer- not yet, anyway. Which begs the question, do We really matter to ebay? As an aggregate, Yes, but an aggregate number excludes the Scarce / Rare / Unique and that is our concern. It should be ebay’s concern, as well. Peeps may flock to ebay for the consumer electronics and gadgets, but it is the serious collectors that come back, time and again (through any roadblocks, remember?).
With BM in force, the serious Collectibles buyer cannot search for and find that Treasure amid all the scattered clutter and hodgepodge of listings. Not easily, anyway. But they will try. Again and again. So, what about the newbie just coming to ebay to “take a look around”? When they browse to the collectibles category, will they see that Sky King secret decoder ring, or a screen-full of leopard skin iPod covers sorted with the scam seller as the next to last item on page 5? (with that Sky King ring being the last item)
I really don’t see any Fix or Optimization that will keep BM from damaging the collectibles arena. It, like ebay’s metrics, deal in the *aggregate* and that is not Collectibles; not their Supply, not their desirability factor, not even in the percentage of buyers looking for them. Quote: “Best Match will only be successful if it delivers more bids and more sales, in the aggregate, to the marketplace, period.” Again, I say, what is good for the aggregate, is not necessarily good for the individual seller.
When enough individual sellers have lost enough money – when enough individual buyers have lost enough patience… they will become an aggregate. It’s only a matter of When, the ‘If’ was passed long ago. BM will speed-up that process. Ebay’s monopolistic stance seems to be hard-coded and what now appears as a Slide/Downturn to TPTB, has been obvious to the sellers for a long time now. We believe that we actually have some insight into this Buyer/Seller/eCommerce thingy. We warn – Fraud, Scams, Hijacks in 2004. We were/are ignored. We have been proven correct every time – ebay REacts to threats in 2006-7. There’s no pleasure in being Right.
WARNING: Best Match is hazardous to the Collectibles arena.
That warning is not meant to be taken literally, exclusively, for just the items themselves. The Perceptions of the sellers and buyers are also being impacted. The damage is done, and continuing to grow, and ebay’s failure to address these issues does nothing but add to the damage. The old ebay mindset of “wait it out”, is not just failing, it is causing the ‘dragon to eat its tail’. Even though ebay is aware that the marketplace is drastically changing, the only changes they make in their response to customers’ pleas is to be LESS forthcoming and receptive. Classic 900-pound gorilla methodology. A diet is long overdue.
I agree that ebay asked the right question… I just wish they had asked the opinions of a large amount of sellers. Maybe all of them.
Tony
P.S. Adam, sorry this reply is almost blog-length itself. I am (secretly) hoping that you still have some input, in whatever fashion, to the ebay peeps in charge of this roll-out. And if you don’t, just keep that a secret. I don’t want to abandon Hope, just yet.
From the day this was first foisted I’ve said, “Present two search buttons, like Google has ‘I’m Feeling Lucky.'”
People who do not buy or sell have been put in charge of the world’s largest buying and selling website. MBA’s do not have the shopkeeper mentality needed to run ebay correctly. And auctions are receiving short shrift because they are culturally despised by management- period.
Keyword search is immediate. Intuitive. And absolutely necessary for Collectibles, obsoleted machine and car parts, hand-manufactured goods, and just as importantly, items that have not existed before. Best match filters all the above out as junk. It’s why I no longer sell.
Furthermore, I respond to a well-written title. When someone is working within the constraintsof the 50-character title field, they really have to tell it like it is. You can learn much about a seller just by the choices they make for the title. For that to happen, I need to be able to SEE the title. I will decide.
But enough complaining, it will be what it will be. There should be an ebay.com and a keyword-searchable ebayauctions.com . Plain and simple.
permacrisis,
I don’t know what to really say here except that I believe you are almost completely and totally wrong on all counts save one.
The best accounting software is not actually built by accountants. The best music software is not actually built by musicians. The best CAD software is not actually built by architects. In a perfect world, maybe these type of specialists would actually know how to design and build software. Maybe they would if they invested part of their lives studying the topics and issues necessary to understand the technology and user experience issues involved. But then again, if they did that, they might not be as good at accounting, music, or architecture…
In the real world, we have to find a way to get the insights from actual users of software and provide it to the people who have expertise in both designing and building it. To do that, you need open and honest communication between customers and providers, and that communication cannot happen without a high degree of trust between the parties involved.
Best Match does not remove keyword search. It just changes the sort order. Period. So, your argument would be more relevant to a debate about different technologies for search recall than about sort order. Best Match does not filter anything. It is not a filter.
You are, of course, correct about the item title. You can learn a lot about a seller from it, and Best Match actually uses the title keywords to inform it’s relevance algorithm. Once again, Best Match does not hide any titles. It does not hide any items. It just changes the order.
To argue that eBay employees “culturally despise” auctions is an absolutely ridiculous and inflamatory statement. Auctions are the lifeblood of eBay, and everyone is immersed in that at eBay.
Most importantly here, your post displays an incredible lack of respect for the hard working men & women who work extremely long hours on a regular basis to make eBay the best possible for millions of sellers and hundreds of millions of buyers. You are disparaging them, their efforts, and their motivations. There is nothing good that is going to come from that type of toxic behavior.
Please don’t post on my blog again.
Adam
Ho hum. You have a wonderful collection of fancy theories. But you had to develop those to get paid, didn’t you? The fact is, however, that anyone who COLLECTS anything and now stumbles on eBay for the first time will most likely lose patience and leave before he figures out what’s going on and turns off your nasty Best Match. Not to mention the “New Search”, which is simply a horror for all collectors and will drive us away in droves if implemented.
Did your employers decide to throw away the collector buyers and sellers? If not, you have been very stupid in what you have done. A beginning collector wants and needs to see the things in his category–mine is old radios–spread out before him in the order they are closing; in other words, in all their exciting randomness. Or maybe, once he’s a bit sophisticated and looking for a bargain, he will want to see things in the order of their listing (for “Buy it Now” items.) In the former case he might find something cool that’s just about to close. I remember how exciting that was! In the latter case he might snag a bargain before anyone else sees it.
Whatever the individual’s method of trolling for collectibles, it does not include buying want a bunch of people who don’t know the field have preferred. In the field of old radios, for example, all my searches have to to explicitly kick out all replica radios: Non-collectors love a nice plastic ’30s-look radio. They are worthless to me.
The collector, in other words, wants to see a nice random spread of things in his favorite category. Seeing this when he is a novice will help him form an understanding of the available models and their usual prices. Later, seeing this lets him deploy his expertise to find misnamed gems, etc.
Under the traditional eBay search the collector gets to be the fisherman all collectors inherently are. Learning about the fish and finding them on your own is most of the fun. Certainly no fisherman worth his salt wants to have some third party swim up and cram some fish on his line that he doesn’t want, just because a bunch of other folks do. I resent the fact that the founders of eBay built a collectors’ paradise (on which I have happily spent many thousands of dollars), and now you people are crapping it up in order to sell more everyday commodities people could buy anywhere else at all.
Cordially,
Leonard
Hi Leonard,
I’m going to ignore, for the moment, your introductory sentences. You are clearly trying to be inflammatory here, and congratulations, you have achieved your goal.
Let me make this exceptionally plain – eBay cares about collectors. eBay cares a disproportionate amount about collectors. eBay cares about collectors more than any other major internet site, and always will, even though collectors and collecting make up an increasingly small portion of eBay’s business.
You can still “kick out” the junk radios through keywords and advanced search terms. As I explained in my post, Best Match is a sort, NOT a filter. You can still filter to your hearts content.
Here is the good news – “time remaining” is a factor in the Best Match sort. The factors for Best Match can be tuned PER CATEGORY.
That means that if it proves true, through experiments, that collectors do best with a sort that is 100% Time Remaining, then eBay can set Best Match in those categories to be exactly that.
However, don’t hold your breath. While you claim to speak for all collectors, eBay will hopefully be looking at the shopping and purchasing results for their buyers, rather than just taking your word for it. It’s incredible unlikely that a sort based on a single variable, like time remaining, will prove to be optimal.
In the future, Leonard, I would highly recommend toning down the invective, and focusing more on the questions and statements you want to make. Like permacrisis, it seems like your primary interest here is picking a fight. Fighting here about eBay really isn’t warranted. I don’t work for eBay anymore, and I don’t speak for the company in any shape or form. I’m not a journalist, and this is not the forum to resolve your issues with eBay.
This is just one topic on my personal blog.
Adam
Pingback: You Still Have a Mail Slot in Building 6… « Psychohistory