The untapped potential of search

The shopping world is run with faceted taxonomies. You know, the standard drill of choosing narrower and narrower categories, and then adding filters on things such as price, manufacturers and features. All of this navigation is powered and made possible through highly structured data with explicit relationships stored in a database or equivalent.

The data is structured, with hard links and concepts, that are also inflexible. If I want to create new attributes to filter by I have to modify the database, create new entries and establish the relationships. In reality what ends up happening is that taxonomy designers spend a lot of time at the beginning of development figuring out what is the best Hierarchical structure because they know it won’t get changed a lot in the future.

Now, what if you could accomplish the same drill-down and filtering use cases without storing hard database relationships?

It turns out that you can. With a BUT.

Main point: Text indexing is a superset of structured taxonomies.

Let me say that again…

TEXT INDEXING IS A SUPERSET OF STRUCTURED TAXONOMIES

Remember that a service like Google indexes everything. You can search every page against any character sequence. Well, if you place your taxonomy information on the page in a text format then it gets indexed just like everything else.

Taxonomy information.. indexed? Isn’t that the same as hard database relationships? EXACTLY! Without the database! Or rather, Google’s index IS the database.

Here’s the huge boost from this: If the search index is an isomorphism of your explicit taxonomy, then it’s also an isomorphism of unknown taxonomies that you haven’t even thought of.

As long as you put as much information as possible on the page then Google will index it, and voila every kind of taxonomy you can think of is created and buried inside the search index. What this means is that you can go back and create taxonomies without any loss of information!

In the structured approach if I wanted to be able to filter on HDTVs that have a 120Hz mode, I have to create a new facet called “Refresh Rate” and then go back and add the 120Hz attribute to all those televisions that apply.

In the unstructured approach I just write down in the text somewhere that the television supports 120Hz, alongside the contrast ratio, and all the other specifications that may or may not be important. Now, I can just search for those features and I’ll have the filter applied automatically. Beautiful!

Now for the problems.

A raw text search of “120Hz” doesn’t differentiate between Does have 120Hz, and doesn’t have 120Hz. Also there’s no way to apply your own sorting, and GOOG doesn’t handle ranges well. And this is why there is untapped potential. Google just announced that they’re creating an supplemental index for Custom Search, so why not add some extra extensions?

As the webmaster of ProductWiki I know the structure of the page better than a bot ever will. If I can provide search hints to say “THIS PART OF THE PAGE IS MORE IMPORTANT” that would be nice.

Also, these search companies need to handle date, and numeric ranges a lot better. I should be able to do $1000..$2000 and it’ll return me everything that has $1103.23 to $1,500. Same with dates, let me put in a variety of formats (isn’t even that important) and the parser understands what to look for.

Now I can do this kind of expansion of terms myself, but damnit this is their core competency.

In conclusion: I finally realize the power of unstructured search. It really does become the Database of Everything and that’s really friggin cool. Now with that power comes great responsibility, so search companies let’s step things up a notch and get some more advanced query handling happening.

Ads on Video

TechCrunch takes a look at MSN’s new video site, and their new policy on showing advertisements. For fear of being cliche, video on the web is still an infant frontier and monetizing the hundreds of millions of eyeballs is the 64 billion dollar question.There are many options available, each one with their own advantages and disadvantages. Pre-rolls, post-rolls, overlays, banners, etc.

It bothers me that overlays are starting to get attention because I absolutely HATE them. I hate anything that obstructs my view of the action that’s happening on screen. Personally, even though they get a lot of flack, I think pre-rolls have the best potential. Why? Because they’re inline with my experience. Look to my previous write-up about PPP and things being out or in context of the experience. The reason why pre-rolls ads have gotten such a hard time is because they’ve been executed upon so poorly. If I’m just watching a 30 second clip it doesn’t make sense to show me a 30 second commercial in front.

Instead commercials should be short, very short, engaging, and tailored to what I’m interested in. 5-10 second ads are bearable. Heck, just use the time before a video loads to display an ad, instead of waiting for the damn ad to show up in the first place! Don’t show me some animated circle, show me a deal on some new 360 games, or news about a new service launching.

 From my own personal experience I know that videos are incredibly popular and can spread very quickly. So even if people can’t monetize well right now everybody’s going to be trying their hardest to figure it out. In the meantime I’m going to enjoy this Golden-Age of minimal advertising on sites like Stage6 before someone DOES figure it out, and we’re stuck with ads, just like AdSense did to the text web.

PayPerPost and Making Money Online

TechCrunch has another post about PPP and its evils. I don’t want to talk about this specific incident since it’s pretty silly all around, but like Arrington says, PPP always brings up discussion and this is a good time to share my thoughts on it. While some people disagree with PPP’s methods, there IS something to be said about rethinking the way that bloggers make money. Look at the current situation… you have:

1. Bloggers who write and want to make money

2. Companies that want to get their products and services known

It’s only natural that these two forces come together. Bloggers want to make a living. Companies want to promote. Nothing groundbreaking here. Traditionally the way these two entities have come together is in the form of advertising. On the web this includes AdSense, or other ad networks, or using banner ads through an ad agency like b5Media or Federated Media.

However, those kinds of advertising schemes are pretty broken. When you have technology such as AdBlocker existing, it shows that advertisements annoy a contingent of readers enough to take action, and maybe annoys more than that but are too lazy to do anything about it, or they bite the bullet and deal with the ads because they know the blogger has to eat.

But why does it have to be that way? It obviously doesn’t. For every blog out there, there is a corresponding business model that is a better fit than just throwing up some banner ads. Ads are very EASY to do, just go with an existed system, or even sell the ads yourself, it’s all very straight forward. But most of the time they’re not the BEST thing to do.

Look at ProductWiki, or any Price Comparison service. These are great business models because the money making is built directly into the service. People don’t consider the “ads” as being an out-of-context message, but an integrated part of the experience.  By using a model that is harmonious with your users expectations you end up making more money, and getting a better experience.

PayPerPost is another kind of way for bloggers to make money. The idea isn’t that bad in theory, it’s the execution that leaves you wanting. Personally, I looked at PPP to see if it’d make sense to have some bloggers write about the site and check it out. Looking at the blogs they recommended, most of them were crap. Garbage. Useless.

And it makes sense! A lot of bloggers that work with PPP make some decent money with it, so they use the service: a lot. The end result though is that most of these blogs have horrible quality where it’s just one payed post after another. That doesn’t benefit anyone, unless you’re just interested in straight link development. But if you were, you could get much better return on your investment through other means.

That being said, I’m not against having companies pay bloggers for their attention to review and examine their products and services. Bloggers don’t have infinite time to review everything. Looking at the web industry, you have your A-list, B-list and so on. These are busy guys that don’t have time to write about the 10 new startups that launch every single freaking day. What does that mean for the startups? You gotta pay to get noticed.

It’s already happening even if it isn’t explicit! Startups get funded, they hire PR firms, marketing agencies. They hold parties. They attend conferences. They spend A LOT of money to get themselves noticed by the people that matter. They spend A LOT of money in the hopes that they will be written about, and refered to by the people that matter.

You could say that bloggers just write about things they think are WORTH writing about. Well obviously. However, there’s a lot that they’re NOT writing about that is WORTH writing about but they don’t have TIME to. That’s where the paid model comes in. Let’s take out the middle-man. Using this medium why go the “old media” route of the payola to get bloggers to write about us. Just pay them directly. But you’re not paying for their writing. You’re paying for their attention so that they MAY write about you.

Naturally this creates a supply (blogger attention) and a demand (companies) and the result should and will be a bidding system to see which companies get covered first.

Let me take the side of a blogger for a second. I’m a busy person. I’m doing a lot of things with the site, I’m enjoying life outside of the Internet (try to), doesn’t leave much time to blog. However, if somebody comes to me and says “Yo, here’s $XXX so you can take time out of your busy schedule to check us out” I’d be down with that. And if I’m really busy they’d be like “Yo, we know you’re REALLY busy, so we’ll give you $XXX*2 for your time!” I’d be down with that too.

If these companies approaching me have a strong overlap with my audience, then it makes sense for EVERYONE.

Since it makes so much sense, then why are we left with such crappy options such as ineffective advertising or garbage PayPerPost? There’s a big opportunity here for someone to make a lot of money. But it’s definitely a balancing act because you can very quickly get into PPP territory of crappiness.