Panning for Gold With Search Engines

Search engines kind of suck right now.

Let me explain. For the average user trying to drum up useful information, the first few pages of results after the ever-pervasive sponsored links are often a battleground of terrible content that has gotten really good at gaming the system.

How often have you wanted to find in-depth information about a specific hotel or place to visit on vacation, only to be inserted into a nightmarish maze of Tripadvisor reviews and crappy travel sites, SEO’d straight to the top of the results? Or, when looking for factual information, come across robot content from Demand Media or Associated Content that is auto-generated based on keyword popularity? Just the other day, I was researching the evolution of Marine Corps boot camp post-Vietnam (odd, I know), only to find a content-farm penned piece, “How to Survive Marine Boot Camp,” written by a 48-year-old woman in middle America who clearly had no firsthand experience. Not likely the best source of information.

We’re creating a lot of useless junk and the gems are getting harder to find. In reality, some of the best information comes mainly from two places: word of mouth from your friends; and people that have niche expertise often found on a post buried on a message board that hasn’t changed in style or format since 1994 (essentially inverse SEO). With the former, there’s not exactly a formalized system to search and process this information. With the latter, it’s often hard to find and hard to suss out the credible content.

With the proliferation of content over the past few years, we’ve created too many rivers and not enough dams. And the dams we need today need to be much smarter than a mere barricade. They need to be filters that are only letting in hyper-contextual information, optimized for one’s own social and taste graphs, or update information from expert sources. Sounds super cerebral, but it really isn’t.

There are companies experimenting with ways to make the system work better. Quora, for one, seeks to replicate word of mouth by allowing you to ask questions to niche specialists in a qualified community of socially validated subject matter experts. In its early stages, this is incredibly useful due to the engaged early adopters using it. Netflix CEO Reed Hastings promptly answered a question asking how much the company pays annually in postage (between $500-600 million per year, for those keeping score). The travel section has interesting information about where to stay and how to make business trips more efficient, penned by people logging 100,000 miles or more annually. I do wonder, however, how this type of community will remain useful when it approaches any type of scale. Will it become Yahoo Answers, which has around a second-grade level of discourse on any given subject?

Another interesting approach is Hunch, a service that seeks to personalize the Internet to each user’s tastes and preferences by getting to know them through both a series of questions and Facebook connect data. It looks as if Hunch will soon allow users to port their own taste preferences into other sites. So, you could get tailored Amazon search results based on a wider set of criteria drawing upon questions you answer, your network of friends, and what it can deduce from all these connections.

But the real opportunity here is something slightly different. We need a “priority inbox” for all of our feeds and information, the discovery part based on social validation from our friends and a simple, adjustable algorithm. My guess is it isn’t too far away from reality. Until then, I’ll keep wading through the river of junk to pan for the gold.

Colin Nagy is an executive director at The Barbarian Group. He can be reached at cnagy@barbariangroup.com.