Search Engines Should Be Opinionated

One of my fundamental beliefs is that almost everything is a search problem, I believe that for many people, there probably exists an advice or piece of information out there, that if revealed to them in this moment can significantly improve their life, but they’re what in Chinese you might call 可遇不可求 - that you may chance upon it, but you can’t seek it out.

The beautiful thing about search is that it’s a method of turning those things you could only chance upon into things you can seek out. Before web search, your ability to seek out information was largely limited by the content that’s in your local library.

The main issue I have with search engines is that the top results are very generic - like creative writing from an LLM, or art created by an image gen model - they’re smoothed out across all preferences and you get a very ‘flat’ result, something satisfactory to the most number of people, but not particularly attractive to anyone.

For example, if I’m a young, technical person who doesn’t just want to work in big tech, Googling blogs for young ambitious technical people wanting to make a disproportionate impact in the world surfaces:

• a medium interview of the founder of HomeHero, a platform for paying bills.
• an instagram reel from the world bank about troubles young people face when entering the work place
• an article titled ‘The Art of Activity Innovation: How to Be Impressive Without an Impressive Amount of Work’

When in reality what you want is probably 80000 hours + blogs about open problems in AI safety, synthetic biology, nuclear fusion, material science etc.

Interestingly for finding these ‘high taste’ content, social networks like Instagram or Twitter can work better than Google. A great example is fashion - Googling pants in the style of Yohji Yamamoto by indie designers under 300 dollars should in theory surface hundreds of small time shops each with a dozen or so items, but in reality always surfaces other high end avant-garde designers like CDG, or even worse, simply more Yohjis. However, simply liking a few posts from Yohji on Instagram will flood your feed with these labels you can’t find on Google (albeit many of them still a poor match, and they’re mostly ad driven, so you’re still not seeing the truely good stuff you find by walking through a random side street in Tokyo)

So I guess I want my search engine to be opinionated, it should be able to surface me the one link that’s perfect just for me, and the 10 blue links should not be homogenous all of the time. Especially in the age of AI, we have agents that aren’t lazy, they can form long comprehensive queries, they can apply multiple filters, and they can read through lots of results at once, a truely great search system should honestly be able to take my diary, and when I ask a question be able to apply a dozen or so ‘soft filters’ and find me a list of results that perfectly address my problem - no matter how obscure.