Company
Date Published
Author
Josh Dzielak
Word count
3449
Language
English
Hacker News points
None

Summary

Google's famous "I'm feeling lucky" button is a standing invitation for users to test their relevance by jumping through directly to the top hit. Great relevance requires a careful understanding and application of complex techniques, several of which we'll look at in this article. Consumer-quality search experiences are streamlined that it’s hard to imagine what’s going on underneath, even for engineers. Instant search introduces a brand new set of relevance challenges. The search engine can't wait for a full word to be typed, it has to find and rank matches using only the first few letters. On-the-go users misspell, omit and juxtapose words, and frequently get distracted and stop typing mid-sentence. This requires search engines to be much more forgiving than they were originally designed to be. Thousands of pages on the internet can tell you how to treat the common cold. To solve this problem for web search, Google created the PageRank algorithm which determines a page’s importance based on how many other pages link to it (and what those pages’ rankings are). Consumer-quality search experiences have become the standard since Google Instant was introduced in 2010. Search-as-you-type experiences introduce new relevance challenges because the search engine can't wait for a full word to be typed, it has to find and rank matches using only the first few letters. Instant search produces real search results with every keystroke, no clicks or enter keys required. The user benefits from a streamlined experience and the development team has less moving parts to tune and maintain. However, achieving the necessary speed and relevance is much harder for instant search than instant suggest. Mobile keyboards make forgiveness a must-have for two reasons: it’s easier to make mistakes and it takes longer to correct them. To make the search engine more forgiving, we’ll need to apply a variety of text and language processing techniques, designed to handle everything from typos to languages like Dutch and German that don't always have spaces between words. Algolia is built for full instant search, while Elasticsearch has limitations when it comes to speed and relevance. The path to great relevance looks different whether you are using Elasticsearch or Algolia. Recent Elasticsearch additions like the Completion Suggester are designed to help developers build an instant suggest experience that is fast enough for as-you-type search, with a tradeoff of some functionality.