Keyword foraging: the key to better search tools

Before you look up something online, you might first need to figure out what keywords will help you find it. This has become a common enough occurence that the usability experts at the Nielsen Norman Group (NN/g) recently released a term for it: Keyword foraging.

In their article, they focus primarily on how online creators and businesses could use knowledge about keyword foraging to ensure their content can be found. In one example, they discuss how someone searching for a “duster cardigan” may have to start by using less precise terms, such as “long sweater cardigan.” However, I was more intrigued by an observation made later in the article:

Search tools often fail when users can’t supply the right query terms.

This issue has captivated my imagination for quite some time. Based on my experience helping startups identify competitors, many people simply assume that if nothing comes up when they search using certain terms, then there’s likely nothing to be found. In many cases, however, after I’d encouraged these entrepreneurs to use synonyms or search LinkedIn for people in the industry, we would find a keyword that turned up a bevy of competitors.

A while back, the notion of keyword foraging came to mind as I explored the role of designers who make products that rely heavily on AI and machine learning. If you look up keyword phrases like “design and machine learning” or “designing AI” on most search engines, you’ll be shown engineering resources and research articles about how to design the code and algorithms themselves. This isn’t what I was looking for at all, so I had to begin the process of figuring out what alternative keywords I could use, just like the entrepreneurs I once guided.

When I finally used the phrase “AI and designers”, I started to uncover more relevant results. That specific term led to some pertinent blog posts, authored by people whose names I then used to unlock further information.

What struck me as problematic about this situation was not the effort required to find the right search terms, but rather how similar the phrases “designing AI” and “AI and designers” are, despite yielding wildly different results. The problem is that search engines have over-indexed the verb design to the engineering aspects of artificial intelligence rather than the work of a product designer like myself. To overcome this, I had to use a term more explicitly reserved for the field of product design: “designers”.

Although I couldn’t be sure this would get me the results I wanted, I figured it would at least turn up designers indexed alongside the term “AI” and that their online profiles might lead me to my bounty. Unfortunately, this whole process was unnecessarily tedious. If I were to ask any human about “designing AI,” they would immediately understand what I, as a designer, meant by “designing” in the context of artificial intelligence. Today, looking up those same terms will actually bring up much more relevant results as more designers started writing about “designing AI.”

In their article, the Norman Nielsen Group writes that users with “lower research skills’’ are the ones who suffer from the frequent need to forage for keywords in order to find relevant results. While this might be unfortunate for a shopper searching for a particular style of cappuccino-coloured shoes, it’s downright awful for people who end up reading misinformation about an important topic simply because a search engine misinterpreted their keywords.

Another issue with keyword foraging is that the internet is mostly filled with low-quality content that manipulates keywords to claim the top spot on the first page of Google. This means that typing in something like “stomach pain” will turn up a bunch of blogs trying to sell you an herbal cure rather than provide legitimate medical resources about an H. pylori infection (based on personal experience).

In my opinion, the most upsetting part is that we’ve created “helpful” technologies that require us to adapt to them rather than make them adapt to us. So long as search engines are designed to prioritize keyword-optimized content, they won’t reliably turn up insightful results.

Some people advocate for computer literacy to be more widely taught as a way to overcome some of the tech’s shortcomings in understanding us. This was even proposed by NN/g themselves. However, computer literacy too often focuses on teaching people to cope with existing technology rather than understand it. And yet, without this fundamental understanding, how can we expect people to know when to demand more from search engines?

Since we can’t just overhaul the entire system tomorrow, for now we have to mold ourselves to the existing search paradigms. Now that keyword foraging is a coined term, here’s how you can learn from it to shape your process when building new websites and writing content.

Avoid inventing new terminology

This first suggestion comes from the NN/g article:

language people might use to search for your products or content if they didn’t know

Though the article makes this seem like a problem only for e-commerce, in fact all kinds of apps and blog posts suffer from it as well. I’ve even written an article about what we can call new tools like Notion, AirTable, Coda, etc. because each of them was using their own terms even though they mostly do the same thing.

Always express things in the simplest terms

It’s very easy to lean on acronyms and technical terms when writing a new post because they provide shortcuts to talking about big concepts. My wonderful editor was slapping my wrists when I used “SEO” in this article. You might be thinking if you’re writing for a specific audience, then using technical terms would be alright. To address that, I’d like to refer you to this wonderful conversation with interface design legend, Steve Krug about writing.

In it, he says that even if you believe your audience knows a subject, the experts reading will always appreciate a well-put, simple explanation of a concept.

Add alt text to your images

Alt text (alternative text) is used within the code of your site to describe an image’s appearance and function on a page. There’s a lot of focus on how this could help people with visual impairments understand what’s on your site, but as with many accessibility upgrades, everyone ends up benefitting.

Adding alt text is quite easy to forget but it can be an ideal place to overcome some of the challenges with keyword foraging in contexts that are highly visual. The old adage of an image being worth a thousand words could not be more true. From my own experience, I have come across interesting content countless times through Google Image search thanks to the thoughtful alt text that was added to it.

One of the better resources out there for understanding search engines is MOZ, and they have an article on how you can optimize your alt text.

Explore search techniques that don’t rely on keywords

In 2004, a miracle of search happened for us music lovers: Shazam. The Shazam app completely changed our perception of how we can search for music. Their promise was: “pull out your phone and hit record, we’ll take care of the rest.”

No more scrambling to remember the lyrics so that you can later put them in Google.

Image search is growing in popularity with Google leading the way, but there are some other efforts by Amazon and Spotify as well. Both of these companies provide a simple way for their users to pick up their phones and scan a bar code to find the product.

Keep up with the latest in natural language processing

The development of artificial intelligence is moving at an incredible pace, and new advances in the way natural language is processed is enabling all kinds of new possibilities. That latest such development is called GPT-3, and it’s poised to significantly alter the way we search online. Check out the tweet below for an example of what it could do to search:

Perhaps in a few decades, we’ll have developed an entirely new way of searching because machine learning will have become more sensitive to context. Until then, we risk having far too many people locked out of their own curiosity if we don’t start implementing small new techniques today and thinking of bigger picture solutions for the future.