Search Engine Basics

Search engines catalogue the Internet to help connect searchers with exactly what they’re looking for. That makes them a great marketing tool. So where did search engines come from? One of the earliest search engines was a program called Archie, which debuted in 1990 and allowed people to access and search file names— basically the names of the web pages. But, Archie couldn’t tell you what was on those pages.

Fast forward a few decades, and search engines like Google, Bing, Yahoo!, Ask.com, AOL, Baidu, and Yandex have come a long way. These search engines use incredibly sophisticated computer programs to sort through a massive number of web pages.

Most search engines basically work in the same way. When a person wants to find something, they type in a word or phrase, called a search query. Then, the search engine compares that query to its catalogue of web pages, pulling out the best matches to show the searcher. These are displayed on a search results page.

Their goal, is to create the most relevant list of results possible, to help searchers find what they are looking for. The results page includes links to websites, but you might also see local business listings, items for sale, advertisements, images, maps, videos and more. So how does this apply to you?

Why? Because the words entered into the search engine indicate the searcher is interested in your products and services, right now. Search is a great place to be because it’s a way to target people who are already looking for you.

(1) How Search Engines Work

Search engines examine all the pages on the World Wide Web, categorise them and put them into a logical order when you search for something. Each search engine uses their own software programs, but the way they work is pretty similar. They all perform three tasks – Crawling, Indexing and Ranking. First, they examine content they learn about and have permission to see (that’s called crawling). Second, they categorise each piece of content (that’s called indexing). And third, they decide which content is most useful to searchers (that’s called ranking). Let’s take a closer look at how these work. 

(a) Crawling 

Search engines “crawl” the Internet to discover content, like web pages, images and videos. Each search engine uses computer programs called “bots” (short for robot), “crawlers” or “spiders” to make their way through the pages. The bots hop from page to page by following links to other pages. These bots never stop; their sole purpose is to visit and revisit pages looking for new links and new content to include in the index.

(b) Indexing 

Indexing is the second part of the process. The index is a gigantic list of all the web pages and content found by the bots. The search engine uses this index as the source of information displayed on the search results pages.

But, not everything the bots find makes it into a search engine’s index. For example, search engines may find multiple copies of the exact same piece of content, located on different websites. How is that possible? Well, imagine you’re not searching for a coffee shop, but a coffeemaker. You might notice that the top-of-the-line FruitJuicerT2000 has the same word-for-word description on the websites of many major retailers. The description might have been provided by the manufacturer… but now the search engine has decisions to make: which version to keep in the index? There’s no need for hundreds of duplicates, so it’s unlikely that every page will be added. So if you own a website that’s selling coffeemakers, you’re likely better off writing your own description of the FruitJuicerT2000.

When you type in a search, the engine compares the words and phrases you use to its index, looking for matching results. Let’s say, for example, the search engine finds 230 million matching results. 

(c) Ranking 

Now it’s time for the last part of the search engine’s task: ranking. The way search engines rank pages is top secret—it’s their ‘special sauce.’ There are hundreds of ways search engines determine rank, including things like the words on the page, the number of other websites linking to it, and the freshness of the content. But no matter what formula they use to determine rank, the goal remains the same: to try to connect the searcher with what they are looking for. 

(2) Optimise your pages

How Search Engines see the web In simple terms, when you ask a search engine to find something, it looks through a huge list of previously indexed pages, called “the index,” and pulls out relevant results based on what you’re looking for.

Pages make it into “the index” only after the search engine has determined what they’re about. That way, it can file them in exactly the right place amongst the other pages, and find them the next time a search relates to their content.

By knowing “how” a search engine decides what a page is about, you can “optimise” your pages to make sure they show up in the search results of people looking for websites just like yours.

(a) HTML

Let’s say you own a juice shop, and you’ve got a website to promote it. When you look at a page on the site you see the imagery and text content. But when a search engine looks at the same page, in addition to seeing what you see on your screen, it also sees the code behind it, called HTML (HyperTextMarkupLanguage). Specific parts of this code help the search engine understand what the web page is all about. And knowing which parts are important can help you to optimise your site. First, the title of the page in the code. In this example, you can see the title in the tab at the top: “Juice Shop.” The search engine sees the title enclosed in a piece of code called a title tag. Many websites can be edited using tools that handle all the HTML coding for you – that’s called a content management system, or CMS. If you use a CMS to make changes to your website, there’s probably a place to add this title, too. You can help the search engine index your page properly by making sure your page title accurately describes its content. That way it can show up in relevant searches.

(b) Text

The next thing you’ll want to think about is the page’s text. Think about who you want to visit your page, and what words they’re using to describe your products and services. Do they talk about fresh grape juice? Fresh grape juice with pulp – these are probably the terms they’re also using to search. Try to speak the language of your customers when you write your content. Because this can help ensure they’ll find your pages when they search.

(c) Alternate Text

Finally, let’s talk about the page’s images. Search engines won’t see the mouth-watering photos of your juice creations in the same way we do – which is a shame. But what they will see is the code behind it. To help search engines identify the image, give it a descriptive name. For example, image.jpg is not a great file name for search engines. Whereas, something that describes exactly what’s in the picture, like fresh-grape-juice.jpg, is.

You can even take it one step further by adding “alternative text” in the code with your image. Known as an “Alt tag,” it describes the image, which is useful for people using web browsers that don’t display images, or for people with visual impairments who use software to listen to the content of web pages. In the HTML , the ALT tag will appear something like this: src=”http://www.example.com.com/fresh-grape-juice.jpg ” alt=”Fresh Grape Juice”>

So remember: Use descriptive, unique titles for each page on your site. Write for your customers, but remember to include important words and phrases that can help search engines understand what your pages are all about. And don’t forget to name image files with descriptive words and include alternative text.

(3) Popular Search Engines