Search engines examine all the pages on the World Wide Web, categorise them and put them into a logical order when you search for something. Understanding how this works can help your business. This video will cover:
- how search engines find web pages
- what they do with the web pages they find
- how they decide what to show on search results pages.
Search for a few terms that interest you, or for the products and services that you sell. What types of results do you see on the search engine results pages? Do you notice things on top-ranked sites that might explain why they appear in such prominent positions? Make a wish list of content you might add to your website, so youâll have more opportunities to be in the index, and rank well on search results pages.
View Transcript
Hi there! Welcome to our video explaining how search engines work. Weâll give you the basics on how search engines find web pages, what they do with the pages they find, and how they decide what results to show. When youâre using a search engine to find the closest coffee shop, youâre probably not thinking about search engine technology. But later you might wonder, how did it do that? How did it sort through the entire Internet so quickly, and choose the results you saw on the page?
Each search engine uses their own software programs, but the way they work is pretty similar. They all perform three tasks: First, they examine content they learn about and have permission to see (thatâs called crawling). Second, they categorise each piece of content (thatâs called indexing). And third, they decide which content is most useful to searchers (thatâs called ranking).
Letâs take a closer look at how these work. Search engines âcrawlâ the Internet to discover content, like web pages, images and videos. Each search engine uses computer programs called âbotsâ (short for robot), âcrawlersâ or âspidersâ to make their way through the pages.
The bots hop from page to page by following links to other pages. These bots never stop; their sole purpose is to visit and revisit pages looking for new links and new content to include in the index. Indexing is the second part of the process. The index is a gigantic list of all the web pages and content found by the bots. The search engine uses this index as the source of information displayed on the search results pages.
But, not everything the bots find makes it into a search engineâs index.
For example, search engines may find multiple copies of the exact same piece of content, located on different websites.
How is that possible? Well, imagine youâre not searching for a coffee shop, but a coffeemaker. You might notice that the top-of-the-line CoffeeKing2000 has the same word-for-word description on the websites of many major retailers. The description might have been provided by the manufacturer⊠but now the search engine has decisions to make: which version to keep in the index? Thereâs no need for hundreds of duplicates, so itâs unlikely that every page will be added.
So if you own a website thatâs selling coffeemakers, youâre likely better off writing your own description of the CoffeeKing2000.
Make sense? That covers crawling and indexing, which just leaves us with ranking. When you type in a search, the engine compares the words and phrases you use to its index, looking for matching results. Letâs say, for example, the search engine finds 230 million matching results. Now itâs time for the last part of the search engineâs task: ranking.
The way search engines rank pages is top secretâitâs their âspecial sauce.â There are hundreds of ways search engines determine rank, including things like the words on the page, the number of other websites linking to it, and the freshness of the content.
But no matter what formula they use to determine rank, the goal remains the same: to try to connect the searcher with what they are looking for.
Say youâve read about an Australian-style cappuccino called a flat white and you want to try it. If you search for âflat white coffee near meâ the search engine will show you nearby shops selling the drink, because your search indicated your location. You might even see a map to help you find them. So, what have we learnt? Search engines are constantly working to scour the web for content, organise it and then display the most relevant results to searchers. Understanding this process will help you make your website the best it can be.