September 20, 2016
Before “Googling” became a verb, before Google developed page-ranking algorithms, and before Google became the backbone of today’s digital marketing infrastructure — how the hell did search engines work?
The earliest search engines began popping up in the early 90s, but they were vastly different from (and more primitive than) the search engines that we have today.
Let’s take a trip down computer memory lane, back to before Google became the reigning king of internet search.
The Beginning of the Web
In the beginning of the web, search engines often utilized indexed content of public FTP servers, which are archival servers used to transfer files between computers on a network.
Archie, created in 1990, was the first form of search engine that indexed FTP archives, allowing users to find specific files. An Archie server indexed the contents of public FTP servers, and users could search the file names in the indexed server.
An Archie server administrator had to manually add new public FRP servers to the Archie index upon request, and these servers took monthly snapshots of the files in each of the FTP servers.
Archie could not, however, display the contents for each site — only the listings. And although Archie was a good directory of indexed servers, the directories had to be manually created and offered limited search functionality.
Fun fact: the name “Archie” comes from the world “archive” — without the v.
The Rise of Crawlers
The next big step for search engines was the development of web crawlers, which are programs that automatically crawl the web for new and updated content. The introduction of automated web crawlers vastly sped up the indexing process.
JumpStation was the first search engine that implemented a web crawler to create a searchable index that documented titles and headings of websites. When users entered queries on JumpStation’s web form, the search engine returned a list of URLs that matched the queries.
A subsequent search engine called WebCrawler went a step further: its web crawler indexed entire texts of every single page. Hence, WebCrawler became the first search engine to enable full text search.
Then came Lycos, which pushed the limits of web indexing by showcasing a catalog of 54,000 documents during its launch in 1994, which skyrocketed to 60 million documents by 1996.
Advanced Search Techniques
Little by little, search engines became more sophisticated.
Excite, which debuted in 1995, became the first search engine to use word relationships and statistical analysis to produce relevant and accurate search results.
AltaVista, on the other hand, introduced natural language search queries — allowing users to type in what they were looking for instead of searching for strings of awkward queries.
AltaVista was also the first search engine to use the Boolean Operator, which uses the words AND, OR and NOT to broaden or narrow search results. For example, the query “mountain AND hiking” would include results that contain: “mountain hiking” and “hiking on tall mountains at night.” The query “ham OR eggs” would show results that include ham but not eggs, and vice versa.
Then, Ask Jeeves (now Ask.com) put natural language search to use by creating a question-and-answer model of search, which introduced a much more fluid and natural search experience.
The Rise of PageRank
The most groundbreaking development of search engines was what Google is famous for now: the PageRank system.
The basic idea of PageRank was this: internal and external links among sites could be used to measure the quality of websites.
This network of links became a great indicator of site authority — more incoming links meant the site is more trustworthy.
PageRank, along with other page-ranking algorithms such as Panda, Penguin and Hummingbird, became the main tool for deciding which sites would show high up in the SERP — thus creating the modern practice of Search Engine Optimization, or SEO.
And the rest is history.
------
Want to learn more about how Google ranks your site? Find out how you could be damaging your site’s Google ranking or get informed on how to build up your link juice with internal links.
Do you know any other search engines that came before Google? Tweet us @Perfect_Search.