About Search Engines

How to Search on Search Engines

Type the following combinations in Google (or other engines) search bar to limit your search. Pay attention to spacing, must be used precisely. Limit your search to the following:

Web page title
allintitle:seo ottawa
intitle:seo ottawa

Web pages that contain link titles with your desired term
allinanchor:seo ottawa

Website or Domain
site:whitehouse.gov “global warming”
site:edu “global warming”

File type
filetype:ppt site:edu “global warming”

Definitions
define:pixel
define:“due diligence”

Specify exact phrase
“tom bates”
“what you're looking for is already inside you”

Exclude or require a word ('minus' or 'plus' with no space after)
proliferation -nuclear
seo +internet marketing

Source: Research quality web searching from U of Berkley

How Search Engines Work

Search Engines for the general web (like all those listed below) do not really search the World Wide Web directly. Each one searches a database of the full text of web pages selected from the billions of web pages out there residing on servers. When you search the web using a search engine, you are always searching a somewhat stale copy of the real web page. When you click on links provided in a search engine's search results, you retrieve from the server the current version of the page.

Some of the major search engines are Google,  Altavista, MSN, Excite, Hotbot, Infoseek, Lycos, and Webcrawler. Note that Yahoo is a directory, not a search engine. The term Search Engine is also often used to describe both directories and search engines.

Search engine databases are selected and built by computer robot programs called spiders. Spider is that part of a search engine which surfs the web, storing the URLs and indexing the keywords and text of each page it finds. For example, Google's spider, also referred to as a "crawler", is named Googlebot. Although it is said they "crawl" the web in their hunt for pages to include, in truth they stay in one place. They find the pages for potential inclusion by following the links in the pages they already have in their database (i.e. already "know about"). They cannot think or type a URL or use judgment to "decide" to go look something up on the Internet.

If a web page is never linked to in any other page, search engine spiders cannot find it. The only way a brand new page - one that no other page has ever linked to - can get into a search engine is for its URL to be sent by some human to the search engine companies as a request that the new page be included. All search engine companies offer ways to do this.

After spiders find pages, they pass them on to another computer program for "indexing." This program identifies the text, links, and other content of the page and stores it in the search engine database's files so that the database can be searched by keyword and whatever more advanced approaches are offered, and the page will be found if your search matches its content.

Some types of pages and links are excluded from most search engines by policy. Others are excluded because search engine spiders cannot access them. Pages that are excluded are referred to as the "Invisible Web", i.e. what you don't see in search engine results. The Invisible Web is estimated to be two to three or more times bigger than the visible web. (Source: University of Berkeley).

Here is a short but great overview page on search engines from U of Berkley. Some useful tips are included.

Statistics on search engines from WordTracker

According to Wordtracker (2005 info), three search engines get roughly 70% of all traffic, with the following distribution among them:

• Google gets 47.0500% of all search engine traffic (approx. 268 million a day).

• MSN gets 9.0000% of all search engine traffic (approx. 51 million a day).

• Yahoo engine gets 12.7700% of all search engine traffic (approx. 73 million a day). Yahoo gets 12.7700% of all search engine traffic (approx. 73 million a day).
  The table below shows only the engines included in the WordTracker system; who they power, and who powers them. Pay per bid engines other than Overture have not been included.

The table below shows only the engines included in the Wordtracker system; who they power, and who powers them. Pay per bid engines other than Overture have not been included.

 

Included Engine	Main Results From	Powers
AllTheWeb	Yahoo! Search Marketing, Yahoo	None
Altavista	Yahoo! Search Marketing, Yahoo, Open Directory	None
Google	Open Directory	Lycos, AOL, Ask, Hotbot, Dogpile, Netscape, Alexa
HotBot	Ask, Open Directory, Google	None
Lycos	Looksmart	None
LookSmart	Inktomi	Lycos
MSN	Yahoo! Search Marketing	Dogpile
Open Directory	None	Lycos, Altavista, Google, Netscape, Alexa, AOL, Hotbot
Ask	Google	Lycos, Dogpile, Hotbot
Yahoo	Yahoo! Search Marketing	Altavista, Alltheweb, Dogpile
Yahoo! Search Marketing	None	Altavista, Alltheweb, Yahoo, MSN, Dogpile