A little info on the Crawler Search Engines
Crawler search engines are actually automated computer programs designed to roam the web and find web pages to index into their databases. The primary way the crawler engines find new sites is through a link pointing to that site. Crawler engines differ from the directory search engines in that your site will not be reviewed manually by a live person. Instead a computer program within the crawler search engine automatically adds your site to its index and then ranks your site on key terms based on a pre-determined set of criteria. The following table explains the key search engine requirements for the major crawler engines.
Crawler Chart - Submission Requirements
| Specification | | Google | | Alta Vista | | Lycos |
| Inktomi |
|
| Cost to Submit: | | Free | | Inclusion fee $39 for 6 months | | Inclusion fee $30 for 1 year |
| $39 for first url $25 for additional urls |
| Feeds Listings To: | | Yahoo, Netscape | | None | | None |
| AOL, MSN, iWon, HotBot, LookSmart, Overture |
| Receives Listings From: | | Yahoo, LookSmart and Open Directory increase page rank | | LookSmart | | Fast, Open Source |
| None |
| Title max length: | | 60 characters | | 78 characters | | 150 characters |
| 80 characters covers all partners |
| Uses Keyword Meta Tags? | | No | | Yes | | No |
| Yes and boosts rankings |
| Keyword Placements | | Weight and proximity important. Use in header and bold tags | | Use in title and top of page | | Use in body text, top of page and in header tag |
| Will depend on partner. General rule: Frequency and weight in body text, title and url |
| Likes Page Length of: | | 500 - 600 words | | 600 or more words | | 100 - 200 words |
| Will depend on partner. General rule: 100 - 250 words |
Intomi - Additional Notes
Inktomi is not a true search engine in that there is no way to directly query the Inktomi database as with Google, Alta Vista or Lycos. However the Inktomi search results are fed to several other secondary engines including AOL and MSN. Each secondary engine displays the Inktomi listings with a slightly different
twist or set of criteria. To submit your site's url to Inktomi use one of the Inktomi partner sites including:
Generally your site will be listed in the Inktomi database within 48 hours. An added benefit of this search engine is that your site will be crawled every 48 hours or so after inclusion allowing for continued optimization updates and quick web site ranking evaluations.
Google - Additional Notes
Along with proper site optimization techniques, Google employs a page rank or link popularity standard for each site it indexes into its database. Simply put, Google's page rank methodology considers a site more relevant for a key phrase if the site has other links pointing to it from similar key word them ed sites.
The number and quality of the links pointing to a site combine to either enhance or detract from the site's total link popularity ranking.
Ask Jeeves - Teoma
Ask Jeeves is a combination search directory and crawler engine. This search engine displays results from its own directory database, Overture, Teoma (a crawler based engine) and secondary search engines.
To obtain a listing in the Ask Jeeves database you can request your site be listed via email at url@askjeeves.com. You must associate your site to a question and will have the best results if your question is currently not listed in the Ask Jeeves database. Another way to get listed with Ask Jeeves is to pay for inclusion into the Teoma database. Currently Ask Jeeves displays Teoma listings after its own directory and Overture listings. The price for inclusion into the Teoma database is $30 for the first url.