huridocs-tech Search Engines Lag Behind Internet - 3 articles (fwd)

Edited/Distributed by HURINet - The Human Rights Information Network
---------------------------------------------------------------------
## author     : gagliajn@NETCOM.COM
## date       : 07.07.99
---------------------------------------------------------------------
          July 7, 1999
          By The Associated Press

          Internet search engines are not keeping pace with
          the explosive growth of the Web.

          A study found that search engines -- which enable
          a computer user to find information by typing in a
          word or combination of words -- cover a
          diminishing fraction of Web pages and take a long
          time to list new sites.

          The most comprehensive engine, Northern Light,
          covers only about one-sixth of the Internet pages
          that search engines can reach, the study found.
          That is down from one-third for the best engine a
          year and a half ago.

          Northern Light is closely followed by Snap and
          Altavista. Hotbot, which led with 34 percent
          coverage in the previous study, was down to 11
          percent.

          The study also found that it takes more than six
          months on average for a new Web page to make it
          into a search engine's listings.

          The study of 11 search engines was conducted by
          computer scientists Steve Lawrence and C. Lee
          Giles at the NEC Research Institute in Princeton,
          N.J. It was published in Thursday's issue of the
          journal Nature.

          Lawrence and Giles estimated that as of February,
          the searchable Web consisted of 800 million pages
          containing more than 6 trillion characters. Their
          December 1997 survey put the number of pages at
          about 320 million. By comparison, the 532 miles of
          shelves in the Library of Congress contain an
          estimated 20 trillion characters.

          Search engines use computers called ``spiders''
          that continuously surf the Web. They save each
          page they visit, then follow the links on the page
          to find other pages. When a user types in a word,
          the engine looks in its index to see which pages
          contain it. A page that's not listed in the index
          will not be found.

          The spiders are more likely to find pages that
          have more links going to them from other pages.
          Lawrence said that may make it hard for new sites
          to make it into search engine listings.

          Lawrence also said search engines may be lagging
          because their databases become more expensive as
          they grow, without necessarily creating more
          advertising profits.

          Marc Krellenstein, chief technical officer at
          Northern Light, said he believes the fraction
          covered by the search engines is actually larger,
          because he considers the study's estimate of the
          size of the Web a bit high.

          Krellenstein also questioned whether search
          engines need to concern themselves with every new
          site. He said they focus on good sites that are of
          more interest to users.

          ``The fact that the Web is growing so fast doesn't
          mean the quality part is growing as fast,'' he
          said.

=====================================================================
(2)

Internet Search Engine Comparison
By The Associated Press

Internet search engines and the percentage of the searchable
Web that they cover:

Northern Light: 16.0

Snap: 15.5

Altavista: 15.5

HotBot: 11.3

Microsoft: 8.5

Infoseek: 8.0

Google: 7.8

Yahoo: 7.4

Excite: 5.6

Lycos: 2.5

Euroseek: 2.2

Source: Computer scientists Steve Lawrence and C. Lee Giles
at NEC Research Institute in Princeton, N.J.

====================================================================
(3)       Some Tips for Using Search Engines
          By The Associated Press

          Computer scientist Steve Lawrence recommends Web
          surfers looking for something well known, like
          weather forecasts, use engines like Google or
          DirectHit that rank sites according to their
          popularity.

          ``These tend to be good for finding and ranking
          highly information which is well known and has
          been around for a while,'' he said.

          If you are looking for something very specific or
          obscure -- say, Victorian rocking chairs -- he
          recommends searching engines like Altavista or
          Hotbot and making the query as specific as
          possible.





----------------------------------
Send mail for the 'huridocs-tech' list to 'huridocs-tech@hrea.org'.
Mail administrative requests to 'majordomo@hrea.org'.
For additional assistance, send mail to: 'owner-huridocs-tech@hrea.org'.
Archives of previous messages posted to the list can be found at:
http://www.human-rights.net/huridocs-tech.