See the New Growth Report

submited by

Style Pass

2021-07-22 10:00:08

The World Wide Web was started originally proposed in 1989 and the first implementation appeared in 1990. The Web however, did not gain any widespread popular use until NCSA Mosaic became available in early 1993.

In Spring of 1993, I wrote the Wanderer to systematically traverse the Web and collect sites. I was initially motivated primarily to discover new sites, as the Web was still a relatively small place. As the Web grew rapidly, the focus quickly changed to charting the growth of the Web. This report covers the period from June 1993 to June 1995.

Tools The primary tool used to collect the data presented here was the World Wide Web Wanderer, the first automated Web agent or "spider". The Wanderer was first functional in spring of 1993 and performed regular traversals of the Web from June 1993 to June 1995.

Methodology The Wanderer was run on a monthly basis from June 1993 to June 1995. While the Wanderer certainly did not reach every site, it was run with consistent methodology, hopefully yielding consistent data for the growth of the Web.