Sign up FAST! Login

How big is The Web?

Stashed in: Google!, Teh Internets

To save this post, select a stash from drop-down menu or type in a new one:

I'm trying to figure out how big the public web is, and how big the "dark web" or "deep web" is.

As of a year ago, the public web grew by 150k URLs each day.

It's probably growing even faster now.

Cisco estimates of how big the Internet is:

Cisco, which releases a Visual Networking Index forecast about the scale of web traffic every year, estimates that by 2015, 966 exabytes of information will be consumed annually by the internet – an increase of 200 exabytes over the estimate for 2014, which is itself more than the amount of all 2010 information consumption online.

By comparison, it took until 2004 for monthly internet traffic to pass 1 exabyte for the first time (For those wondering what an exabyte is, it’s 1,000,000,000,000,000,000 bytes; it was once estimated by UC Berkeley that “all words ever spoken by human beings” could be stored in 5 exabytes, although that would likely have to be in text format).

Read more:

Google had a trillion URLs collected as of 2008:

By 2011 there were a trillion actual Web pages, according to Kevin Kelly: calls it the Invisible Web:

Google's index is between 40 and 50 billion pages: