Beyond the confines of your everyday online lives there is a hidden internet known as the Deep Web.
It’s called the Deep Web because it is just that. Think of it as a vast uncharted ocean if you will, sometimes dangerous to navigate, and often peppered with surprises. There are literally tens of trillions of pages lurking below the surface, offering everything from documents and statistics to human organs for sale. It is estimated that the surface web we are so familiar with makes up less than 1 per cent of the entire world wide web.
Michael K Bergman, a leading authority on the internet and more specifically the Deep Web, undertook research in the late 90s in an attempt to gauge its scale. He told The Guardian, “I remember saying to my staff, ‘It’s probably two or three times bigger than the regular web’,” he said. “But the vastness of the deep web… completely took my breath away. We kept turning over rocks and discovering things.”
In 2001 Bergman published a paper on the Deep Web that is still regularly cited today. “The Deep Web is currently 400 to 550 times larger than the commonly defined world wide web,” he wrote. “The value of deep web content is immeasurable… internet searches are searching only 0.03 per cent… of the [total web] pages available.” Over a decade later, nobody quite knows by how much the Deep Web has grown.
Although much of the Deep Web consists of unremarkable consumer and research data that is beyond the realms of search engines, it has been cast into the media limelight recently due to nefarious sites dealing in illegal arms, child pornography, drugs, hit men for hire and everything else in between.
Bright Planet, a Deep Web intelligence service founded by Bergman, refers to this part of the Deep Web as the Dark Web, and is careful to note the distinction between the two. While the deep web refers to anything that cannot be picked up in a conventional search engine, only the dark web is truly anonymous. A spokesman from Bright Planet told The Newsroom, “The Dark Web refers to any web page that has been concealed to hide in plain sight or reside within a separate, but public layer of the standard internet. While personal freedom and privacy are noble elements of the Tor network, the ability to traverse the internet with complete anonymity nurtures a platform ripe for illegal activity.”
Despite the fact it is used by millions, the dark web remains largely ignored by the greater population and has only recently garnered media attention through well-publicised shutdowns of illegal sites. Only last Thursday a joint effort between 16 European nations shut down 400 dark web sites including Silk Road 2.0 – the newly refurbished illicit drug marketplace which sprang up shortly after the original was shut down. As is usually the case, a new website named Silk Road 3.0 popped up in its place almost immediately.
Accessing the Dark Web is not as hard as you might think. Entry is granted through a straightforward download of a dark web browser, the most well-known of these being Tor, or the Onion browser. Tor allows for browsers to access any Deep Web site with the .onion host suffix completely anonymously. Debuting as The Onion Router project in 2002, it was designed by the US Naval Research Laboratory as a means for anonymous online communications.
Essentially nothing you do on the Dark Web can be censored or monitored. It runs on a relay system that bounces signals among different Tor-enabled computers around the world. Naturally this allows for the aforementioned host of odious sites, but did you know there are far more non-illegal sites that range from the bizarre to the revolutionary?
Amongst publications such as The Anarchist Cookbook and the more ominously named Defeating Electromagnetic Door Locks, you’ll also find an active forum “for those who like spanking”, as well as copious amounts of erotic fiction and more mundane book clubs.
Moving away from the quirky and sinister, however, the dark web also provides an important political space free from censorship. Ever since Edward Snowden revealed the extent of government surveillance, downloads of the Tor browser have increased dramatically. As Tor’s surface website states, the browser is used by “activists, businesses and journalists” among others. But as well as the numerous hacktivist sites, the Dark Web also serves as an important tool for political dissidents and whistleblowers to speak freely.
Tor is particularly valuable in nations such as China, where the internet is highly censored and citizens cannot access social media sites like Twitter. Many Syrians also posted videos of the Syrian uprising on the dark net, free from fear of political backlash.
The enormous cyber world that lies beneath the surface web is perhaps as vast and unpredictable as the ocean itself. In some ways it offers community spirit, even idealism, in other ways it offers the illegal, immoral and downright morbid. Though it’s not a place you’d want your kids to frolic around unobserved, it seems we cannot deny the vital safeguard it provides against censorship.
While aspects of the Deep Web are inherently wrong, true anonymity and freedom of speech are natural rights that have, in this day and age, become increasingly valuable and difficult to find. – Thea Carley
Thea Carley is a former editor of The Newsroom.
Top photo from Stian Eikeland’s Flickr photostream.