We browse only five percent of the internet. Here’s what happens in the other 95 percent

Very little is known about what takes place in the dark web

There’s a reason why the dark web is seen as the seedy underbelly of the internet.

It’s rife with criminal activity – including child pornography, illegal drugs, and contract hitmen. Trying to get an accurate picture of what all takes place there is an insurmountable task as the system is designed to be anonymous and encrypted.

The dark web is a subgroup of the deep, or hidden web, explains Avi Kasztan, CEO of SixGill. The hidden web is the part of the internet that’s not accessible by search engines. Avi claims it makes up “95 to 96 percent” of the total internet landscape.

Most of the hidden web is fairly innocuous. It includes things like databases, pages behind paywalls, and registration forms. But dig a little deeper and you’ll start stumbling across the grisly stuff.



“People [can] use the hidden web to do bad things,” outlines Avi. “But the dark web is a completely different story. It’s much, much more chilling. It’s been made anonymous and encrypted on purpose.”

Avi knows what he’s talking about. His startup, Sixgill, crawls the dark web’s social networks (yes, you read that right) to detect and defuse threats before they’re carried out. The startup’s team comprises of former intelligence officials and cyber security experts. It recently pocketed US$5 million to scale operations.

Dark origins

The most popular way to access the dark web is through the anonymous browser Tor, developed by the US navy to facilitate secure communications. Avi says the project had “very good intentions” as it tried to enable people to communicate anonymously in places where there was no freedom of speech.

In 2004, the navy released the code for Tor under a free, public license and scientists Roger Dingledine and Nick Mathewson were given funding to continue its development. In 2006, both Roger and Nick, along with five others, founded The Tor Project, a research-education non profit aimed at helping democracy advocates in authoritarian states.

The original developers and organisations like the Electronic Frontier Foundation maintain that Tor is principally used by journalists, activists, and whistleblowers. However, the Tor network also started becoming associated with criminal activity. It’s unclear when exactly that happened. What’s sure is that cryptocurrencies like Bitcoin have helped the underground economy to flourish.

To provide some context, a study by researchers at Carnegie Mellon indicates the size of the underground economy in the dark web to be in the region of US$100 million – with the overwhelming majority of transactions conducted in Bitcoin.

Penetrating the network

But even if you download Tor to access the dark web, you won’t know where to go. That’s because there’s no Google equivalent that’ll help you find whatever it is you’re looking for.



“People don’t go there to make new friends,” laughs Avi. “There are directories, but even those give a very small picture.”

One of the reasons the dark web entered mainstream discussion was Silk Road – the marketplace where you could do anything from purchasing heroin to hiring someone to kill an enemy. It went bust after the founder – who went by the online persona Dread Pirate Roberts – was uncovered in an FBI sting operation.

“Silk Road was one story, there are many, many more currently operational,” says Avi.

“People gather in the dark web social networks to commit things that are not good. You want to steal a bank, attack a country, or gather a group of serial killers. The imagination is the limit. But in these social networks there are hierarchies – number one, number two, number three.”

Penetrating these networks isn’t easy. Just like in real life, you need to prove your loyalty and allegiance before you’re allowed in. Some may ask you to rob a bank and wire them part of the bounty. Others demand far more sinister things: attacking a government agency or assassinating an individual.

So who’s at the top of the foodchain? International crime cartels? Terrorists?

Avi refrains from giving a specific answer but says they’re very sophisticated groups with varying intentions. Some may just want lots of money to retire on an uninhabited island. Others want to see destruction and pandemonium.

What Sixgill has done is develop technology that helps it identify the areas of the dark web where criminal activity takes place. Extracting data is another challenge as the last thing these people want is to get caught.

Ironically, the anonymity that’s supposed to protect these criminals is also their weak spot.

“They want friends, they want to be recognised, these are all innate psychological needs. Recognition is very important,” notes Avi.

It’s these human fallibilities that the team leverages. Sixgill might create a bunch of digital avatars, dupe them into thinking they’re actually communicating with a beautiful, blonde girl for example. But it’s far more complex than just luring them into a honeytrap operation.



“The group of people being targeted live and work in groups, part of an ecosystem of cyber crime/terror. They cannot and they do not work alone […] they change names, places, etc,” explains Avi.

In order to identify these criminals and accurately pinpoint their next target, the startup uses its digital personas to penetrate part of the network. For example, it may extract disparate information from various parts of the chain. It will then harness the power of big data and machine learning to understand the key players and their plans.

“The name of the game is connecting the dots. You may extract information from here and there but if you don’t know how to put it together, it’s useless. That’s where machines are always better than humans, and can analyse the data far more efficiently,” explains Avi.

Clients are offered a SaaS dashboard where they can monitor, in real time, potential threats and data breaches. Sixgill will also provide updated alerts so that attacks are quelled from the outset.

Scary, but true

The startup works with several large organisations, including financial institutions, government bodies, and intelligence agencies. Avi says they’ve quelled “a lot” of threats since launch and that they’re struggling to keep pace with demand.

Part of the cash they’ve raised will be invested in strengthening product and “expanding bandwidth.”

“The world has changed. The amount of damage cyber attacks have caused is tremendous and the market is huge. You need to act proactively, understand their intentions, and prevent harm.”

This article originally appeared on Tech in Asia.