Dark Social: Dimming the Lights on Web Traffic

Kathryn Kure of Data Myna

We are particularly good at lying, cheating and deceiving – even to ourselves. One of the most singular results from brain research indicates that, in the absence of other information, we tend to fill in the blanks by concocting coherent stories that we can believe. In split-brain studies, for instance, when a man's left hemisphere saw a chicken with a claw and his right a shovel and a snow scene, he chose the matches correctly, but then generated a story as to why he chose the shovel, claiming it was to clean out the chicken shed.

It is for this reason that eye-witness accounts are notoriously prone to error since with the little information you have, you tend to start cobbling together a believable scenario which can then be picked apart on the stand on the basis of too little evidence for the inferences made.

The more we study the brain, the more we realise we are dealing with a cacophony of voices, a multitude of processes occurring simultaneously, which we handle by assigning the left-hemisphere the task of creating a coherent story from all this information.

Now, much of marketing research focuses on attitudes, which in turn, from a marketing research perspective, are often defined as a mixture of information, feeling, and intended behaviour, and it is widely accepted that attitudes are precursors to behaviour; if you like a brand, you are more likely to chose it – at the same time, you have to be aware that participants will sometimes generate interesting reasons for their feelings, and then go on and make completely different decisions.

But this is why anyone fascinated by consumers find what people actually do (i.e., consumer behaviour itself) always to be of particular interest, and it is for this reason that the large data-sets created, in particular, by “always on” mobile apps, offer such promise; these data-sets potentially are able to relate to us, at scale and with great detail, consumer attitudes within very precise and specific contexts (both geographic and, increasingly, personal). However, this cornucopia of data is, in very large part, predicated upon the ability of an app, when downloaded to your personal phone, to go on a major content grab and upload a veritable treasure-trove of personal data, and this is very much seen as part of the deal since, “If you are not paying for the app, you’re the product”.

Reasonable Price by Manu Cornet 

At present, you don’t have the ability to fine-tune what the app can take, or not; either you download the app, and grant it permissions which it decides upon, or you don’t. There is currently not much middle ground on offer, once you click on “Accept” , apps can, and often do, upload all your images to their cloud (which some actresses have recently discovered may not be deleted from the cloud even though they have been deleted from the device itself) and they also often start initiating other services, such as ‘Fine Location’ tracking (which tracks you via cell-phone towers even if you’ve switched off your GPS), all the better with which to serve you ads, my dear.

But with increased surveillance comes increased awareness of such surveillance, especially on the back of Snowden’s revelations, and we are increasingly discovering that our mobile phones are acutely vulnerable. Recently, for instance, security researcher GironSec analysed Uber's Android app and discovered it was sending a lot of sensitive personal information back to base which it didn’t have explicit permissions for, including “your call logs, what apps you've got installed, whether your phone is vulnerable to certain malware, whether your phone is rooted, and your SMS and MMS logs”.

At the same time, UK-based VisionMobile has found that most app developers make very little money from them, especially if they are offered ‘free’ to the public. Far from flattening competition, in a maturing internet market, start-ups are increasingly unable to compete against the established behemoths - which leads to monopoly, winner-take-all situations, and where the smaller companies, in order to compete in this market-place, attempt to grab more and more content in what one German commentator has referred to as Cowboy capitalism.

The most commonly downloaded apps include those for social networking sites, but even when they are able to understand their own consumers intimately through their collection of big data, marketers in turn are a) not privy to this information and b) even if they were, this in turn won't tell you everything, given the preponderance of Dark Social, a term coined by Alexis C. Madrigal which refers to web traffic that emanates from sources such as email, instant messages and forum posts that web analytics are not able to track. Radium One has recently quantified Dark Social and reports that 69% of all sharing activity takes place via Dark Social globally versus 23% via Facebook; furthermore, 32% of people who share content online will only share via Dark Social.

One of the key concerns in research is that of obtaining a sample which is representative of the population, and one must be concerned, particularly with that 32% of people who only share via Dark Social, as to whether or not this group of people is markedly different in any way from the group you can more easily track via social networks. For instance, it could be hypothesised that more tech-savvy or highly educated people privy to more confidential information are those who are accordingly more wary of privacy issues and hence are those sharing via Dark Social, and if this is the case, it would skew your results. Even within social network sites, there is a growth in sharing that is private, such as in Google Plus, which specifically enables sharing to private circles, which is not then amenable to analysis.

Furthermore, as our knowledge of surveillance, however seemingly benign increases, so we see that we adopt a series of strategies, from multiple email accounts linked to multiple identities, to surveillance detection to the increased adoption of software such as Tor, which defends you against traffic analysis. Tor arose from a project of the U.S. Naval Research Laboratory and was intended initially to protect government communications, given that, as the researchers point out, “traffic analysis is the backbone of communications intelligence, not cryptanalysis”. Tor seeks to ensure that people, governments and companies can’t watch your internet connections and learn what sites you visit, and it prevents the sites you visit from learning your physical location. How many people use Tor, is, of course an interesting question in and of itself.

As those who researched the introduction of New Coke onto the market are now well aware, we are masterful at deceiving even ourselves; it turns out we are becoming increasingly good at deceiving others too: the hive mind has sucked down the revelations given by Snowden and we exhibit remarkable rapidity in evolving defensive strategies, both technological and human, by avoiding software, evading detection or even hiding in plain sight with communication strategies which boyd and Marwick refer to as “social steganography”, best exemplified by a teenager whose Facebook post is: Always look on the bright side of life thereby telling her friends she’s depressed while her mother, not picking up on the Monty Python reference, commends her attitude.

In this brave new world where complex humans intersect with increasingly sophisticated technical products and services, it all gets very complicated very quickly, since the very object and subject of our scrutiny, continuously and deliberately set out to elude our gaze, whether we are governments or marketers.

It is a philosophical truism that the more trivial the question, the easier it is to answer, but when “knowledge is a lock and its key is the question”, the kinds of questions you need to ask to interrogate the seeming cornucopia of data and the possible lack of data or lack of complete and comprehensive data are key.

It’s not that you can't get no satisfaction, but you sure have to try and try and try.


A shortened version of this was first published in the Saturday Star on 6th December 2014, and has been republished here with permission:

Dark Social dims light on web traffic, Saturday Star, 6 December 2014