Big Data is watching you, and remembering everything you do

WE are constantly exposed in public. Yet most of our actions will fade into obscurity. Do you, for example, remember the faces of strangers who stood in line with you the last time you bought medicine at a drugstore? Probably not. Thanks to limited memory and norms against staring, they probably don't remember yours either.

This is what it means to be obscure. And our failure to collectively value this idea shows where we've gone wrong in the debates over data and surveillance.

Lawmakers and industry leaders are missing the big picture. They are stuck on traditional concepts like "transparency", "consent" and "secrecy", which leads to proposals that reinforce broken mechanisms like consenting to unreadable terms of service. They are operating under the dangerous illusion that there's a clear distinction between what's public and what's private. Most people probably intuitively know that their most deeply held secrets are private while the things about them that are commonly known or widely broadcast are not. But what about information about our everyday actions that is shared with some but not all?

Obscurity bridges this privacy gap with the idea that the parts of our lives that are hard or unlikely to be found or understood are relatively safe. It is a combination of the privacy you have in public and the privacy you have in groups. Obscurity is a barrier that can shield you from government, corporate and social snoops. And until lawmakers, corporate leaders and citizens embrace obscurity and move to protect it, your freedom and opportunities to flourish will be in jeopardy.

The concept was first meaningfully articulated in a landmark 1989 Supreme Court decision, Department of Justice vs Reporters Committee for Freedom of the Press, which recognised a privacy interest in the "practical obscurity" of information that was technically available to the public but could be found only by spending an unrealistic amount of time and effort. After the concept was introduced, most lawmakers, judges and the tech industry promptly forgot about it. This has dulled the potential of obscurity as a rallying point for change and leaves society struggling to protect something in dire need of defining and defending.

Understanding obscurity means paying attention to how space, time and people's cognitive limitations make it difficult for others to surveil us or find out things about us.

Consider space. The further away people are, the harder it is to see with the naked eye who they are and where they're going. Usually, we are visible only to those nearby. But cellphone location data can reveal that same highly personal information to anyone with means and motivation. We should be disturbed that apps often collect location data simply because they can and that T-Mobile, Sprint and AT&T reportedly have illegally sold customers' 911 location data to third parties. And we should be pleased that the Supreme Court ruled last year in Carpenter vs United States that, under the Fourth Amendment, cellphone records can't be seized without a warrant.

Consider time. Memories fade as time passes. But the Internet has impeccable recall. That's why we should applaud the senators who have proposed a "no-follow" bipartisan update to the Children's Online Privacy Protection Act that would give parents and children an "eraser button" to remove a child's data from platforms like Google and Facebook.

It's also why we should be sceptical of Mark Zuckerberg's announcement that Facebook will become a more "privacy-focused platform" that emphasises features like ephemeral, expiring information. Truly ephemeral communications are certainly obscure, but it's unclear what information Facebook will scrub on the front end (for users) and the back end (from the company). The company's history suggests we shouldn't let our guard down.

Obscurity is also a better way to understand Europe's "right to be forgotten" law, which can't make anyone forget anything, but can make the remnants of your awkward youth harder to find online.

Finally, consider cognitive limitations. Facial recognition technology poses an immense danger to society because it can be used to overcome biological constraints on how many individuals anyone can recognise in real time. If its use continues to grow and the right regulations aren't instituted, we might lose the ability to go out in public without being recognised by the police, our neighbours and corporations.

Creating strong regulations for the technology is going to be an uphill battle, especially because it's already become widespread, being deployed at airports to make boarding easier and adopted by schools to increase safety. It is even being used at summer camps so parents can automatically receive photos in which their children appear.

Threats to our obscurity are growing because technology is making our personal information easy and cheap to aggregate, archive and interpret - with substantial growth in predictive analytics, too. To see what we mean, just look yourself up on the website MyLife and marvel at how much information has been cobbled together from different moments in your life for anyone to see at the click of a button. Even speaking in hushed tones to a friend at a crowded cafe might not be enough to protect your obscurity if cameras are someday equipped with lip-reading artificial intelligence software.

Obscurity is vital to our well-being for several reasons. It gives us breathing room to go about our daily routines with little fear of being judged, sent unwanted ads, gossiped about or needlessly shamed.

Obscurity makes meaningful and intimate relationships possible, ones that offer solidarity, loyalty and love. It allows us to choose with whom we want to share different kinds of information. It protects us from having everyone know the different roles we play in the different parts of our lives. We need to be able to play one role with our co-workers while revealing other parts of ourselves with friends and family. Indeed, obscurity is one reason we feel safe bonding with others over our shared vulnerabilities, our mutual hopes, dreams and fears.

Obscurity enables us to develop and grow as individuals. It provides us collectively with spaces to explore new and controversial possibilities, to transgress taboos and ignore arbitrary rules. You might have thought you could skip church without anyone knowing, but churches are being marketed facial recognition systems that will make sure your absences are duly noted. It won't stop there.

Obscurity protects us from being pressured to be conventional. This buffer from a ubiquitous permanent record is especially important for the exploratory phase of youth. To develop as humans, people must be free to try things they might later regret. This is how we become better people. Without obscurity, we wouldn't have the freedom to take risks, fail and dust ourselves off. We'd be stymied by the fear of being deemed failures by a system that never forgets.

Finally, obscurity is crucial to democracy. Obscurity fosters civic participation and gives us the confidence to attend political protests and engage in political speech online without worrying about ending up on a government watch list. It's why the American Civil Liberties Union worries about the government monitoring social media.

Things can be private even if others can see them. And there is so much of our lives that industry and governments have yet to find. But in our status-obsessed culture, it can be hard to appreciate that the opposite of obscurity isn't fame, but chillingly oppressive fear. NYTIMES