Big data is people

[C C]

https://aeon.co/essays/why-big-data-is-a...very-human

EXCERPT: We live in what is sometimes called the ‘petabyte era’, and this pronouncement has provoked much discussion of the sheer size of data stores being created, as well as their rapid growth. Claims circulate along the lines of: ‘Every day, we create 2.5 quintillion bytes of data – so much that 90 per cent of the data in the world today has been created in the last two years alone.’ This particular statistic comes from IBM’s website under the topic: ‘What is Big Data?’ but similar ones appear regularly in the popular media. The idea has impact. Among other things, it is used to initiate a conversation in which an IBM representative, via a pop-up entreaty, offers big-data services. Merely defining big data, it seems, generates more opportunities for big data.

And the process continues. Ever more urgently in the press, in business and in scholarly journals the question arises of what is unique about big data. Often the definitions are strangely circular. In 2013, a writer for the Columbia Journalism Review described big data as ‘a catchall label that describes the new way of understanding the world through the analysis of vast amounts of data’ – a statement that amounts to: big data is big… and it’s made of data. Others talk about its transformational properties. In Wired magazine, the tech evangelist Chris Anderson claimed the ‘end of theory’ had been reached. So much data now exists that it is unnecessary to build a hypothesis to test scientifically. The data can, if properly handled and analysed, ‘speak for themselves’. Many resort to definitions that stress the ‘three Vs’: a data set is ‘big data’ if it qualifies as huge in volume, high in velocity, and diverse in variety. The three Vs occasionally pick up a fourth, veracity, which can be interpreted in a number of ways. At the least, it evokes the striving to capture entire populations, which opens up new frontiers of possibility.

What is often forgotten, or temporarily put aside, in such excited discussions is how much of this newly created stuff is made of and out of personal data, the almost literal mining of subjectivity. In fact, the now common ‘three Vs’ were coined in 2001 by the industry analyst Doug Laney to describe key problems in data management, but they’ve become reinterpreted as the very definition of big data’s nearly infinite sense of applicability and precision.

When introducing the topic of big data in a class I teach at Harvard, I often mention the Charlton Heston movie Soylent Green, set in a sci-fi dystopian future of 2022, in which pollution, overpopulation and assisted suicide are the norm. Rations take the form of the eponymous soylent-green tablets, purportedly made of high-energy plankton, spewed from an assembly line and destined to feed the have-nots. Heston’s investigation inevitably reveals the foodstuff’s true ingredients, and such is the ubiquity of the film’s famous tagline marking his discovery that I don’t think spoiler alert applies: Soylent green is people!

Likewise, I like to argue, if in a different register: Big data is people....

[Ben the Donkey]

I'm a data analyst by trade. I know all about it.

This is why I'm heading to Tasmania. 
Also, why I've never had kids. Seriously. Because I know.

[elte]

I'm glad I never had kids too.  The future ain't what it used to be.

[Secular Sanity]

Big data is people....

And if you’re a wise fisherman, you cast out your net into a sea filled with little fish, and discover a large fish.

C C, is uncertainty a fundamental part of reality?

[Magical Realist]

https://aeon.co/essays/why-big-data-is-a...very-human

EXCERPT: We live in what is sometimes called the ‘petabyte era’, and this pronouncement has provoked much discussion of the sheer size of data stores being created, as well as their rapid growth. Claims circulate along the lines of: ‘Every day, we create 2.5 quintillion bytes of data – so much that 90 per cent of the data in the world today has been created in the last two years alone.’ This particular statistic comes from IBM’s website under the topic: ‘What is Big Data?’ but similar ones appear regularly in the popular media. The idea has impact. Among other things, it is used to initiate a conversation in which an IBM representative, via a pop-up entreaty, offers big-data services. Merely defining big data, it seems, generates more opportunities for big data.

And the process continues. Ever more urgently in the press, in business and in scholarly journals the question arises of what is unique about big data. Often the definitions are strangely circular. In 2013, a writer for the Columbia Journalism Review described big data as ‘a catchall label that describes the new way of understanding the world through the analysis of vast amounts of data’ – a statement that amounts to: big data is big… and it’s made of data. Others talk about its transformational properties. In Wired magazine, the tech evangelist Chris Anderson claimed the ‘end of theory’ had been reached. So much data now exists that it is unnecessary to build a hypothesis to test scientifically. The data can, if properly handled and analysed, ‘speak for themselves’. Many resort to definitions that stress the ‘three Vs’: a data set is ‘big data’ if it qualifies as huge in volume, high in velocity, and diverse in variety. The three Vs occasionally pick up a fourth, veracity, which can be interpreted in a number of ways. At the least, it evokes the striving to capture entire populations, which opens up new frontiers of possibility.

What is often forgotten, or temporarily put aside, in such excited discussions is how much of this newly created stuff is made of and out of personal data, the almost literal mining of subjectivity. In fact, the now common ‘three Vs’ were coined in 2001 by the industry analyst Doug Laney to describe key problems in data management, but they’ve become reinterpreted as the very definition of big data’s nearly infinite sense of applicability and precision.

When introducing the topic of big data in a class I teach at Harvard, I often mention the Charlton Heston movie Soylent Green, set in a sci-fi dystopian future of 2022, in which pollution, overpopulation and assisted suicide are the norm. Rations take the form of the eponymous soylent-green tablets, purportedly made of high-energy plankton, spewed from an assembly line and destined to feed the have-nots. Heston’s investigation inevitably reveals the foodstuff’s true ingredients, and such is the ubiquity of the film’s famous tagline marking his discovery that I don’t think spoiler alert applies: Soylent green is people!

Likewise, I like to argue, if in a different register: Big data is people....

I suspect, that just as it takes an infinite amount of information to specify any real number between 1 and 0, that it would take an infinite amount of information to fully specify any given point in space, not to mention any object or event existing there. Big data is everything. We are all data compressed down and compounded to the point of Big Bang explosiveness!

[C C]

C C, is uncertainty a fundamental part of reality?

Since there's no direct reference to QM here, I'm going to offer my opinion at a generic level (which can embrace a "fundamental" for the whole spectrum ranging from macro to micro scales).

A mindless world (or the vast majority of it which was such) by definition wouldn't have epistemological concerns or stances about itself. One might say it "just is", "happens", etc without awareness of and having conceptions / interests about itself. Lacking emergent beliefs and knowledge, it would entertain neither certainty nor uncertainty.

There are non-artificial patterns based in this or that physical medium (like EM waves) that humans can abstractly conceive of as "information". But those feral, passive representations aren't an example of the cosmos systematically trying to understand and anticipate its developments in terms of those patterns. ([*] footnote elaboration at bottom.)

Shifting to people and their claims of certainty about ___: The distinctions, classifications, orderings, rules and principles which we do make are least practical -- these "tools" often work enough to one extent or another. Scientists are good at predicting / tracking the motions of non-irregular bodies in space; and perhaps some general Earthly affairs which don't have to be specific or accurate about how particular objects and events will play out.

Plus, a handful of proposals we make about particulars or their situations can at least seem certain. "It is inevitable that I will die" is only questionable from the standpoint of the remote possibility of super-technological ETs intervening, cryogenics, (etc); and excluding the eventual demise of all stars / potential power sources in the distant future. "It is certain that the Moon will rise again" or seem to from _x_ perspective is expressing warranted confidence that grand catastrophes within the boundaries of interplanetary space aren't expected immediately.

But usually the more demand there is for an absence of unknowns and elimination of multiple possibilities regrading particulars and specific details, the more the requested accounts / answers are subject to being unsuccessful with that high standard. In the end, our "maps" (simulations, representations, principles, formulaic procedures treated as knowledge) lack complete details and can (at times) be inaccurate descriptions / interpretations of the territory with regard to what they are able to feature or output in terms of prediction.

- - - - - - - - -

[*] Accumulating "data" and processing it meaningfully accordingly requires just that: The filter of an operating system. That is, such a complex of functions is "filtering" against disorder and the usual "noise" and indifferent tendencies of nature at large. And carries "working preferences or pre-conditions" (somewhat figurative reference to parts of its mechanistic / relational structure) that have been selected for its operation by either the "stupid design" approach of evolution or the deliberate design of existing sapient agents. (If not for their common psychological context, the following might have served as analogous replacements: bias for "filter"; working prejudices for "working preferences").

[Secular Sanity]

[*]Accumulating "data" and processing it meaningfully accordingly requires just that: The filter of an operating system. That is, such a complex of functions is "filtering" against disorder and the usual "noise" and indifferent tendencies of nature at large. And carries "working preferences or pre-conditions" (somewhat figurative reference to parts of its mechanistic / relational structure) that have been selected for its operation by either the "stupid design" approach of evolution or the deliberate design of existing sapient agents. (If not for their common psychological context, the following might have served as analogous replacements: bias for "filter"; working prejudices for "working preferences").

Does "big data" shape people or do people shape "big data"?

Prophets predict and big data anticipates.

What are some ways of reducing confirmation biases and maintaining autonomy?

The influence of propinquity is strong.  Can you opt out of clustering and gain more exposure to the unfamiliar?

[Ben the Donkey]

Does "big data" shape people or do people shape "big data"?

A bit of both. While it's true that data is analysed in hopes of finding a target audience (in the case of advertising), its also true that that target audience can be "shaped" to the desires of the pitcher.
We're basically told what we like, and the majority like it without really knowing why. 
Body image is a good example.

[Secular Sanity]

I like 'Ben the Donkey'.  It suits you well.

What I mean by propinquity, Ben, is the "people you may know" effect. The mere-exposure effect generates higher click rates.  What’s the first thing you do when you walk into a crowded room?  I look for a familiar face. 

You’re not only good at anticipating behavior, you’re also good at seeing the bigger picture.  I’m nobody, but if I was somebody, I’d utilize your creative side.  More autonomy wouldn’t kill you, and as the poem suggests, it’s preferable to fame.

I'm Nobody! Who are you?
Are you - Nobody - too?
Then there's a pair of us!
Dont tell! they'd advertise - you know!

How dreary - to be - Somebody!
How public - like a Frog -
To tell one's name - the livelong June -
To an admiring Bog!

I don’t think that you have to be a data scientist, but I do think that applying the scientific method is extremely useful.  Science is as much about creativity as it is facts.

Hang on... am I confusing my style?

It’s none of my business but I am curious.  Your style may be improving, but has your midlife crisis yielded any useful personal development paths?

http://badhessian.org/2013/10/big-data-a...ic-method/

https://hbr.org/2012/10/data-scientist-t...t-century/

[Ben the Donkey]

Useful development paths? Yes and no.  I'm about to do a runner again now, but it's nothing unusual. There was a time before my last job I only ever lasted two years in the same position at a time before moving on (usually interstate). This last one has been nearly eight years, which is something of a record for me. I'm a ghost, most of the time.
That's actually a very tough question to answer. I have no idea, really. What's "useful" anyway?

As far as the propinquity goes, yes I do the same thing - but I'd imagine less than other folks. To be honest, I'll usually have a couple of drinks, then wander off and hang over a balcony or something and wait to see who comes to me. If no one, or uninteresting people, I'll usually go home after a couple of hours. 
To be even more honest, most by far tend to fall into the 'uninteresting" category.

I suspect my "development" is fairly retarded by normal social standards, if that means anything. But I'd say the lack of social interest (or skills) has led me to be more able to see things sans emotion. Gives me a less-clouded outlook on life and events.