Big Data: Potential to Deliver ‘Perfect Personalization’ or Discrimination and Segregation

The “Big Data: Seizing Opportunities, Preserving Values” report from the 1st of May focuses on the impact of big data on the United States government, people, and inner workings. Big data is a technological phenomenon that gained momentum at the beginning of the “Internet of Things” epoch. Big data analytics is a promising tool for governments to catch criminals, monitor expenditures, allocate funds, and greatly advance our quality of life. It can plausibly merge the industrial and informational economies. With such promising applications of big data analytics, the question arises of whether our existing legal, ethical, and social norms are sufficient to protect privacy and other values. “Big data applications may be the driver of America’s economic future or a threat to cherished liberties.”

The authors likened big data analytics to finding a needle in a haystack (see also this report by McAfee, Inc.). “Big data analytics enable data scientists to amass lots of data, including unstructured data, and find anomalies or patterns.” The ease with which big data can help us identify these anomalies and patterns is increased to the point that finding a needle in a haystack is now both possible and practical. The identified problem with this analogy is that to find a needle, it first must be in a haystack. The haystack is a certain quantity of data on individuals who may have no connection to the anomaly.

“The fusion of many different kinds of data, processed in real time, has the power to deliver exactly the right message, product, or service to consumers before they even ask.” This is the idea of “perfect personalization”. A risk of this “perfect personalization” is that it leaves room for discrimination in pricing, services, and opportunities. It further risks assigning people to segregated enclaves of ideological or cultural “filter bubbles”.

In the end, this report is “about more than the capabilities of big data technologies. It is also about how big data may challenge fundamental American values and existing legal frameworks.” It looks at and suggests adjustments to the government’s role in assuring enduring values and evolving legal protection of consumers and citizens. To access the whole report without an HSDL login, click here; the fact sheet is available by clicking here

Article formerly posted at