The Now: What is Big Data?

Lesson 25: What is Big Data?

/en/thenow/what-are-startups/content/

What is big data?

How does UPS optimize its delivery routes? How do police departments predict where future crimes will occur? How does Target always know the best location to build a new store? These complex questions are answered through the power and utility of big data.

Big data is an incredibly large set of information that cannot be analyzed by typical data processing methods. It could include any type of information, from customer buying habits to delivery routes to search engine requests. Companies and organizations of all types and sizes collect and analyze big data in order to discover opportunities and pinpoint problems.

How is it used?

Let’s say a clothing retailer wants to find the most profitable location to open a new store. Using data pulled from millions of customer surveys, the retailer analyzes and compares each answer using computers and complex algorithms. Once the immense amount of data has been interpreted by experts, the retailer will be able to determine the best location for its new store.

UPS tracks its drivers’ delivery routes. Based on its continual analysis of each driver's data, UPS optimizes its routes and saves millions in fuel costs every year. Also, police departments around the United States study years' worth of crime data to predict where future crimes are most likely to take place. In areas where the Los Angeles Police Department used big data, burglaries went down by 33 percent, and violent crime decreased by 21 percent.

Even your online activity contributes to big data. Every time you visit a website, use social media, buy a product, or do almost anything online, your activity automatically generates a tiny piece of data. With millions of people also generating data, these pieces add up to a gargantuan amount of information that companies like Google and Facebook use to study and improve online activity.

Are there problems with relying on big data?

There are some flaws, yes. With so much information being collected, some experts believe there is too much data and not enough rational judgment of the results. People interpreting these incredible amounts of data must be careful not to reach the wrong conclusions, or it could lead to poor decision making.

As an example, in 2013 Google predicted that flu outbreaks would spike, using data taken from specific search engine requests around the United States. Instead, it vastly overestimated the number of flu cases that year due to a misunderstanding of the data. Also, numerous polling agencies used big data to predict the outcome of the 2016 U.S. presidential election, but this also led to numerous inaccurate predictions.

What about my privacy?

As big data collection becomes more widespread, keeping your information private will become more difficult. But in most cases, you agree to let some of your information be collected whenever you buy a product online, visit a website, or in the case of grocery stores sign up for a discount loyalty program. This is how companies flood you with advertising, whether it be online or in your physical mailbox.

One problem is that after your information is analyzed, we don’t know how these big data collectors are using what they've collected. Data collectors are not required to notify you when they sell your information to big data brokers, who also don't say what they're doing with it. They get away with this behavior because there are no major laws in place that regulate how data collectors handle your information.

Humans have developed this extraordinary ability to record and catalog an effectively limitless amount of data, but we still haven’t mastered the ability to interpret this data into meaningful action. Big data is likely here to stay, and it will only become more powerful and accurate as the years go by.