Every day, we create 2.5 quintillion bytes of data -- so much that 90 percent of the data in the world today has been created in the last two years alone.
90% of the #data in the world today has been created in the last two years. #bigdata
TWEET THIS
Those eye-popping figures were tallied by IBM … in 2012. Three years’ proliferation of sensors, mobile devices and Internet-enabled products has only multiplied the data collected about us on a given day. The challenge now, say experts at Bentley and elsewhere, is extracting value from that tsunami of information.
“Whether companies are large or small, established or brand new, they are all collecting data. The difference is that the successful ones are already leveraging it,” says Christopher Lynch MBA ’91, who works with young companies as a partner at venture capital firm Accomplice. He was the keynote speaker at Bentley’s 2014 research colloquium on big data. “Data analytics will be the lifeblood of all companies and the price of admission into the 21st century and beyond.”
A Look Back
It’s worth noting how Lynch and the rest of us landed in this brave new world. David Oury, lecturer in mathematical sciences, describes how large computers and computing centers were once up to the task of storing, processing and analyzing the available data. Then came the rise of smartphones, websites and other 24/7 conduits of information.
“The old tools weren’t set up to deal with that much data and deal with it quickly enough,” says Oury. “So you saw new tools being created to work with data in different formats … tools that people needed to manipulate the data.”
Specialties were born. “Data mining and machine learning came out of theoretical computer science, artificial intelligence, statistics and information systems. You create models of the data, more complicated descriptions of the data,” Oury explains. “They were techniques used only by geeks and nerds and academics.”
Over time, people outside the pocket-protector demographic came to appreciate data mining. “This theoretical thing, which we had not considered useful, suddenly seems to be very useful. We can gain insights about our data, how it works, how it’s tied together -- and even make decisions based on these observations.”
Tool Time
With 27 years’ experience in data analytics, Professor of Mathematical Sciences Sam Woolford sees a decided turn toward greater use of data in the last decade.
“Businesses have always sought to apply data to their strategies. What’s changed are the tools for doing that,” he says of data-capturing systems that can communicate directly with each other. “Analytics are easier to run now than 10 years ago, when all the data was in separate systems.”
Directing the Center for Quantitative Analysis at Bentley gives Woolford a front-row seat to the changes underway. He and students in the Master's in Business Analytics program routinely work with companies such as CVS Pharmacy, Gorton’s and National Grid. Their data-driven projects have focused on improving operations, learning more about customers, and exploring stakeholders’ interactions with the business.
“Improved technology has broken down some of the barriers,” he says. “And greater awareness of the value that can be derived from analytics is inspiring greater cooperation among different departments within an organization to share data.”
Woolford’s colleague in the Mathematical Sciences Department, Mingfei Li, pays close attention to the companies that tap the Center.
“We can learn what companies need in analytics professionals … what kind of skills, experience and training,” says Li, who directs the Business Analytics program. “Industry feedback is so important.”
Seeing Patterns
Credit card companies were among the first to use big data analytics for identifying patterns. They were on the lookout for unusual purchasing behavior, which often signals a card’s unauthorized use. Businesses across the spectrum are learning to apply the same kind of algorithm to other organizational needs.
Will Markow ’12 works for one such company, Burning Glass Technologies, which combs through job listings and other labor-related data to find meaningful patterns. The work includes identifying skill sets and functional domains in high demand by employers, as well as positions that are proving the hardest to fill.
“A mismatch between job candidates’ training and experience and employers’ needs has created serious gaps in the fastest-emerging sectors,” reports Markow, a senior analyst with the company. “One of the key sectors with explosive growth is data analytics. But, the demand for skills has spilled over into other functional domains, such as marketing managers who are required to learn SQL.”
Learn about this new job trend: hybrid jobs.
Context is King
Companies today need a triad of “good data, advanced analytics and domain expertise,” according to thought leaders such as Dean Abbott, co-founder and chief data scientist at SmarterHQ and author of Applied Predictive Analytics: Principles and Techniques for the Professional Data Analyst.
In other words, algorithms don’t work in a vacuum. You need insight from the people “who know the customers about which metrics are significant,” Abbott says in an interviw for AllAnalytics.com. Without deep domain expertise, “you don’t know what you’re looking for.”
The principle applies across industries -- as Zebra Technologies learned in partnering with the National Football League. Jill Stelfox, Zebra vice president and general manager, location solutions, reflected on the issue in CIO magazine.
“The analytics from manufacturing weren’t the same as the analytics from football,” she says of developing the NFL’s Next Gen Stats platform.
“We could see correlations in the data that seemed important … and then they weren’t. We had to bring in people that had the football expertise who could say, ‘Look, this is why it matters.’”
Meeting of the Minds
The rush of organizations to embrace data analytics raises some flags for Bentley’s Heikki Topi.
“Data is meaningless unless it is used to create value for the organization and society as a whole,” says the longtime professor of computer information systems. “A big question right now is, how can big data be harnessed to create value and do good?”
Bentley’s work in this area is global, Topi notes. M. Lynne Markus, the John W. Poduska Sr. Professor of Information and Process Management, led an international effort to promote rigorous academic research on implications of big data. The research colloquium that Topi spearheaded last fall brought together fellow academics such as Sam Woolford and business leaders such as Chris Lynch to explore the relevant issues.
“We looked at applications and implications of big data -- and what it all means for individuals and society,” Topi reports. “We always have to ask: What is possible from a technical perspective and what is meaningful from a business perspective.”
4 Tips for Skill Building
Want to get your own bearings in big data? Here’s advice from Bentley faculty members Mingfei Li, David Oury and Heikki Topi, as well as other sources.
- Get Your Read On
Check out these books for a crash course on data analytics:
- Data Science for Business: What you need to know about data mining and data-analytic thinking (Foster Provost)
- A Practitioner’s Guide to Business Analytics: Using Data Analysis Tools to Improve Your Organization’s Decision Making and Strategy (Randy Bartlett)
- Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die (Eric Siegel)
- Competing on Analytics: The New Science of Winning (Thomas H. Davenport and Jeanne Harris)
- Learn the Language
SQL has become the de facto standard language for retrieving and managing data, says Topi. “Learning it is useful even for those who focus on the latest big data technologies.” Look for free courses on SQL, Python, Java and more on Codecademy (codecademy.com).
Find an analytics program (R, SAS, Tableau, RapidMiner) that you can run on your computer.
Many of these offer free trials, says Oury, who suggests doing a tutorial and producing a visual output such as charts and graphs.
This article on 21 Essential Data Visualization Tools from KDnuggets reviews the pluses and minuses of tools such as Google Charts, Leaflet, Visual.ly and ZingChart.
- Join a Community
Organizations such as the American Statistical Association (ASA) and the Institute for Operations Research and the Management Sciences (INFORMS) have comprehensive websites with resources for education, career development, social networking and events.
Online communities include KDnuggets (kdnuggets.com), Data Science Central (datasciencecentral.com) and Kaggle (kaggle.com).
Oury recommends meetups (meetup.com) on analytics, data science and big data.
- Listen to Experts
Presentations by academic and corporate leaders already versed in big data will provide insight on using analytics to solve practical problems, says Li. Her top picks are the annual meeting of INFORMS, the American Statistical Association’s Conference on Statistical Practice, and the Open Data Science Conference.