How big is Big Data?
Walmart handles more than 1 million customer transactions per hour, over 400 million tweets are sent per day, over 6 billion cell phone calls per day in the U.S., 10,000 payment card transactions made every second worldwide, etc., etc. Just imagine the hard drive and computing power the NSA needs to record all electronic communications across the entire world.
Big data is everywhere. What does it take to manage it?
Data collection and analysis are a part of every industry and business function these days. McKinsey & Company estimates that by 2009 already “nearly all sectors in the US economy had at least an average of 200 terabytes of stored data (twice the size of US retailer Wal-Mart’s data warehouse in 1999) per company with more than 1,000 employees.” How quickly and accurately that data can be digested and assimilated makes the difference in how competitive firms can be.
Small and large companies alike are harnessing big data via outsourcing their business logistics and data management to cloud computing vendors like the quasi-utility Amazon. The technology infrastructure platform these companies offer include storage services, database services, messaging services and payment services. Even Netflix is moving to outsource its data center. The company plans on using Amazon’s cloud for internal applications (such as its streaming services) and software-as-a-service providers for other business systems.
The Talent Needed
In order to take the competitive advantage that big data promises, organizations need to recruit the technical staff who knows how to manage it, analyze it, and understand it.
The trend of moving towards better use of point of sale data, supply chain management, customer engagement and relationship data, the growing need for big data professionals will continue. The job titles for roles that are in demand include: data scientist, data analyst, business intelligence analyst, business analytics, cloud computing.
Over the next 5 years, we may face a talent shortage in the U.S. of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions, according to McKinsey.