Peering into Big Data’s crystal ball

Jun 12, 2013

Big Data body

Until now, it’s been a fanciful idea revolving around tarot cards and crystal balls. But thanks to the advent of Big Data, we may soon be able to predict the future.

As Long View’s Practice Director of Storage and Data Management, I am always keen on identifying and staying on top of industry trends, including those that include virtualization and converged infrastructure. Recently, I attended EMC World 2013, a partner conference held in Las Vegas in early May, along with seven other Long View representatives — and I was particularly intrigued by a keynote session I attended called Cloud, Big Data and Trust.

Thanks to a big industry push that we’ve seen over the past couple of years to virtualize storage, we’ve arrived in the era of Big Data — essentially, the capacity to collect enormous amounts of data, and then harvest value out of that data.

What does this mean in the real world? The possibilities, frankly, are endless. Big Data means using existing data patterns to anticipate events in the future. It could apply to matters of national security. It could mean increased safety in the auto industry, with an analysis of driving patterns. It could even mean anticipating consumer behavior — if there’s less RFID activity in the fridge, the water’s getting turned off, and the furnace is being turned down, it’s because the Smiths are probably going on vacation.

What does the arrival of Big Data say to me from a careers perspective? First of all, if you want to specialize in storage, you’ll need to be a virtualization expert as well. And secondly, if you’re looking at a career in data management, you’ll need to understand math — particularly analytics. In fact, because of Big Data, I can see analytics-based careers opening up in areas like finance, science, and security, just to name a few. By identifying the correlation between certain behaviors — whether it’s people, genomes, or computer viruses — IT professionals will be able to predict and possibly avoid certain events.

Still think this is the world of science fiction? Think again. Because Big Data is where the smart money’s going. We heard at EMC World 2013 that 90 per cent of Fortune 500 companies are using some form of analytics today. That’s a huge number. And it means that companies are leveraging big data to transform their business.

None of this would be happening, of course, without virtualization of storage, which gives us the capacity to manage massive amounts of data. Those Fortune 500 companies I mentioned are tracking 100 billion real-time events every month. And one engine on a passenger plane, on average, racks up 30 terabytes of telemetry data during a transatlantic flight.

From where I’m sitting, Big Data has enormous applications . . . and presents big opportunities for IT professionals.