"Here’s a list of what I find are the most interesting trends in analytics in 2015.
Why the italics? because most of what will happen this year can be summarized with a single word: more. Yes, there will be more data, more mobile analytics, more cloud analytics, more data discovery, more visualization, etc.—but these are the trends that I personally will be paying closer attention to over the year:
Arthur C. Clark famously said that “Any sufficiently advanced technology is indistinguishable from magic.” The analytics industry has recently seen big advances in technology, but it hasn’t yet turned into magic—tools and interfaces that “just work.”
Today, people are required to shepherd every step of the analytics process, determining what data is available, how it should be joined, how it should stored, and how it should be analyzed and visualized.
But the new power of advanced analytics and machine learning is now being applied to the process of analytics itself—so that more of the process can be automated.
We should be able to point our tools at the data, and let the algorithms figure out it how it should be joined and cleansed, propose complementary data, and optimize how it should be stored (e.g. between cost-effective “cold” storage and operations-optimized “hot” storage). We should be able to let our tools identify outliers, find statistically-valid correlations, and propose the right types of visualization.
Today, companies like SAP offer Smart Data Access to connect data seamlessly between Hadoop/Spark and new in-memory analytics systems. And the SAP Lumira data discovery tool uses advanced statistics to automatically generate Related Visualizations based on the data sets being viewed. 2015 will see more advanced automation based on these capabilities.
Datafication is what happens when technology reveals previously invisible processes—which can then be tracked and optimized. This isn’t a new trend, but it’s gathering speed as real-time operational analytics systems become available and the price of gathering data continues to plummet.
Connected devices are the highlight of this year’s CES conference. Beyond the dozens of fitness tracking devices already available, there are now chips that stop you slouching, and sensor-enabled soccer balls, basketballs and tennis rackets to help you improve your game. Sensor tags can even help you find your keys.
The key insight is that even simple data can lead to big insights. For example, sensor-equipped carpets promise to help seniors stay independent longer—not because the sensors themselves are complex, but because powerful pattern-detection algorithms can learn a resident’s normal gait and sound an alert if it starts to deteriorate. And who would have thought that fitness devices could locate the epicenter of an earthquake?!
And of course all this applies to commercial uses. Shoppers can be tracked with beacons, Inventory can be tracked via drones. You can spot process bottlenecks, optimize beer sales, and track real-time purchases.
Here’s an instant business opportunity for 2015: find a process that is poorly tracked. Install simple sensors along the process and feed the collected real-time data to the cloud. Then use sophisticated analytics to feed actionable insights back to business people using mobile interfaces. For bonus points, add complementary third-party data sets, offer industry benchmarking, and encourage community best-practice sharing.
The layer-cake best-practice model of analytics (operational systems and external data feeding data marts and a data warehouse, with BI tools as the cherry on the top) is rapidly becoming obsolete.
It’s being replaced by a new, multi-polar model where data is collected and analyzed in multiple places, according to the type of data and analysis required:
- New HTAP systems (traditional operational data and real-time analytics)
- Traditional data warehouses (finance, budgets, corporate KPIs, etc.)
- Hadoop/Spark (sensor and polystructured data, long-term storage and analysis)
- Standalone BI systems (personal and departmental analytics, including spreadsheets)
There are clear overlaps with each of these systems, and they will converge over time, but each is a powerful hub that is not going to be replaced by the others any time soon.
In 2015 we will see the development of more best-practice guidance for how to get the most out of this pragmatic—but complex—collection of analysis hubs. This will involve both regular data feeds between poles and federated analysis to provide a connected view across the enterprise (including, hopefully, some more “magic”—see point 1).
Questions that enterprise architects will have to answer for different uses include:
- Where will this data arrive first?
- Will it need to be move to another pole as part of an analysis? When and why?
- Where and when will the data be modeled, and by whom?
- What are the different levels of access that will be given to different users, with what governance?
In their recent book, the Second Machine Age, authors Erik Brynjolfsson and Andrew McAfee argue that we are now in the “second half of the chessboard” when it comes to computer technology. The exponential trend means the increases in data processing power this year will be the equivalent of decades of progress in the past.
Nowhere is this clearer than in the area of analytics, where the biggest problem is increasingly that organizations just don’t know which of the myriad business opportunities to implement first.
2015 will be a wonderful year for analytics, just like it has been for the last quarter-century—as long as we remember that great power brings great responsibility, and that we must also strive to adapt our information culture and processes." by