Given the tech boom that has occurred in recent years, there are a number of topics that have periodically taken over the collective imagination of Silicon Valley. Though “disruption” might be the most well known of these, the concept that likely has the most crossover potential for business at large is that of “data mining,” or “big data.” As so much business has moved completely online – from advertising revenues through social media to ordering home goods via Amazon – much of this business comes with a veritable mountain of untapped data on customer preferences and market patterns: in other words, data that can be used to improve business in the future. Unsurprisingly, then, data mining has become a burgeoning industry, as nearly every industry has sought to get a better grasp on these mounds of inscrutable (and potentially profitable) data.
For those interested in the field of data mining as a potential career, they will find themselves entering what is very much a bull market. Since the beginning of this decade, all predictions have pointed to continued growth in the data mining market. Some prognosticators have suggested that not only will growth occur, but that it will be quite massive: the big data market could grow by over 58% by 2017, evidence of a massive push to take advantage of this untapped resource of information. Likewise, the consulting firm McKinsey has estimated that more efficient data mining could be valued at $300 billion for the United States health care industry alone, whereas retailers benefiting from improved knowledge of their customers’ spending habits could see operating margin profits increase by close to 60%. Likewise, some of the most enthusiastic observers believe that the efficiency brought on by the use of big data in the form of more efficient consumer decisions could save customers $600 billion annually by 2020. From this vantage point, all signs point to an optimistic view towards the future of the data mining industry.
Of course, there are also realities of the data mining industry to suggest that while there will certainly be growth over the next decade, investors need not be overly optimistic about what data mining could mean for companies and consumers alike. The most obvious reason for caution is that there is still something of a skills gap: though the data exists to be mined, there is a finite number of mathematicians/statisticians who have been trained to properly analyze this information. Taking a closer look at the United States, in order to respond to all big data and analytics business needs, it may be that over 150,000 new analysts would need to be trained in data mining. While good news for recent graduates in statistics, this also means that growth in data mining might be limited by personnel needs.
Given the potential profit in this sector, though, there is certainly reason to be optimistic about the growth of data mining in the coming years. Growth in data mining will not simply benefit the companies who are taking advantage of these analytics to refine their business plans and better reach out to potential consumers. It could also mean customers themselves will see the effects of more efficient data analysis, as their weather, traffic, and social media updates become more and more personally tailored.