in December 2014 I wrote a post titled Finding the elusive great wisdom in big data. We’ve made great strides since then in storing, processing and overall handling large volumes of data in a digital context, but humans still have issues when it comes to reliably finding relevant nuggets that can improve business results. I’m revisiting the idea to see how far we’ve come and how far we still have to go.
Baseball is one of the best examples of how data can impact a business. the book and the movie “Moneyball” showed how Oakland Athletics GM Billy Beane changed the sport by introducing advanced statistical analytics rather than simply relying on human scout reports. Today, baseball is governed by analysts as well as seasoned baseball professionals — but could there be too much data?
Alex Spier writing for Boston Globe’s Sunday baseball notes column recently pointed out that Boston Red Sox coach Alex Cora will have 11 coaches on the team this year for 26 players. Compare that to Terry Francona in 2011, who scored six over 25, and you can see that number has almost doubled.
Spier attributes this in part to the growing amount of data teams are collecting, which requires more people to observe, interpret and implement a plan to leverage. As Spier wrote: “The result? Several teams now feature three hitting coaches, and teams continue to grow in an effort to distill mountains of information into digestible form for the 26 players on a roster.”
“Startups that analyze and extract insights from fast data have garnered extremely high interest from public and private market investors.” Deepak Jeevankumer
Baseball is like a laboratory for advanced statistical analysis, and companies can learn a lot from watching how the sport handles expanding datasets.
Businesses dump data into data lakes, tinker with machine learning models and use that information to make decisions, but they still face fundamental everyday decisions that involve their ever-expanding mountain of information. Sure, machines and software can help solve this – in fact, they’re getting better at it – but anyone who’s dealt with misdirected advertising or email marketing knows there’s a lot of work to be done.
Deepak Jeevankumer, managing director of Dell Technologies Capital, says the difference at play now from 2014 is that companies are now focused on getting insights to the people who need them faster. “Big data has become less relevant than ‘quick data’. E-commerce buyers, streaming media consumers, gamers, stock/cryptoactive traders and enterprise marketers are demanding quick insights and quick knowledge,” Jeevankumer told me.
He believes the data needs to be analyzed ‘at its source while streaming and in real time after creating a query, and that startups that build solutions that allow companies (and baseball teams) to do this will be more successful long-term.