If you think about it, many boring things become interesting when taken to an extreme. Archaeology is a tedious science 99% of the time – unless you excavate across two continents and find thousands of Clovis spear points. Then you can tell a story about people who migrated across the Bering Straight, thereby influencing thousands of years of human development on a global scale. Compiling a dictionary is admittedly not exciting, unless you are creating the first edition of the Oxford English Dictionary by organizing a national campaign to gather as many definition submissions from the public as possible – on slips of paper that you then organize and sort. And practicing scales on the piano is infuriatingly boring at times (so I hear)… Until it leads to the ability to play Chopin.
Data mining is much the same. Students who sat through Stats 101 understand this. Hours of looking through probability tables, looking at probability distributions, and deciding whether or not to reject the null hypothesis… Nobody looks back on those days fondly (at least I hope so, because I sure don’t). But the courses that let computer programs do the math and let you create and test ideas actually become interesting and engaging. They are incredibly tough because they force you to think creatively as well as technically.
The transition from boring to substantial exists at the split second when numbers and data points become words, stories, and – later on – strategies. In retail, we can collect enough information about customers to direct entire marketing campaigns and merchandising efforts. There is a movement in a few companies to collect customers’ postal codes. This seems basic but can potentially give life to countless strategies. Which urban clusters shop most often? Which locations do they travel to? Why do they travel to a store that is farther than the one closest to them? During what hours of the day do they shop? What items do they buy, based on where they live? And this is only from the postal code. Many more types of information can be collected and analyzed.
At the risk of oversimplifying, the overall trick is knowing two things: what questions to ask (big-picture strategy) and how to answer them (being comfortable with pieces of technical information and with numbers). Being a business school graduate, I know many of my peers went on to land first jobs dealing exclusively with the latter. This is inevitable because we are entry-level employees, but it is also a great thing: knowing how to do analysis on minute details in any field is not a negligible skill – it is the foundation of intelligent leadership. Data mining, in any field, can be transformed into stories and strategies, but only at the hands of those who are not afraid to get into the lowest ranks of its details. Once this is embraced, there is no limit to the creativity that, however paradoxically, will follow.