Perhaps it is simply that I am like the other 3 million people or it is my exposure to Southern heritage (being based in Atlanta), but I am totally addicted to Discovery Channel’s Moonshiners.
The docudrama follows the activities of a group of Appalachian mountain men who produce and sell illegal moonshine. What is most intriguing to me is the ingenuity of these men as they develop their illegal stills from cobbled together components while hiding their enterprise from the “law.”
The other night it struck me how IgnitionOne’s Digital Marketing Suite (DMS) technology stack and “Big Data” has connections to my favorite show.
At first blush, the linkage is pretty disjointed. But closer examination shows them to be very similar to each other.
A still is nothing more than an extraction engine designed to efficiently create alcohol (C2H5OH) from natural ingredients. Along the way, these natural ingredients impart desirable flavors and properties that are valued by its consumers.
Likewise, a “Big Data” engine like the DMS is an extraction engine designed to efficiently create marketing “white lightning” from digital data. Along the way through all the transformations and manipulation, the basic ingredients (data) is refined and distilled into products that extract the maximum kick while still maintaining its consumption value. And white lightning for marketers won’t give you a buzz (or a hangover) but it will deliver better results and efficiency.
The Mash: GIGO (Garbage In, Garbage Out)
Beyond just the basic purpose of a distillation engine, the individual pieces of each machine also have direct relationships. While each moonshiner has his own personal recipes, the basic inputs to a still are grains/sugars, yeast and water, which are combined into a “mash”. Recipes vary during the year based on the price/availability of raw ingredients. The result is a balance between the strength of the mash (how much alcohol can be concentrated into the final mixture) and the potableness (suitability for drinking). A mash that is too “hot” (not able to be consumed, due to the burning feeling when consuming) is as undesirable as a mash that is too “cold” (no alcohol but is more potable). Balancing these competing properties is the art in the process that has been handed down through generations of shiners.
Likewise, within any Big Data marketing system, a multitude of different inputs are available to gain marketing insights and knowledge. Referral data, search engine data, onsite tracking, social data, CRM, demographics and a host of other data sources all contribute to the digital marketer’s mash. Some of these sources are easy to obtain and some are more difficult, but ultimately to be most useful to the final product, they must be combined into a single mash. Similarly, some data is very hot (contains great marketing insights and knowledge) but is very difficult to digest, while other data is too cold (and while consumable is ultimately not useful in action).
Additionally there is the question of scale. In a still, it takes a lot of mash to make even a little product. The same is true in Big Data Stills where low information value data may require large amounts of processing to enable the accumulation of enough insights. Extracting the maximum amount of information at the minimum cost enters into all Big Data Stills as well.
Mash Pot: Adding Heat to Create Pressure
During the fermentation process, the mash is transferred to the mash pot. Here it is exposed to heat which releases the alcohol from the mash in the form of steam. Pressure in the mash pot builds and is captured by the steam cap which is then used to propel the alcohol heavy steam through the rest of the distillation process. An improperly designed mash pot will leak steam and therefore not be able to propel the steam through the additional processing. Another more dangerous situation arises when the pressure is too great and the still explodes.
Big Data Stills have the same problems. Improperly designed data stores are unable to bring to the surface marketing insights in a timely fashion or may leak valuable information. Or Big Data stores will simply blow up due to scalability as the pressure and volume of the data mash and boiled up marketing insights are too great for the system to handle. Big Data marketing systems need to be carefully designed, flexible enough handle multiple set of inputs and built to scale.
There is another dangerous phenomenon that occurs in Big Data Stills as well as moonshine stills. During the fermentation process, a certain portion of the mash will be converted into methanol (C3OH). Methanol is a highly dangerous substance that has been known to cause blindness and even death. As a still heats up the mash releases this methanol (methanol boils at a lower temperature that alcohol), shiners call this the foreshot and discard this dangerous byproduct. This is very similar to Big Data Stills where the initial set of analysis (foreshot) from the data can be very disorienting and can blind marketers to the insights behind the data. Likewise, many times marketers overfit and/or over analyze the data mash which can lead to marketing paralysis or dangerous results.
From Steam to Hooch: The Rest of the Journey
Once the alcohol heavy steam has left the mash pot, it flows through a series of pipes and tubes. The first stop is the thump keg, which filters the impurities from the steam, which if allowed to continue through the process, will clog up the increasing smaller diameter piping that makes up the rest of the still. Filtering the haze of analysis to ensure that it is accurate, reliable and true is also necessary in a Big Data Still. Outliers, site outages, and data delays need to be removed so that they do not lead to erroneous results and/or excess pressure in the system. A common mistake among Big Data systems is ignoring the need to control for these impurities, and while it would appear that working with digital data is surprisingly clean, these cleansing functions are surprisingly complex.
As the now purified steam leaves the thump keg, it enters the worm box. Here cool water is circulated over coiled pipes known as the worm. A shiner’s worm is one of his most prized possessions (they have been known to be passed down in some West Virginia wills). Carefully crafted from hand selected copper, a superior worm imparts warmth to the final product by slowly condensing the vapor (and the flavors of the underlying mash) across the entire length of the pipe, thereby allowing the flavors and the raw alcohol to infuse into each other. The cold water condenses the alcohol vapor within the pipes which then emerges as final product for distribution. Without the constant supply of cold water, the alcohol vapor will simply shoot out of the end of the pipe without condensing, lost forever to the wind.
Within a Big Data marketing system, the worm is the interface system which exposes the marketing vapor to the cold water of analytics and experience. Some digital marketing systems are great at producing huge amounts of processed data (and vapor) but little of it is in form that is true marketing white lightning. It is only by the constant application of judgment, experience and analytic brainpower which is constantly changing and adapting within the machine itself that the final product is effectively delivered.
Result: White Lightning
By no means am I trying to romanticize the illegal production of moonshine; instead I am trying to focus the discussion around Big Data marketing systems to their core competency.
With all the talk about Big Data marketing systems, it is important to recognize that, in the end, they are nothing more than distillation machines. There is no need for huge data stores if the information content of the data is unusable, blinding or dangerous. Additionally, your Big Data marketing system needs to be designed and run by deep institutional knowledge about how the mechanics of the system are implemented, how to mix the data ingredients, and finally how to effectively present and deliver the actionable insights and marketing knowledge that the data presents. While I would like to think that we are slightly more refined, professional (and hopefully better looking) than our mountain men brothers, our goal is the same delivering the best white lightning for our clients. Cheers!