Tag: sales analytics
Connect With Us
Do Customer Buying Patterns Exist?
Customers have to make choices in order to buy configurable products. Do they make these choices at random, or are there patterns? When we look at the sales history for a configurable product, like a car or a computer, can we tell if customers have just been flipping coins and rolling dice? Or do their choices hang together and make sense? To answer this question, we would have to look at how they buy combinations of options. In the previous post, I took a pizza as a simple configurable product, and looked at how customers ordered pairs of toppings. Just by looking at the sales numbers we could detect that the selection of pineapple and Canadian bacon are not independent. Even if we had never heard of a Hawaiian Pizza, we could discover it in the data.
Even more information is hidden in combinations of three toppings at a time, or four toppings at a time. Any combination of toppings will have appeared on some of the pizzas that have been sold (or maybe none). The relative popularities of all the different combinations has a clear message: customers are not flipping coins. Some toppings naturally go together, and others do not. Pepperoni, broccoli, and anchovies is just unlikely. If a particular pizza restaurant has a few “house specials”, like the Meat Lovers and the Veggie Delight, we can see them in the data, even if we don’t know their names.
What is true of the pizza is also true of other configurable products: computers, trucks, tractors, lighting fixtures, industrial fans, and so on. All products that have variety. Customers make choices, but not by rolling dice. There are combinations that go together and combinations that do not. A pizza maker can juggle the preferences of his customers in his head. But when a product has 30 or more features, intuition is overwhelmed. The number of combinations explodes so fast that the unaided human mind can’t see the patterns. At this point, mathematical models and intense number crunching can reveal the patterns and let the product manager for a line of trucks be as confident as a pizza maker.
Do Customer Buying Patterns Exist?
Buying patterns are real, and they manifest themselves in how customers buy combinations of options. With the computing power we have available today we can detect and capture them. These patterns can then be used to design “house specials”, forecast future sales, and guide customers to what we want to sell them.
So, who else is talking about customer buying patterns?
Intel Talks about Changing technology buying patterns
Customer Buying Patterns have Changed. What’s Your Plan?
An entire report that summarizes the results of a consumer usage and purchasing pattern survey conducted in March of 2007. The survey was conducted with In-Stat’s Technology Adoption Panel (TAP) — a dynamic, online panel of more than 19,000 technology users and decision makers. Over 1,400 technology users responded to this focused survey.
Findings in this report include consumers’ time spent on PCs, when they last purchased a personal-use PC, the PC’s features/form factor/usage, the desired features of future PC purchases, changes in usage patterns, and consumers’ thoughts about new technologies.
The changing patterns include -
- When consumers are likely to make their next PC purchase.
- The features consumers state they want
- The features consumers state they really want, based on changes in their usage/buying patterns.
- How consumers view new technologies
However – buying patterns are constantly changing. As social networking grows, we are watching new markets emerge every day. There is gold for companies who can continually detect these patterns and offer the right products and feature mix.
Customer Buying Patterns – What you can learn From Pizza Sales
There are 1,140 ways of ordering a pizza with 3-toppings, if the pizza offers 20 choices
First order take rates tell us about the relative popularity of different options. For example, consider a small set of possible pizza toppings.
|
Topping |
Take Rate |
|
Pepperoni |
40% |
|
Mushrooms |
20% |
|
Pineapple |
3% |
|
Canadian Bacon |
3% |
|
Green Peppers |
10% |
Customer buying patterns really start with second order take rates, which tell us about pairs of options, or toppings. Second order take rates tell us about relative popularity, but they also reveal something deeper: dependence. If you know that a pizza has pineapple on it, there is a very good chance that it also has Canadian bacon. This is dependence. In this case the reason is that there is a widely known “Hawaiian Pizza” that has both of these toppings. In general, customers don’t flip coins or roll dice. They select options that “hang together” in some way. The patterns can be seen in the combined take rates. Let me illustrate with three examples that are contrived to illustrate some important points. First consider pepperoni and mushrooms together.
|
|
|
Mushrooms |
|
|
|
|
No |
Yes |
|
Pepperoni |
No |
18% |
12% |
|
Yes |
32% |
8% |
|
In this table you can see that pepperoni has a 40% take rate, since 32% of pizzas have pepperoni without mushrooms, and 8% have pepperoni with mushrooms. In the same way, 20% have mushrooms, because 12% have mushrooms without pepperoni and 8% have mushrooms with pepperoni. This illustrates the first law of second order take rates: the first order take rates must be preserved. But this table contains no new information. Customers are apparently ordering pepperoni and mushrooms independently. This is revealed by the fact that 8% is exactly 20% of 40%. Knowing that a pizza has pepperoni does not give us any clue about whether or not it has mushrooms. Similarly, knowing that it has mushrooms is useless in guessing if it has pepperoni.
As a second example, consider the two toppings that are on every Hawaiian pizza: pineapple and Canadian bacon.
|
|
|
Canadian Bacon |
|
|
|
|
No |
Yes |
|
Pineapple |
No |
97% |
0% |
|
|
Yes |
0% |
3% |
For simplicity, I have made this an example of complete dependence: a pizza has pineapple if and only if it has Canadian bacon. Notice that the first order take rates (3% for each) are preserved.
The third example is the really important one: partial dependence. This is illustrated here by pineapple and green peppers.
|
|
|
Green Peppers |
|
|
|
|
No |
Yes |
|
Pineapple |
No |
89% |
8% |
|
Yes |
1% |
2% |
|
In this case, the first order take rates are also preserved: 3% for pineapple and 10% for green peppers. But these two choices are not independent. The take rate for pineapple and green peppers together is 2%, which is much greater than 10% of 3%, which would be only 0.3%.
Exercise for the reader: show that if we know that the pizza has green peppers, then there is a 20% chance that it has pineapple. (Much greater than its 3% first order take rate.) And if we know that it has pineapple, then there is a whopping 67% chance that it has green peppers!
So second order take rates capture information about how customers are combining toppings, and we can use that information to make predictions.
Comments Off
Value of SKU Intelligence (What Are Customers Buying?)
One of the most frequent questions we are asked about Emcien’s methodology is “why hasn’t anyone done this before?” The answer seems obvious to us, but we should probably write it down once and for all so we can just point people to a definitive document.
Exploding SKU’s to attributes generates a LOT of data. It is only useful if this translates to actionable intelligence. As we all know, we don’t need more data. We need actionable items and recommendations to improve business. The simplest answer is that collecting all the attribute data for SKUs has not been done before, because the algorithms have not existed to farm intelligence from them. In this blog I am going to address –
- Why having the data with the SKU attributes is invaluable!
- What analytics capabilities are needed, if you have that data
- Value of having SKU Intelligence (To drive SKU velocity)
Consider a product that has attributes, and offers lots of variety, also called configurations. Almost all products come under that class today! Think car, computer, or cell phone. The product has features, and each feature has alternative options. For example, a car has an engine (V-6 or V-8), a body type (sedan or convertible), and a color (red, green, blue, black, or white). Any specific car has many choices for each of the features. Customers make choices on the features. Do you want the cloth seats or the leather seats? Do you want the DVD player? How about the iPod connector? Make a choice for about 30 features and you are done. This applies for shampoo, toothpaste, computers, light fixtures, consumer electronics, … all products that have attributes choices.
We consider a configurable product that has a large number of features, each of which many alternative choices/options. Notice that we have finessed the hierarchy problem by allowing only two levels: feature and option. A customer will typically call out only a few attributes during a purchase. They expect you to know how you can complete that spec to fill the order. That is the single biggest opportunity at every point of sale!
If we have data with attributes for every SKU, we can begin to talk about buying patterns! A buying pattern is groups of options that are bought together, like the red color with the convertible body style. Or the DVD player with the leather seats. Or the pattern might involve 3 options, 4 options, 5 options…. many options.
Auto detect most popular attributes in fastest sellers
This brings me to a very important point. If you have sales data with attributes, you need an analytics engine that will automatically detect and tell you what attributes are bought together and are highly popular. A reporting tool that makes you query every choice combination will NOT work! You will be very old by the time you get an answer to most popular choices, as there are millions of attribute combinations. Emcien offers an analytics engine with cluster analysis that will tell you what attribute groups are popular. This answers the questions – what are customers buying and what attributes are popular. This knowledge can also be used for planning the SKU definition and knowing what products need to be on the top landing pages of your web site/ store aisles. (SKU Definition is the list of attributes in the SKU.)
Once the SKU definition is in place, 75% of the cost structure and efficiency of your supply chain has been fixed ! Your supply chain operates under the assumption that the SKU definition is correct. What does this mean? This means that if you offer 2 SKUs with slightly different attributes, that could have been consolidated into one, the supply chain will suffer that cost and inefficiency. The SKU definition has to be optimized before you send it into the supply chain. This is a key driver to SKU inventory.
Customers buy products based on choices at the attribute level. If you cannot gather demand intelligence at the attribute level, you are out of touch with your customer. Customers DO NOT buy SKUs. They do not know the SKU numbers, and they do not care.
SKU Challenges Based on Supply Model
Your business falls into one of these categories based on your supply model (see table). Knowing what attributes customers are buying can dramatically improve your demand response. You will be able to improve -
- SKU definition -This means knowing how many SKUs you need and what needs to be in the SKU
- Demand Forecasting – You will be able to forecast demand at the attribute level, which is the level that customers are buying
- SKU Inventory - plan what to stock to have highest turns.
- Sales Productivity and Efficiency – If you know what attributes are selling together, you can implement an automated recommendation engine (EmcienMatch) and your sales rep can recommend a good choice to the customer during the ordering process. This is probably the single biggest value of knowing what attributes customers are buying. Your sales reps are a trusted advisor to your customers. If the sales person know that all customers of a certain type bough a particular configuration, he could recommend that choice to the customer. Win! Customer is happy because he done! Sales person is happy because he looks smart! You are happy because you increased your sales repeatability!
For our Nerdy Math readers, here is technical nugget that you will love:
SO – here is another reason why what Emcien does has not been done before. Statistics deals with numerically valued variables. A numerically valued random variable X has a domain that can be classified as a ratio scale, an interval scale, an ordinal scale, or a nominal scale. In a nominal scale, the numbers {0,1,2,3,…} are just labels and have no numerical significance. In an ordinal scale the numbers provide an ordering, so that 2>1 in some appropriate sense. An interval scale allows real numbers (like 3.27) and differences are significant, but ratios are not. The classic example is temperature. A difference of 10 degrees is bigger than a difference of 5 degrees, but 100 degrees is not twice as hot as 50 degrees. Finally, a ratio scale allows all of the usual numerical operations on real numbers. Most of statistics assumes a ratio scale, and almost all of it assumes at least an ordinal scale. This leads directly to the mean, variance, covariance, and correlation. It also leads to metric spaces with distance functions.
But we want to consider a random variable X that represents a choice between cloth seats and leather seats. You might argue that leather > cloth, because leather is more expensive. But what about color? Is there any sense in which green > red? We are really interested in nominal scales. We may assign 0, 1, and 2 to “none”, V-6, and V-8 respectively, but these are just labels. That has a profound effect on the statistics we can use. And beyond statistics, we will have to develop notions of proximity or closeness that do not depend on distance functions. So it hasn’t been done before because the mathematics you need is not the popular stuff that’s in the textbooks.
Comments Off
Part II: Analytics to Action….. The Holy Grail
Can-you-have-too-much-information?
… in my last blog we talked about reporting, BI and data mining, and ? the information overload. So how can help business users with solutions for better decision making, as opposed to drowning them in more data and pretty charts? That is the Holy Grail and the purpose of all this data!
Lets start by defining analytics. So, what is analytics? Neil Raden of Hired Brains, a market research and management-consulting firm, has said that, “the proper term for interacting with information at the speed of business, analyzing and discovering and following through with the appropriate action, is ‘analytics’. I agree. In the information age, this must be done by specialized applications built on analytics based on the requirements of the actions/recommendations required by a business function. Dumping data on a users lap with the message – “Figure it out!” is NOT analytics, and it not very useful either. (I am reproducing a picture I really like as it conveys the message so very well! The Picture is from mathewingram.com/work)
So – how can we transform the user experience for analytics? As mentioned earlier, this can only be accomplished by focusing the analytics on a business problem with the mission to deliver actionable tasks. The challenge is selecting a business problem that the analytics truly delivers unique capabilities and intelligence that is relevant to that problem. This level of focus can be perceived as very limiting, and hence many choose not to go this route. Why limit the scope of the analytics to one specialization, when we can claim that we can do everything! To that I say – you are better off doing one thing very well, as opposed to many with mediocrity at best.
I am going to bring this back to Emcien, as this is a company that has focused analytics on a very specific business problem. The problem is one of product variety, product variants, and lots of attribution. In this age of product variety, that is a problem that is causing tremendous challenges to various business functions.
The analytics automatically detects what features customers are buying, where you are making money. This SKU or configuration intelligence is leveraged for:
SKU Intelligence Analytics Used to Drive Application Specific Recommendation
- Better forecasting at the mix level - The application uses the analytics intelligence to determine the exact product mix with very high accuracy based on true demand sensing.
- Improving the customer experience at the point of sale - The application uses the analytics intelligence and guides the buyer to a good configuration based on the few features they have called out. And by the way – customers love it when you can recommend a configuration based on the few features they ask for. They want you to stop asking more questions and recommend a good choice.
While the analytics may throw out volumes of data, the user can relax, as he does not have to crawl through volumes of date wondering what it is telling him. Converting analytics to actions and recommendations minimizes human interpretation and error on a day-to-day basis. For analytics to be functional in business applications, this is a mandatory requirement in today’s business environment.
So – when you are evaluating BI tools, Analytics, Data mining….. what ever they are calling it! Ask yourself, how am I adding value to the company? What am I giving my business users? Am I adding more work to their busy schedule by piling on data on their computers???? If the answer is YES, please don’t do it. They will thank you for it.
If the data has not been converted to recommendations the business can act on, you will not get value from your investment!
Recommendation engine for configurable products
The Amazon recommendation engine has received a lot of attention and imitation. It has been successful at increasing sales by pointing out that people who bought book x also bought book y. This simulates a helpful book store employee who has an extensive mental map of how books relate to each other. Recommendations have been most successful for books, movies, and music. Companies that sell complex configurable products could also benefit from a system of automated recommendations. Products like trucks, tractors, computers, lighting fixtures, valves, and industrial fans. These are products that the buyer can customize to his own preferences or needs. The buying process is complex, and sales agents make recommendations, just like the book store expert. But many of these products are far more complex than books.
How is a book different from a truck? Could a recommendation engine for books be used to recommend trucks? The purpose of this note is to consider the ways in which the two cases are similar and different, and to explore what a recommendation engine for configurable products might look like.
Books and trucks can both be described in terms of attributes (also called features or characteristics). Books can be described by their genre, author, language, and publication date for instance. Trucks can be described by their engine, transmission, wheelbase, gross vehicle weight, and many other attributes. Books, movies, and music can be classified in sufficient detail with ten or fewer attributes, while configurable products usually have a lot more. Heavy-duty trucks can have anywhere from 30 to 300 attributes. For books, the attributes are used to classify existing books. For configurable products, each attribute represents a choice for a customer who is ordering the product. Each attribute has several alternative options, so an order is really a list of option choices.
Amazon uses attributes to let customers search for a book, but the recommendation engine does not use them. The recommendation engine remembers each customer’s purchase history. For example, Joe has already bought “War and Peace” and “Crime and Punishment”. When he buys “The Sound and the Fury”, a connection is made between “The Sound and the Fury” and “War and Peace”, and another connection is made between “The Sound and the Fury” and “Crime and Punishment”. More accurately, the weights of these connections are increased. The connection between “The Sound and the Fury” and “War and Peace” depends on the number of customers that have bought both books. Commonality depends on common purchase by individual customers.
One big difference between books and trucks is that an individual will buy many books, but is unlikely to buy many trucks. So the basis for the connections between trucks can’t be the common purchases of individual buyers.
Another difference is that only books that have already been written are of interest. No customer is looking for a book that hasn’t been written yet, and you certainly can’t recommend one. But because of the very large number of choices, a buyer who is customizing a truck may arrive at a configuration that has never been built before. So our recommendation system must be able to accommodate trucks that don’t exist yet.
The helpful book store employee will jump from one book to another book that his customer might like. When we look at the helpful sales agent, he is doing something quite different. Based on the first, and usually most important, choices that the buyer makes, he will suggest ways of completing the order. The buyer may really only care about 10 of the 30 choices, and want guidance on what else to choose. The book store employee jumps from one complete item to another complete item. The sales agent is finding a path from a partial order to a complete order.
So we have an initial set of requirements for a recommendation engine for configurable products. It must help a buyer who has made some choices to make additional choices, so as to arrive at a complete order. From the seller’s point of view, we would want these recommendations to guide the buyer toward items that are advantageous. This might mean popular, in stock, and/or profitable. Its concept of the product (e.g. truck) must be broad enough to encompass configurations that have never been built before, and its concept of commonality must be based on something other than common purchases.
To show one way this might be done, consider a different way of making connections. When a customer buys a red truck with a V-8 engine, he is making a connection between red, a value of the Color attribute, and V-8, a value of the Engine attribute. So the customer’s vote is not recorded as a connection between two complete trucks, but as a connection between two attributes. More precisely, it is recorded as a connection between two values, or options, of the two attributes.
When a customer buys a truck with n attributes, we will record his “vote” n times, once for each of the options he has chosen. Then we will also record it nC2 more times for each pair of options. The symbol nC2 stands for the number of different pairs of attributes there are, and can be computed as n*(n-1)/2. For example 10C2=45. The relative popularity of the different pairs of options (like red and V-8) tells us something about how customers are buying the product, and gives us a different basis for making recommendations. (If you want pineapple on your pizza, you probably want Canadian bacon.)
If pairs of options are interesting, then so are triples of options. The same customer vote can be recorded for nC3 different sets of three options (10C3=120). Every option of every attribute has its own popularity (i.e. number of votes). Expressed as a fraction of total units sold it is known as a first-order take rate. Every pair of options also has a popularity, and as a fraction is defined as a second-order take rate. Similarly, each triple has a third-order take rate. Of course we can also define fourth, fifth, and higher-order take rates.
If customer purchases (votes) are recorded in this way, then we will have captured the buying patterns in the form of first, second, third, and higher-order take rates. A new customer who begins to select the options that are important to him will trigger these patterns, which will then serve as the basis for recommending a complete truck. So we see that two complete trucks may be related because they both contain instances of the same fifth-order take rate. This can be true even if neither truck has ever been built before.
We conclude that a recommendation engine for configurable products could be built by mapping customer purchases into a structure of take rates that record the relative popularity of options, pairs of options, triples of options, and so forth.
Comments Off
Part I: Reporting, Business Intelligence, Data Mining, Analytics: Actionable Tasks!
Software vendors use so many big words and confuse customers. Our customers have often asked us to clarify – so here I go. The definitions in this article are based on research of these terms, and the collective opinion of many of our customers and prospects. Over numerous conversations with our customers and the discussions of the terminology, the clarifications always go back to the origin of the terms and then move on to change in usage. Hence this article folows that flow. I would love your feedback as it is important to help buyers understand this.
Business Reporting
Business Reporting, as the term suggests presents the data from the database in an easy to read format. This originated when business users were frustrated that all the data was locked up in databases. There was a lot of data, but no one could get access to it without calling on IT folks. Hence Business Reporting was born.
Business Intelligence
This is a fancy name for business reporting. Business intelligence (BI) is a broad category of technologies that allows for gathering, storing, accessing and analyzing data to help business users make better decisions. In a 1958 article, IBM researcher Hans Peter Luhn used the term business intelligence. He defined intelligence as: “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.”
In 1989 Howard Dresner (later a Gartner Group analyst) proposed Business Intelligence as an umbrella term to describe “concepts and methods to improve business decision-making by using fact-based support systems.” Then in the late 1990s the usage became widespread (Remember the Bubble!). Then. everything with any data reporting was called Business Intelligence. So today, Business Intelligence is a glorified term for “Business Reporting”.
Data mining
Simply put, Data mining is hitting the data with all mathematical methods available to a mathematician! The data source can be almost anything – news papers articles, financial reports, sales data, medical data, … . This means that the data can have structure or can be un-structured. And the mathematical methods that can be applied can include neural networks, genetic algorithms, statistics on steroid and anything else they can think of.
One may ask – why are they doing this? What are they mining? Well, the simple answer is that they are mining the data looking for patterns; any patterns that can reveal relationships. So the methods used are varied and the kinds of data that are mined can come from a myriad of sources.
The results of data mining are lots of data! In fact – the result of Business Reporting and BI has been data overload. Now that’s the bad news. In a world of information overload, the last thing that we need is more data. We have less time today than we have ever had before. Business users do not need more data. They need quick conclusions on what the data is saying, converted into actionable tasks. Simply put – “Please tell me what to do”.
… More on the discussion of analytics to action in the next blog.
Comments Off