Tag: patterns
Connect With Us
#GartnerSym – From Twitter Stream To Cohesive Summary
Emcien is monitoring the tweets from the Gartner IT Symposium in Cannes (#gartnersym).
Emcien’s Pattern Recognition Engine extracts the core essence of what people are saying….
Here is summary of the 5,235 tweets from the 3-day conference.
The summary shows the highest ranked word-groups……. Enjoy!
#GartnerSym…. What Are They Saying (Automated Cohesive Summary of the Tweet Stream)
Making Analytics Actionable
I just read an article titled “Making Analytics Actionable” by Michael Vizard. He makes two good points – predictive analytics is not new and analytics needs to be actionable. I could not agree more.
On the first – predictive analytics has been around for a long time. We used to call it forecasting. It was difficult then, and it is difficult now. Forecasting gurus, or should I say Predictive Analytics gurus have thrown every mathematical trick at the data to predict the future. It reminds of a quote by the CEO of a Fortune 500 company, who said “No one knows how to forecast. If they did, they would be in a different business.” I think by different business he meant forecasting money on Wall Street. But we all know that has gone! And now may be the quants on Wall Street would agree with him as well.
On the second point – yes, analytics should give you actionable information. As I hear from our customers, time and time again, they do not need more data. Companies are drowning downing in data. (maybe “downing” is the right word!!! We are downing in data! It is a downer!!!) In the name of Business Intelligence, they now have the capability to slice and dice this data, creating more data! The purpose of analytics is to convert all that data into something meaningful and actionable. If the analytics does not accomplish that, it is just another BI tool.
As you investigate analytics for your business, here are a few best practices:
- The analytics need to be focused by business function.
- The analytics needs to answer the question “What do I do with this?” and “what is the business value”.
- The analytics should make your job easier, and the recommendations should want you coming back to it over and over again.
The answer to the question “What do I do with this?” should be actionable tasks that a business should be able to run with. That is called Business Analytics. The reason the third point exists is because you, the user, would only come back to it over and over again if it made your job easier. That is the key to business analytics. “Take all that data and convert it into actionable tasks and make your job easier”. How is that for a tag line!
Comments Off
Do Customer Buying Patterns Exist?
Customers have to make choices in order to buy configurable products. Do they make these choices at random, or are there patterns? When we look at the sales history for a configurable product, like a car or a computer, can we tell if customers have just been flipping coins and rolling dice? Or do their choices hang together and make sense? To answer this question, we would have to look at how they buy combinations of options. In the previous post, I took a pizza as a simple configurable product, and looked at how customers ordered pairs of toppings. Just by looking at the sales numbers we could detect that the selection of pineapple and Canadian bacon are not independent. Even if we had never heard of a Hawaiian Pizza, we could discover it in the data.
Even more information is hidden in combinations of three toppings at a time, or four toppings at a time. Any combination of toppings will have appeared on some of the pizzas that have been sold (or maybe none). The relative popularities of all the different combinations has a clear message: customers are not flipping coins. Some toppings naturally go together, and others do not. Pepperoni, broccoli, and anchovies is just unlikely. If a particular pizza restaurant has a few “house specials”, like the Meat Lovers and the Veggie Delight, we can see them in the data, even if we don’t know their names.
What is true of the pizza is also true of other configurable products: computers, trucks, tractors, lighting fixtures, industrial fans, and so on. All products that have variety. Customers make choices, but not by rolling dice. There are combinations that go together and combinations that do not. A pizza maker can juggle the preferences of his customers in his head. But when a product has 30 or more features, intuition is overwhelmed. The number of combinations explodes so fast that the unaided human mind can’t see the patterns. At this point, mathematical models and intense number crunching can reveal the patterns and let the product manager for a line of trucks be as confident as a pizza maker.
Do Customer Buying Patterns Exist?
Buying patterns are real, and they manifest themselves in how customers buy combinations of options. With the computing power we have available today we can detect and capture them. These patterns can then be used to design “house specials”, forecast future sales, and guide customers to what we want to sell them.
So, who else is talking about customer buying patterns?
Intel Talks about Changing technology buying patterns
Customer Buying Patterns have Changed. What’s Your Plan?
An entire report that summarizes the results of a consumer usage and purchasing pattern survey conducted in March of 2007. The survey was conducted with In-Stat’s Technology Adoption Panel (TAP) — a dynamic, online panel of more than 19,000 technology users and decision makers. Over 1,400 technology users responded to this focused survey.
Findings in this report include consumers’ time spent on PCs, when they last purchased a personal-use PC, the PC’s features/form factor/usage, the desired features of future PC purchases, changes in usage patterns, and consumers’ thoughts about new technologies.
The changing patterns include -
- When consumers are likely to make their next PC purchase.
- The features consumers state they want
- The features consumers state they really want, based on changes in their usage/buying patterns.
- How consumers view new technologies
However – buying patterns are constantly changing. As social networking grows, we are watching new markets emerge every day. There is gold for companies who can continually detect these patterns and offer the right products and feature mix.
Customer Buying Patterns – What you can learn From Pizza Sales
There are 1,140 ways of ordering a pizza with 3-toppings, if the pizza offers 20 choices
First order take rates tell us about the relative popularity of different options. For example, consider a small set of possible pizza toppings.
|
Topping |
Take Rate |
|
Pepperoni |
40% |
|
Mushrooms |
20% |
|
Pineapple |
3% |
|
Canadian Bacon |
3% |
|
Green Peppers |
10% |
Customer buying patterns really start with second order take rates, which tell us about pairs of options, or toppings. Second order take rates tell us about relative popularity, but they also reveal something deeper: dependence. If you know that a pizza has pineapple on it, there is a very good chance that it also has Canadian bacon. This is dependence. In this case the reason is that there is a widely known “Hawaiian Pizza” that has both of these toppings. In general, customers don’t flip coins or roll dice. They select options that “hang together” in some way. The patterns can be seen in the combined take rates. Let me illustrate with three examples that are contrived to illustrate some important points. First consider pepperoni and mushrooms together.
|
|
|
Mushrooms |
|
|
|
|
No |
Yes |
|
Pepperoni |
No |
18% |
12% |
|
Yes |
32% |
8% |
|
In this table you can see that pepperoni has a 40% take rate, since 32% of pizzas have pepperoni without mushrooms, and 8% have pepperoni with mushrooms. In the same way, 20% have mushrooms, because 12% have mushrooms without pepperoni and 8% have mushrooms with pepperoni. This illustrates the first law of second order take rates: the first order take rates must be preserved. But this table contains no new information. Customers are apparently ordering pepperoni and mushrooms independently. This is revealed by the fact that 8% is exactly 20% of 40%. Knowing that a pizza has pepperoni does not give us any clue about whether or not it has mushrooms. Similarly, knowing that it has mushrooms is useless in guessing if it has pepperoni.
As a second example, consider the two toppings that are on every Hawaiian pizza: pineapple and Canadian bacon.
|
|
|
Canadian Bacon |
|
|
|
|
No |
Yes |
|
Pineapple |
No |
97% |
0% |
|
|
Yes |
0% |
3% |
For simplicity, I have made this an example of complete dependence: a pizza has pineapple if and only if it has Canadian bacon. Notice that the first order take rates (3% for each) are preserved.
The third example is the really important one: partial dependence. This is illustrated here by pineapple and green peppers.
|
|
|
Green Peppers |
|
|
|
|
No |
Yes |
|
Pineapple |
No |
89% |
8% |
|
Yes |
1% |
2% |
|
In this case, the first order take rates are also preserved: 3% for pineapple and 10% for green peppers. But these two choices are not independent. The take rate for pineapple and green peppers together is 2%, which is much greater than 10% of 3%, which would be only 0.3%.
Exercise for the reader: show that if we know that the pizza has green peppers, then there is a 20% chance that it has pineapple. (Much greater than its 3% first order take rate.) And if we know that it has pineapple, then there is a whopping 67% chance that it has green peppers!
So second order take rates capture information about how customers are combining toppings, and we can use that information to make predictions.
Comments Off
True Demand Intelligence – Knowing who is buying what, where and why
Demand Intelligence – Knowing Who is Buying What, Where and Why
SKU numbers are an easy way to keep track of items that are built, stocked and sold. The SKU number itself is arbitrary and contains no ‘intelligence’.
The SKU number was invented for a very good reason. When this practice started, the number of SKU’s was small, and people and systems needed a simple way to track what they had. So ASP-678 may be the SKU number for a toothpaste tube with spearmint flavor, whitening, and tartar control. However, customers do not look for SKUs. They look for toothpaste with whitening, tartar control, flavors, sizes, etc. Customers buy attributes, and combinations of attributes. Some companies code the attributes into the SKUs, by concatenating two or three character codes (like SPWHTC). But this is at best a clumsy way of handling a few attributes. Companies want to know what attributes customers are looking for, and SKU numbers hide the attributes.
As the number of attributes starts to grow, whether you have them coded into the SKU number or not, the problems start to mount! The most important one is that companies do not know what customers are buying, or trying to buy. As the number of choices grows, the number of combinations grows much faster and companies drown in their own SKUs.
SKU intelligence is going behind the SKU numbers and ‘detecting what attributes customers are buying’. Knowing who is buying what, where and why is “True Demand Intelligence”.
Products have attributes. For example, a computer has a processor, a memory, and a hard drive. For each attribute there may be several alternative choices. This means that there are many different product configurations. Some companies make only a fixed subset of all the possible configurations and give each one a SKU number. Other companies allow customers to order exactly what they want, and if this is something new, then they create a new SKU number for it. In either case, a SKU number is supposed to represent a unique product configuration.
If you are trying to figure out what your customers want, then SKU numbers are a form of encryption. You have to look at your sales history in terms of the underlying attributes, and the choices for those attributes. Instead of looking at one SKU number you need to look at perhaps 20 separate attributes. The SKU number is a way of collapsing those 20 dimensions into a single dimension, with tremendous information loss. One of the things that is lost is proximity to other SKU’s, based on attribution. A customer who bought SKU A-1234 might have been satisfied with (or really looking for) SKU B-3728. These two SKUs have the same choices for 18 of the 20 attributes, and differ on only two. This is obvious when the unique configurations are represented as a set of attribute choices, but hidden when they are represented as SKU numbers. The first step in analyzing a sales history has to be expressing it in terms of the underlying attributes. Each SKU number has to be expanded into a list of choices. Then we can begin to find patterns in how the choices are made. The leather seats and the DVD player are usually bought together. Engine block heaters are not ordered on convertibles. Buying patterns exist at the attribute level, not at the SKU level.
“Buying patterns” are popular combinations of attribute choices. These can be pairs of attributes, triples of attributes, or even more. Popularity is measured by the share of sales that have that combination. Buying patterns are helpful in selling, because they reveal how customers can be moved to configurations (SKUs) that we have in stock, or that we would prefer to build. Experienced sales people are skilled at moving customers, but If these patterns are represented in some kind of knowledge base, then a computer can make the recommendations.
Customers also have attributes. The simplest is perhaps geographical location. There are patterns that involve both product attributes and customer attributes. Customers in Florida are more likely to buy convertibles; customers in North Dakota are more likely to buy engine block heaters. Customers may have several attributes, for example demographic attributes for individuals or industry attributes for companies. (We don’t assign SKU numbers to customers!) If your sales history contains information about the customer as well as information about the product, then we can look for buying patterns that are associated with certain kinds of customers.
As an example, for a desktop computer the list of attributes might be: Processor, Memory, Hard Drive, Keyboard, Monitor, Mouse, CD/DVD, Application. A specific SKU number like A-1234 is a code for a specific configuration, say (2GHz, 2GB, 120GB, Ergonomic, 22” flat panel, Wireless, R/W Combo, Gaming). The Application attribute is really a customer attribute, with values like Home, Small Business, or Entertainment, as well as Gaming. This would make it possible to look for typical Gaming configurations and typical Entertainment configurations.
SKU numbers are a useful shorthand for record keeping. Each SKU number represents a unique product configuration. But analyzing SKU numbers is like analyzing telephone numbers. To see the buying patterns, you have to go to the attribute level. The patterns exist among the attributes, so you have to decode the SKU numbers to see them.
Part II: Analytics to Action….. The Holy Grail
Can-you-have-too-much-information?
… in my last blog we talked about reporting, BI and data mining, and ? the information overload. So how can help business users with solutions for better decision making, as opposed to drowning them in more data and pretty charts? That is the Holy Grail and the purpose of all this data!
Lets start by defining analytics. So, what is analytics? Neil Raden of Hired Brains, a market research and management-consulting firm, has said that, “the proper term for interacting with information at the speed of business, analyzing and discovering and following through with the appropriate action, is ‘analytics’. I agree. In the information age, this must be done by specialized applications built on analytics based on the requirements of the actions/recommendations required by a business function. Dumping data on a users lap with the message – “Figure it out!” is NOT analytics, and it not very useful either. (I am reproducing a picture I really like as it conveys the message so very well! The Picture is from mathewingram.com/work)
So – how can we transform the user experience for analytics? As mentioned earlier, this can only be accomplished by focusing the analytics on a business problem with the mission to deliver actionable tasks. The challenge is selecting a business problem that the analytics truly delivers unique capabilities and intelligence that is relevant to that problem. This level of focus can be perceived as very limiting, and hence many choose not to go this route. Why limit the scope of the analytics to one specialization, when we can claim that we can do everything! To that I say – you are better off doing one thing very well, as opposed to many with mediocrity at best.
I am going to bring this back to Emcien, as this is a company that has focused analytics on a very specific business problem. The problem is one of product variety, product variants, and lots of attribution. In this age of product variety, that is a problem that is causing tremendous challenges to various business functions.
The analytics automatically detects what features customers are buying, where you are making money. This SKU or configuration intelligence is leveraged for:
SKU Intelligence Analytics Used to Drive Application Specific Recommendation
- Better forecasting at the mix level - The application uses the analytics intelligence to determine the exact product mix with very high accuracy based on true demand sensing.
- Improving the customer experience at the point of sale - The application uses the analytics intelligence and guides the buyer to a good configuration based on the few features they have called out. And by the way – customers love it when you can recommend a configuration based on the few features they ask for. They want you to stop asking more questions and recommend a good choice.
While the analytics may throw out volumes of data, the user can relax, as he does not have to crawl through volumes of date wondering what it is telling him. Converting analytics to actions and recommendations minimizes human interpretation and error on a day-to-day basis. For analytics to be functional in business applications, this is a mandatory requirement in today’s business environment.
So – when you are evaluating BI tools, Analytics, Data mining….. what ever they are calling it! Ask yourself, how am I adding value to the company? What am I giving my business users? Am I adding more work to their busy schedule by piling on data on their computers???? If the answer is YES, please don’t do it. They will thank you for it.
If the data has not been converted to recommendations the business can act on, you will not get value from your investment!
Stairway to (Product) Complexity (a.k.a. Why Do I have SO Much Stuff!!!)
In the last post I introduced two ideas about the sales history of any product. First, the number of unique configurations, or build combinations, depends on which features are included in its description. Second, the number of unique combinations drops whenever a feature is removed. A natural question to ask at this point is: Which feature, if it were removed, would lead to the greatest decrease in the number of unique configurations?
This is an easy question to answer, if we have a way of counting the number of unique configurations in any input set of configurations. This is really just a matter of sorting, so suppose we have such a counting algorithm. We try removing the features, one at a time. In each case we apply our counting algorithm to get a score. The score is the number of surviving unique configurations. The feature with the lowest score is the winner (like golf).
Now suppose that we permanently remove the winner, and repeat the contest again. This will determine a second winner, which can also be removed. We keep repeating until there are no features left. Now imagine a graph with the number of features removed on the x-axis and the number of surviving unique configurations on the y-axis. This is the stairway to complexity.
The stairway shown above is for a product (commercial stoves) with 23 features. The number of unique configurations starts at 1223 and drops to just 1 as the features are removed. The features are removed by looking for the biggest drop at each step.
Abstract:
Product complexity is driven by large number of options. Companies struggle to determine which feature choices are driving complexity. They typically “randomly cut choices” to streamline and rationalize SKUs. The cost of product complexity is tremendous on engineering. The current PLM systems do not have a method to measure this and provide intelligent feedback to engineers on how to standardize platforms to reduce engineering and maintenance costs. This article clearly details the metrics around product complexity and how to solve this issue.
Comments Off
The Number of Choice Combinations Depend…
The number of choice combinations depend on which product features are included. The build combinations is the product mix or the marketing mix.
Let’s consider the sales history of our product. There are two very important numbers: the number of units sold and the number of unique configurations. The number of units is well defined, but the number of unique configurations is ambiguous. The ambiguity comes from the fact that there will be more unique configurations if we use more features, especially soft features, to describe our product.
One very special soft feature is a Serial Number, or VIN (Vehicle Identification Number). The whole purpose of the Serial Number is to make each instance of the product unique. So if we look at our sales history and include Serial Number, we will see that the number of unique configurations is exactly the same as the number of units of the product (instances).
If we want to begin to understand the demand for our product we have to see which instances are actually the same. That means we have to get rid of the Serial Numbers. When we do, the instances collapse into groups of now unique configurations; that is, unique without Serial Number.
If we are interested in the tangible features of the product, then we may want to take out other soft features as well. Geographic region is important for some purposes, but may be a distraction when we are interested in the physical product. Taking out the geographic region feature will cause another reduction in the number of unique configurations. The red, V8, convertible in Florida will get combined with the red, V8, convertible in New York.
Sometimes we are interested in the variants of our product ignoring color. We know that every real variant is going to come in several colors, but we want to look at the product without the distraction of color. This is sometimes called the “body in white”. So the red, V8, convertible and the green, V8, convertible collapse into the V8, convertible.
The point I am making is that the number of unique configurations depends on which features are included, and this number drops whenever a feature is taken away. Mathematically, this is called “projecting out the feature”.
The number of unique configurations is at most the number of units sold, and at a minimum it is just one. If we take away all of the features, then every unit looks the same, which means just one configuration. There is a path from one extreme to the other that we will introduce next time.
By the way – understanding this is important as product complexity is a key driver of process complexity.
Comments Off
Shopping for a new monitor
I do a lot of work from home. My office has a huge picture window, a desk and my laptop. As a system administrator, I often read long log files or wade through large amounts of data; sometimes I need to work from an online reference as I’m tweaking system settings on my servers. A big monitor really helps with this, so last weekend I went off in search of the Right One for my office.
I started with some online searches to establish a baseline price for my budget. I really don’t like buying monitors online, though, because the picture quality from model to model can vary widely and it’s something that I like to see firsthand. So, wife in tow (to tell me when I’m going overboard – I do get excited about new hardware at times), we head to the local shopping extravaganza to look for monitors.
I settled on a few things I was looking for: size (the bigger, the better), resolution and contrast ratio. I do enough image editing for website tasks that picture quality matters, but not enough that I care about color correctness, so I just want a good, solid, clear picture. I’m not gaming, so I don’t care about super-fast update times, just a clear picture that makes log files and code easy to read. I’ve found that by clearly stating your priorities, it’s a lot easier to compare similar products and make a satisfying decision on which thing it is you’re going to buy. With these goals in mind, I found a few possible candidates:
- 22″ Acer, reasonable resolution, poor contrast ratio: $159
- 23″ Samsung, reasonable resolution, excellent contrast ratio: $209
- 28″ Viewsonic, fair CR, good resolution: $549
- 26″ Samsung, good resolution, good CR: $399 (with $100 rebate, so $299 if the rebate works)
- 24″ Dell, good resolution, good CR: $279
I settled on the 26″ Samsung. The price was fair. Although more than I had originally wanted to spend, it was a great deal – and even better if the rebate comes through. The picture was superb, and it sure does make working at home even nicer than before.
Comments Off
The typical tail graph
In a previous post, I discussed two types of sales history: raw and collapsed. The collapsed sales history can be displayed in a table or spreadsheet, with a special column for volume. If this table is sorted on decreasing volume, then the most popular configurations (popcons) will be at the top. The graph with the volumes displayed in decreasing order (popcons on the left) is called the tail graph
We have drawn tail graphs for cars, computers, washing machines, lighting fixtures, trucks and tractors, and they all look basically the same. The first tail graph shown below is small but typical. It represents 2,884 tractors, with 1,997 unique configurations, or build combinations. On average, there are 1.44 units per unique configuration. The most popular configuration was ordered 23 times. The graph quickly drops to two of a kind and finally one of a kind (our technical terms are “twosies” and “onesies”). Combined, the onesies and twosies account for 2,000 tractors, or 69% of total volume. Rather high, though this number is usually at least 40%.
Comments Off