Confirming the Trivial, Uncovering the Mysterious, Exploiting the Useful

What is Basket Analysis?

Do all these items belong in the same customer basket? — Photo by Julius Drost on Unsplash

Basket analysis is a pretty simple concept. Imagine customers shopping at a grocery store. They grab items off the shelf, walk to the register and pay. You then collect all the receipts from your customers and create a list, customer by customer, of what they bought (i.e., what was in their basket or shopping cart). You might end up with something like the following with {…} denoting items in a customer’s basket:

Why that student in class who asked your math teacher “When am I ever going to use this stuff?” is probably broke

In an earlier article, I discussed how to use the STL algorithm to break any time series into its seasonal, trend and noise component. I walked you through an example using the stock price for Amazon. In this article, I am going to show you how you can use the same technique to save money when renting an apartment. I am going to use monthly average rental prices for 1 bedroom apartments that Apartment List makes available here between 2017 and 2021. If you want to see example code, see my earlier article.

If you didn’t read my earlier article…

Making Every Day Decisions Based on Data Science

Its a pretty common problem for businesses to stare at data plotted against time (e.g., sales over time) and try to separate seasonality from the underlying trend. More often than not one can sort of see the trend but its buried within a strong seasonal component. One typical strategy for dealing with the problem is to track Year-over-Year (YoY) changes expressed as a percentage but this can be just as difficult because the YoY change can swing from low single digits to high double-digits over the course of a few days. Fortunately there is a simple mathematical algorithm that can…

Writing Good Business Requirements Documents

Image by Author

In an earlier article I discussed the importance of segmenting your team’s analytics (or interchangeably data science ) work into four (4) buckets to help you effectively prioritize. At the end of that article I mentioned seven (7) key mechanisms for successfully sticking to your allocation investment strategy and prioritization without being randomized. One of those mechanisms is insisting that all strategic projects begin with a well-written Business Requirements Document (BRD). In this article I will provide a brief example of what makes for a well-written BRD. …

Remember you’re working on a business problem, not a science experiment.

Oftentimes the biggest challenge in analytics or data science is not the actual work itself, it's communicating the results to business users who often do not have any technical background themselves. Communicating effectively can be more important than the data science itself because if you are not able to get your business users to understand what you are doing, then you cannot build trust. People tend to dismiss advice if they do not trust the person delivering it… and they won’t trust you if they don’t understand what you are talking about. So, how can you communicate more effectively?

Notes from Industry

Generally Analytical Work Segments Well into Four Buckets

Image by Author

I am going to write a series of short articles about best practices for data science (or interchangably analytics). I strongly believe that the primary challenge to having a good data science department within organizations large and small is having the right processes; this is more important in many ways than data or algorithms. You can have the smartest data scientists in the world with the most sophisticated algorithms, but if you use them the wrong way it won’t matter.

Learning how to prioritize analytical work well is the first, basic thing you need to create a well-functioning data science…

Hint: It’s Not What Most People Think

Image by Author

An A/B test is pretty much the gold standard for measuring the impact of marketing spend, website features (e.g., language, visuals, fonts) and many other business processes in an objective fashion. It’s as close to a “physics experiment” as you can get in business. It’s also the source of much confusion and misunderstanding. Before diving into what an A/B test actually measures, let’s first discuss what an A/B test is. An A/B test is defined by two versions of something we want to test, version A and version B, sometimes also known as the test and the control, which will…

Notes from Industry

Academics Invent New Ones All the Time but in Business Just Three Carry Most of the Load

Image by Author

There are more statistical tests than one can count; all of them have a reason to exist and mathematicians are paid to invent new ones all the time. But in my experience practically speaking business analysts only need to know three basic tests:

In business analytics the majority of the time you are working with averages, counts or distributions. Let’s look at a few examples and my recommendations of what test to apply:

Photo by Christopher Burns on Unsplash


Four Tools that Enable You to Free Your Data Scientists to Focus on What will Add Value to your Business

Automation is the second key pillar of a well-run, efficient and useful data science (or interchangeably analytics) department within any business. I’ve found it to be one of the most common weaknesses over the years. I spent a few years in consulting and I was consistently amazed to watch organization after organization solve their data science challenges by simply “throwing more people” into the department instead of looking for a better way to use the people they had. I saw the same thing at the first business I joined soon after my years in consulting: sharp analytical minds wasted on…

Properly Layering your Data into L1, L2 and L3 Layers

Image by Author

An analytics data warehouse stores essentially the same data generated by transactional databases — nothing more, nothing less. But whereas a transactional database may be designed to be as fast as possible, an analytics database must be designed to be as flexible and consistent as possible with all the business logic embedded in the data warehouse. The goal of the analytics data warehouse is to enable a data scientist (or interchangeably analyst) to rapidly mix data to answer complex questions and discover useful information that is not obvious. However, the analytics data warehouse has another equally important goal: make it…

Elvis Dieguez

An Amazonian academically trained in Physics and Electrical Engineering experienced in Data Science, Data Engineering, Analytics and Business Intelligence.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store