Geo Experiment Using Google's Matched Markets Package | Open Source | Time Based Regression

Geo-experimentation is one of the most common ways of carrying out incrementality tests. By now, unless you have been living under a rock, you know that incrementality tests are the gold standard for establishing causality of a marketing initiative.

And among popular approaches for geo-experimentation is the open source package from Google called Matched Markets, which uses the principle of time-based regression (TBR).

The best part about this is that you can run this even if you are not a data scientist - you just need to be a little familiar with Google Colab/Python environments and you can start running it by providing just a couple of inputs.

1. The Cost & Sales data for each geo per day
2. The Geo Eligibility criteria, if any

Later, you also need to provide the post-test analysis data.

At the end of the analysis, you will budget recommendations, split of geos into treatment and control groups and lift analysis.

00:00 What is Time Based Regression?
01:00 Why Matched Markets?
02:13 Why Geo-Experiments in the first place?
04:00 Overview of the Google Colab notebook
04:47 Input Data Sheet: Cost & Sales Data by Geo & Date
05:52 Input Data Sheet: Geo Inclusion Eligibility
08:05 Parameter Selection Process
10:26 Greedy Search for Research Design Selection
16:00 Design Diagnostics & tests
19:15 Post-Test analysis
22:14 Visualization of point-wise and cumulative lift
24:45 Summary

Here is the link to the Google Colab:
https://colab.research.google.com/drive/16KUo3EkMQcjlyNCUMS4gTdhVIZB7PfAY?usp=sharing

Here is the link to the official Matched_Markets Github repo:
https://github.com/google/matched_markets

Here is the link to the Google Doc with notes:
https://docs.google.com/document/d/1pl3J7c1tGWJVqB4t01ggndsD2pEp5RzMUd4P5fvEn_c/edit?usp=sharing

Here are the sheets for the inputs:

https://docs.google.com/spreadsheets/d/10BVKhITRbEjCDQUvoiErL81CCtHPO-Rni4PgrwPHNUA/edit?gid=2135415271#gid=2135415271

https://docs.google.com/spreadsheets/d/1O8SQxCm9-qYhRTQEByrQ65m62SgKzHUc4L-S8r4lQW4/edit?usp=sharing

Other related videos on incrementality & experimentation:

Incrementality Test Using DiD
https://youtu.be/Ky0hcsDkhwU?si=W5V2-VA3klvxW_bX

Google CausalImpact vs Meta GeoLift
https://youtu.be/NqMeRRTkGJk?si=jmcWNmwJs5EY97pb

Statistical Significance vs Duration of Testing - When to stop a test
https://youtu.be/qnUhrN-t1xw?si=567KLdAiFyZkGCVi

Different ways to measure incrementality
https://youtu.be/skViTNHj19k?si=u4ENzhOBzmxv9kXu

Incrementality testing with Event Study Model
https://youtu.be/wgOTbzA2WNc?si=Ze9U3JhIUiwqL8wv

BFCM Forecasting
https://youtu.be/i-YJfHSdV2c?si=3YdzwoCKt095HiNM

Incrementality Testing for Causal Impact of Google/Meta
https://youtu.be/5xDeHELcP8w?si=71REl3NhSSNazeIp

Geo-clustering for geoholdout test
https://youtu.be/K6iAZWk72k4?si=YBHqIzsaDumPTytN

Measuring incrementality of branded search
https://youtu.be/K12dVzkXiFs?si=jTBLxCAfYNKzJcx_

PMax incrementality testing template:
https://youtu.be/Ztr_UE_jzJg?si=ODoowFbPHCBcWyPZ

Multi-channel incrementality testing template:
https://youtu.be/TipcZdfm8EY?si=D0GB8oM9w186fcjZ

Understanding geo-experiments | Geo-Holdouts | Causal Inference
https://youtu.be/3dJmdgO3j6c?si=Rq73655s7NyliS_I

#statisticalanalysis #incrementality #geoexperiment #marketinganalytics #marketingmeasurement #marketingeffectiveness #marketingscience #causalinference

Видео Geo Experiment Using Google's Matched Markets Package | Open Source | Time Based Regression канала Marketing Analytics With Kisholoy