Case Study Test&Control

Case Study Test & Control
Group A3
August 8, 2016
Agenda
Section 1: Objectives
Section 2: Executive Summary
Section 3: Understanding the Business Problem

Section 4: Assumptions, Data Cleaning &
Segmentation
Section 5: Control Group Selection
Section 6: Control Pool Selection

Section 7: Control Pool Representation
Section 8: Campaign Impact Measurement
Section 9: Further Insights
Section 10: Annexure
Objectives
What we were asked to do?
- SELECTION OF A CONTROL GROUP
- SELECTION OF CONTROL POOL
- IMPACT MEASUREMENT OF THE CAMPAIGN-DIGITAL MARKETING.
- FUTURE SCOPE
Copyright 2016 Axtria and/or its affiliates. All rights reserved. | Axtria Confidential Internal/Restricted/Highly
Restricted
Agenda

Segmentation

Executive Summary
A quick glance on the data provided
TEST Group Data
Unique Physicians
829
MYRBETRIQ Rx Data for
months
Dec14 to Nov15
MYRBETRIQ Detailing Data for
months
Dec14 to Nov15
MYRBETRIQ Sampling Data for
months
Dec14 to Nov15
Market Brands Rx data for
months
Dec14 to Nov15
Physician Identifier- IMS ID
Other identifiers available:
Oabdecile, STATE, Target Flag,
Specialty Group
Test Group
Data
Control Group Data

Unique Physicians
926
MYRBETRIQ Rx Data for
months
Dec14 to Nov15
MYRBETRIQ Detailing Data for
months
Dec14 to Nov15
MYRBETRIQ Sampling Data for
months
Dec14 to Nov15
Market Brands Rx data for
months
Dec14 to Nov15
Physician Identifier- IMS ID
Other identifiers available:
Oabdecile, STATE, Target Flag,
Specialty Group
Control Group
Data
Restricted
Executive Summary
A quick glance on the Analysis done
Rx_Myrbetriq
800
Post-Campaign
Pre-Campaign
700
718.00
600
683.00
585.00
563.00
582.00
499.00
500
462.00
Rx
406.00
492.00
400
Test Post
Campaign
589.00
397.00
438.00
459.00
502.00
470.00
529.00
447.00
523.00
2786
457.00
386.00
429.00
453.00
IMPACT MEASUREMENT
Test Pre
2927
Campaign
446.00
398.00
A= Test Post- Pre Campaign

Control Post
Control Pre
Campaign
3199
Campaign
141
3001
B=Control Post-Pre Campaign
198
IMPACT=A-B
-57
300
200
100
0
Dec'14
Jan'15
Feb'15
Mar'15
Apr'15
May'15
Jun'15
Jul'15
Aug'15
Sep'15
Oct'15
Nov'15
Month
test
c3
Restricted
Agenda

Segmentation

Understanding the Business Problem

What we understood about the given problem
Myrbetriq wants to know the effect of a digital marketing campaign for its drug.
To study the effect, a group of physicians is targeted for this campaign.
Test group is exposed to the campaign. Besides the effect of campaign, there should not
be any difference in the behavior of test and control group.
Out of given control groups, the control group which replicates the behavior of the test
group during the pre-campaign period, most closely, needs to be identified.
Impact of the campaign is to be calculated by taking the selected control group as the
base.
Copyright 2016 Axtria and/or its affiliates. All rights reserved. | Axtria Confidential Internal/Restricted/Highly Restricted
Agenda

Segmentation

Assumptions ,Data Cleaning & Segmentation

Considered for our analysis
ASSUMPTIONS
DATA CLEANING
The campaign begins after the

end of month of May15 and
ends before start of June16.
Replacing Null values in

Detailing and Sampling data
with 0
There is no lag in effect of the

campaign, i.e. the effect of the
campaign is instantaneous.
Removing physician level data

having Myr_Rx data as NULL
Carry-over Effect is neglected.
Removing outlier data for Rx

sales for each test and control
group
SEGMENTATION
Categorizing Decile into 3
categories-
H: 8-10,
M: 4-7 and
L: 1-3
Categorizing the States into 4
regions:
Midwest
West
North-East
South
Unique ID using Decile, Specialty
Group and Region.
Restricted
10
Agenda

Segmentation

Control Group Selection

Amongst the 3 given control groups, the control group most closely representing test group is selected
On the basis of categorical variables i.e. Specialty Group and Region, we

study the writing practice of physicians in test group and compare it with
those in control groups
For each Categorical Variable, we calculate the average Rx per physician
for both the control group and test group
The control group with minimum Euclidean distance or minimum error
variation is selected
*Source: Sales, Calls and Segmentation data provided AI
Restricted
12
Continued
Average_Rx_Market
Average_Rx_Myrbetriq
60
20
45.71
43.62
41.76
40
10
16.52
13.74
12.54
7.34
20.37
18.43
14.23
8.24
6.31
7.37
0
obg
5.00
6.92
7.59
6.96
0.75
other
pcp
urology
15.97
14.42352941
14.09
14
30
20
17.44
16
Average_Rx
Average_Rx
18
48.49
50
12
10
8 5.17
6
5.54
3.48
7.12
5.40
5.50
4.28
2.49
2.11
2
2.05
0
obg
other
Specialty Group
test
c1
c2
1.27
0.86
1.95 1.93
0.75
pcp
1.43
0.82
urology
Specialty Group
c3
test
c1
c2
c3
From the above graphs and by calculating Euclidean's Distance , we conclude that Control Group 3 is closest to test on the
basis of specialty group.
Refer Annexure for Euclidean Distance Calculation.
Restricted
13
Continued
Average_Rx_Market
25
6.08
19.08
18.62
18.21
15
14.95
14.42
15.43
13.87
10
9.69
10.79
5.93
5.81
16.39
15.87
11.30
10.68
10.00
10.07
7.89
Average_Rx
Average_Rx
20
5.36
5.23
4.34
4.52
4
3.56
4.034.08
3.80
3.73
3.14
3.12
2.97
2.35
5
1
0
0
SOUTH
MID WEST
NORTH EAST
WEST
SOUTH
MID WEST
Region
TEST
C1
NORTH EAST
WEST
Region
C2
C3
TEST
C1
C2
C3
From the above graphs and by calculating Euclidean's Distance , we conclude that Control Group 3
is closest to test on the basis of Region
For Euclideans Distance Calculation, refer to annexure
Restricted
14
Agenda

Segmentation

Control Pool Selection

From the selected control group, we select the subset or pool which closely represents the test group
Each Physician from Test is mapped on to physicians from

control group 3 on the basis of UID
This results in a one to many mapping.
The following parameters are considered for error
variation method:
Last 6 months Rx for Market, Last 3 month Rx for Market,
Last month Rx for Market,
Last 6 months Rx for Myrbetriq, Last 3 month Rx for
Myrbetriq, Last month Rx for Myrbetriq,
Last 6 months Detailing for Myrbetriq, Last 6 months
Sampling for Myrbetriq
After carrying out error variation method, we are left with
only one mapped physician in control group for one
physician in test group
We then apply cut-off on the average error to filter our
control pool
S.No Average_Error Conrol Pool Size

1
35%
374
45%
525
No Cut-Off
814
Restricted
16
Control Pool Selection (Continued)

Control Pool Selected is mapped on Test Group to see the closeness in pre-campaign period at various error cut-off
ERROR CUT-OFF at 35%

1
0.9
Post-Campaign
Pre-Campaign
0.88
0.8
0.84
0.72
0.69
Average_Rx
0.7
0.61
0.57
0.6
0.50
0.60
0.5
0.4
0.71
0.49
0.54
0.56
Feb'15
Mar'15
0.56
0.62
0.58
0.65
0.55
0.64
0.72
0.56
0.47
0.53
0.49
0.55
0.3
0.2
0.1
0
Dec'14
Jan'15
Apr'15
May'15
Jun'15
Jul'15
Aug'15
Sep'15
Oct'15
Nov'15
At 35% Error Cut-off, the

pool size is about 374
physicians.
On the Basis of these
physicians, the control
pool is mapped on to the
test group
We can see that, Rx in
control pool closely
represents the test group
in the pre-campaign period
Month
ERROR CUT-OFF= 35%

test
c3
Restricted
17

ERROR CUT-OFF at 45%

0.7
0.63
0.49
0.5
0.45
0.46
0.41
Average_Rx
Post-Campaign
Pre-Campaign
0.6
0.43
0.58
0.53
0.59
0.47
0.49
0.48
0.47
0.48
0.4
0.39
0.45
0.46
0.40
0.37
0.35
0.2
0.1
0
Dec'14
Jan'15
Feb'15
Mar'15
Apr'15
May'15
Jun'15
Jul'15
Aug'15
Sep'15
Oct'15
Month
ERROR CUT-OFF= 45%

test
0.50
0.41
0.39
0.3
0.63
Nov'15
At 45% Error Cut-off, the pool

size is about 525 physicians.
physicians, the control pool is
mapped on to the test group
We can see that, Rx in control
pool closely represents the
test group in the precampaign period but the gap
between test and control
group is more pronounced in
45% error cut-off
c3
Restricted
18

No ERROR CUT-OFF
0.6
Post-Campaign
Pre-Campaign
0.5
0.41
0.40
0.4
Average_Rx
0.41
0.45
0.36
0.32
0.33
0.28
0.29
Dec'14
Jan'15
0.37
0.35
0.34
0.38
0.3
0.47
0.45
0.44
0.49
0.30
0.34
0.31
0.35
0.36
0.31
0.30
0.2
0.1
0
Feb'15
Mar'15
Apr'15
May'15
Jun'15
Month
test
Jul'15
Aug'15
Sep'15
Oct'15
Nov'15
At No Error Cut-off, the pool

size is about 814 physicians.
physicians, the control pool is
mapped on to the test group
We can see that, Rx in control
pool closely represents the
test group in the precampaign period but the gap
between test and control
group is maximum in no error
cut-off
c3
Restricted
19

Control Pool Selected is mapped on Test Group to see the closeness in pre-campaign period in sampling and detailing
Average Sampling & Detailing

6 Months Sampling & Detailing
5
4.54
4.33
4.5
4
Average
3.5
3
2.5
2
The histograms show the close

mapping of control group on test
group in term of average detailing
and sampling.
Similar Detailing and sampling in
control and test
1.58
1.54
1.5
1
0.5
0
Sampling
Detailing
Test
C3
Restricted
20
Agenda

Segmentation

Control Pool Representation

The test pool selected should be representative of the entire test target
Specialty
Target %
Test %
OBG
3%
1%
UROLOGY
9%
6%
87%
92%
1%
1%
PCP
OTHERS
Region
MID WEST
NORTH EAST
SOUTH
WEST
Target %
28%
14%
41%
17%
Test %
27%
15%
42%
15%
We check that the selected test pool is

representative of the entire test target
on the basis of categorical variables
On the basis of Specialty Group, the test
pool is close enough representative of
the test group
On the basis of Regionthe test pool is
close enough representative of the test
group
Restricted
22
Agenda

Segmentation

Campaign Impact Measurement

Calculation of the effect of campaign on the Rx of Myrbtriq
Rx_Myrbetriq
800
Post-Campaign
Pre-Campaign
700
Test Post
Campaign
718.00
600
683.00
585.00
563.00
462.00
Rx
406.00
492.00
400
457.00
3001
446.00
B=Control Post-Pre Campaign
198
IMPACT=A-B
-57
589.00
500
397.00
438.00
459.00
502.00
470.00
529.00
447.00
523.00
386.00
429.00
453.00
2786
A= Test Post- Pre Campaign

Control Post
Control Pre
Campaign
3199
Campaign
582.00
499.00
IMPACT MEASUREMENT
Test Pre
2927
Campaign
141
398.00
300
200
PRE Campaign Post Campaign

Market Share
%
%
100
0
Dec'14
Jan'15
Feb'15
Mar'15
Apr'15
May'15
Jun'15
Month
test
c3
Jul'15
Aug'15
Sep'15
Oct'15
Nov'15
Test
26
27.6
Control
31.6
31.2
Restricted
24
Agenda

Segmentation

Further Insights
What more could have been done?
The Null values could have been replaced by its decile averages.
Instead of choosing from 3 groups, an entire new group can be created from
the combined pool of 3 which will have better matching characteristics
The deciling can be done on the basis of detailing and sample drops and take
them as categorical variable.
During the error calculation process, weights can be assigned to different
parameters.
During error calculation, small changes resulting in large error can be avoided
26
Agenda

Segmentation

Annexure
Writing on Basis of Specialty Group
Market
obg
urology
pcp
other
Euclidean
Distance
test
7.344828
41.76056
6.955556
0.75
2.054054
c1
12.53968
48.49333
7.588435
20.37209
6.918033
c2
16.51852
43.61875
8.239796
18.42857
5
c3
13.74074
45.71176
7.365108
14.22642
6.311475
21.94088 20.26004 16.01333
Myrbetriq
test
c1
c2
c3
obg
3.482759 5.396825 5.537037 5.166667
urology 17.43662 15.97333 14.09375 14.42353
pcp
2.111111 1.926871 2.494898 1.947842
other
0.75
7.116279
5.5
4.283019
1.27027 1.42623 0.857143 0.819672
Euclidean
Distance
6.811214 6.186704 4.962477
28
Annexure
Writing on Basis of Region
Myrbetriq
SOUTH
TEST
C1
C2
C3
Market
3.564955 5.81372549 6.083333
5.929293
MID WEST 3.798283 4.34351145 4.082609

NORTH
5.35877862
EAST
3.72807
6
5.229412
3.12135922
WEST
2.347826
3
2.974359
4.029536
Euclidean
Distance
2.93458812
9
3.011585
4.519774
3.142077
2.627014
TEST
C1
18.620915
SOUTH
9.694864
03
14.419847
MID WEST 10.7897
33
NORTH
16.389312
EAST
10
98
10.684466
WEST
7.891304
02
Euclidean
11.894428
Distance
0
58
C2
C3
19.08333
18.20539
13.86957
14.94515
15.87059
15.42938
10.06667
11.29508
11.69722
11.43506
29

Case Study Test&amp;Control

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Case Study Test&amp;Control

Enviado por

Direitos autorais:

Formatos disponíveis

Case Study Test & Control

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

- SELECTION OF A CONTROL GROUP

- SELECTION OF CONTROL POOL

- IMPACT MEASUREMENT OF THE CAMPAIGN-DIGITAL MARKETING.

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Control Group Data

A= Test Post- Pre Campaign

B=Control Post-Pre Campaign

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Understanding the Business Problem

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Assumptions ,Data Cleaning & Segmentation

The campaign begins after the

Replacing Null values in

There is no lag in effect of the

Removing physician level data

Carry-over Effect is neglected.

Removing outlier data for Rx

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Control Group Selection

On the basis of categorical variables i.e. Specialty Group and Region, we

*Source: Sales, Calls and Segmentation data provided AI

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Control Pool Selection

Each Physician from Test is mapped on to physicians from

S.No Average_Error Conrol Pool Size

Control Pool Selection (Continued)

ERROR CUT-OFF at 35%

At 35% Error Cut-off, the

ERROR CUT-OFF= 35%

Control Pool Selection (Continued)

ERROR CUT-OFF at 45%

ERROR CUT-OFF= 45%

At 45% Error Cut-off, the pool

Control Pool Selection (Continued)

At No Error Cut-off, the pool

Control Pool Selection (Continued)

Average Sampling & Detailing

The histograms show the close

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Control Pool Representation

We check that the selected test pool is

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Campaign Impact Measurement

B=Control Post-Pre Campaign

A= Test Post- Pre Campaign

PRE Campaign Post Campaign

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

Section 3: Understanding the Business Problem

Section 6: Control Pool Selection

21.94088 20.26004 16.01333

6.811214 6.186704 4.962477

3.564955 5.81372549 6.083333

MID WEST 3.798283 4.34351145 4.082609

Case Study Test&Control

Case Study Test&Control