randomized control trial
WaterSmart reports statistically significant program results by running a Randomized Control Trial (RCT) to compare changes in water usage between treatment and control groups. The control group is a randomly selected subset of the total utility population, and its members do not receive Home Water Reports or access to the Customer Portal. The analysis is performed with a robust fixed-effects regression model.
REGRESSION ANALYSIS
After the control group has been selected, the next step in the analysis involves a fixed-effects (FE) regression model. The FE model utilizes the billing data for all customers in the both the treatment and control groups, while incorporating fixed effects controls for each household and time period. The model specification is Log-Linear, which improves model accuracy and allows for percentage savings coefficients to be calculated with a simple mathematical transformation.
WATER SAVINGS CALCULATIONS
The model produces discrete period savings coefficients for every month in the evaluation period. Savings are then applied using the control group mean GPD, as well as the percentage savings for that period. Results are available through the Utility Analytics Dashboard in both percentage terms and total acre feet at both the individual and cumulative levels for the entire treatment group.
PROPENSITY MATCHING
To perform savings analyses in the absence of a randomized control trial, a statistical technique called propensity matching is applied. The propensity match uses monthly usage to compare the members of the treatment group to potential controls who have not received any Home Water Reports. The treatment members are then matched 1-to-1 with a member from the control group pool based on their monthly usage in the pre-program period.