New survey system a success

Three months ago I changed the survey system to greatly reduce the number of emails to Car Reliability Survey participants. And I’ve been very worried about this change ever since. With fewer emails might we end up with fewer responses? Or fewer repairs being reported, and so lower quality data? These are the questions that keep me awake at night.

Well, the quarter is over, so we have our answer. The percentage of people reporting repair trips in a given month has remained around 10 percent, so the quality of the data has not been affected. The quantity, on the other hand…

For over a year the response rate has been about 36 percent, so this has been our target. Last July the response rate was 36.7 percent, and in October it was 35.7 percent, a bit short.

Last month it was 38.0 percent, a level we haven’t seen since April 2008. Among new members, the response rate was a huge 56.0 percent. Even among members who joined over three years ago–many of these email addresses are probably no longer being checked regularly–the response rate was 28.3 percent. In absolute terms, our goal last month was 14,650 responses. We ended up with 15,424.

The current results include 295 models, plus partial results for another 290. It looks like the February update will include 356 models, plus partial results for another 287. Thanks to the higher response rate, well over half of our stats will now be full results.

So now I can worry about other things…