In designing the Car Reliability Survey and the emails that support it, I’ve had to balance many factors, among them the human memory (or lack thereof), clarity, and the number of emails.

The survey has been on a rigid cadence, always covering the previous month. Emails for it went out every month, so anyone with a repair to report could report it while their memory was still fresh. But to keep the load down, if there was no repair to report a response was only needed following the end of each quarter. To make sure any repairs for the month were reported, reminders were sent to those who hadn’t recently responded.

This system worked well in many ways. The response rate has been far higher than for other surveys, and the data collected have been of very high quality.

Still, there were problems. The number of emails both made it unclear when a response was needed and clogged many inboxes. Also, some people did not want to wait as long as a month to report a repair. They wanted to report in real time, as many times a month as this required.

Starting last month, we eliminated reminder emails for anyone who responded in October. This did reduce the number of reported repairs by about 20 percent–many people need reminders.

To counteract this, the survey has been upgraded. Starting this month, it is possible to report any repair that occurred during the quarter, not just those that occurred the previous month. As a result, this month’s email will serve as a reminder email for October repairs as well as the first email for November repairs. This will probably make up for much of last month’s shortfall.

In addition, it is now also possible to report repairs that occurred during December, this month, and to respond multiple times per month. So people who want to report each repair immediately can now do so.

Much of last month’s shortfall probably involved repairs that occurred in November. With the old reminder emails, sent in the middle and at the end of the month, some people reported repairs that occurred during the current month even though the survey asked only about the previous month. The new system will eliminate this minor distortion.

Going forward, multiple reminders will only be sent to participants following the end of each quarter, the next time in January. So the third month in each quarterly cycle will serve as a sweeper to gather any data that was missed during the previous two months. By concentrating reminders in the third month, it will be clearer when a response is needed even if there have been no repairs.

A further by-product of the multi-month survey: it is now possible to close out sold vehicles entirely on the site itself, as long as there has been a recent response.

Taking all of the changes together, I expect to gather data of at least as high a quality and quantity, with fewer emails and less effort all around.