When TrueDelta first started posting results a year ago, we posted an “official result” with a minimum of 18 responses that included 72 months of data. Asterisked results were provided with at least ten responses that included 40 months. These numbers were chosen based on where the results seemed to stabilize.

In subsequent result sets the minimums were increased because more is always better, more people were participating as the panel grew, and both initial minimums seemed a bit low. In August the minimums were 25 responses / 75 months for official results, and 15 responses / 45 months for asterisked results.

The problem with this change is that TrueDelta starts collecting data when 25 people sign up, and the response rate varies quite a bit from model to model. An unfortunate consequence is that some members have been responding to the surveys for a year or more, but no result has been posted for their car. Their fellow owners have let them down by not responding.

The November results will do something about this. I have noticed that the number of months of data is at least as important as the number of responses for providing a reliable result. So, starting with the new results, there will be an alternate minimum for asterisked results of 10 responses that include at least 60 months of data.

With the average response rate around 40 percent, this should include most models where members have been consistently responding for at least six months. Based on responses so far, this will add about 15 additional models to the results page, for a total near 140. I believe that these new results are at least as reliable as those posted under the primary minimum.

Of course, both the 15/45 and the 10/60 results are asterisked. When trying to gauge the precision of TrueDelta’s results, it’s important to look at both the number of responses and the number of months per respondent. With both, more is better. There are now ten models with at least 50 responses, and some with over 500 months of data; these are the most solid of all.