TrueDelta

When TrueDelta first started posting results a year ago, we posted an “official result” with a minimum of 18 responses that included 72 months of data. Asterisked results were provided with at least ten responses that included 40 months. These numbers were chosen based on where the results seemed to stabilize.

In subsequent result sets the minimums were increased because more is always better, more people were participating as the panel grew, and both initial minimums seemed a bit low. In August the minimums were 25 responses / 75 months for official results, and 15 responses / 45 months for asterisked results.

The problem with this change is that TrueDelta starts collecting data when 25 people sign up, and the response rate varies quite a bit from model to model. An unfortunate consequence is that some members have been responding to the surveys for a year or more, but no result has been posted for their car. Their fellow owners have let them down by not responding.

The November results will do something about this. I have noticed that the number of months of data is at least as important as the number of responses for providing a reliable result. So, starting with the new results, there will be an alternate minimum for asterisked results of 10 responses that include at least 60 months of data.

With the average response rate around 40 percent, this should include most models where members have been consistently responding for at least six months. Based on responses so far, this will add about 15 additional models to the results page, for a total near 140. I believe that these new results are at least as reliable as those posted under the primary minimum.

Of course, both the 15/45 and the 10/60 results are asterisked. When trying to gauge the precision of TrueDelta’s results, it’s important to look at both the number of responses and the number of months per respondent. With both, more is better. There are now ten models with at least 50 responses, and some with over 500 months of data; these are the most solid of all.

One of the major advantages of TrueDelta’s research process is that, if enough owners participate, we can have initial results for new models very quickly.

The best proof of this so far: the 2008 Buick Enclave (and the related GMC Acadia and Saturn Outlook). Thanks to the help of EnclaveForum.net, with assists from AcadiaForum.net, Saturnfans.com,  and SaturnOutlookForum.net, we’ve already received over 40 responses from owners of these vehicles. Panel members have already seen preliminary results, and the general public will see official results in early November. The Enclave went on sale in late April, so this was a six-month turnaround.

These forums deserve our thanks for helping to demonstrate that there is a better way to conduct vehicle reliability research. J.D. Power won’t have reliability information on the Buick Enclave until the middle of next year. Consumer Reports won’t have it until a year from now.

In past years, Consumer Reports included spring intro models in its results as long as 100 or more survey responses were received. Problem was, many of these surveys were returned after just a few weeks or even a few days of ownership, leading to inaccurate results. I pointed this out to them last year. Perhaps in response, this year Consumer Reports’ results appear to include no early 2008s.  

My focus is now shifting to the February results, with a high priority on maximizing the number of 2008 models in those results. This will be tough, because in some cases the turnaround will have to be about five months, a month quicker than with the Enclave. But it is possible.

The models with the most 2008s so far:

Buick Enclave – 73 (includes sibs)

Nissan Altima – 18

Ford Escape – 16 (but spring intro, so not so likely)

BMW 5-Series – 15 (ditto)

Infiniti G-Series – 13

Mercedes-Benz C-Class – 13

Scion xB – 10

Cadillac CTS – 9

Nissan Versa – 9

Honda Accord – 8

Mazda CX-9 – 8

All of these are unlikely to make it into the 2008 results. But a few should. You can help. If you know anyone with one of these models, please tell them about TrueDelta’s research.

Back when the site was being redesigned, the designer I hired included the Acura TL, Honda Odyssey, and Kia Sportage in the header image because use of the images was not restricted and these were among the most researched vehicles on the site at the time. More than a few people have wondered why all of the vehicles are Asian; this is the answer.

He has offered to change the photos, and I think I’ll take him up on that offer. But which vehicles to use? How about the three 2008 models with the largest sample sizes in February. The Buick Enclave likely has a lock on one spot, but the other two remain open.

In the past, results to the Vehicle Reliability Study have first been posted a couple weeks after the end of data collection, or about 45 days after the end of the quarter.

This month I’ve decided to try something different. Most people who respond to the survey do so in the first week of data collection. So this time I started cleaning the data after the second follow-up. A few hundred more people will still respond before the end of the month, so the results could change. But the results at this point should generally be close to the final results.

So, on a trial basis, I’ve gone ahead and posted preliminary results. They will only be visible to people who’ve already responded to the surveys, to avoid potentially biasing the final results. They are not to be shared with anyone else–wait for the official results to do that.

All along I’ve only finalized the official results about a month after their release, to include any late additional responses or corrections.

So, with the new preliminary results, we’ll have three stages:

1. Preliminary results–October for this set

2. Official results–November

3. Final results–December

Then repeat

So, new numbers every month, and way ahead of anyone else’s results. Enjoy!

It dawned on me while traveling this past weekend that the site could be greatly improved, and many current problems eliminated, if the Vehicle Reliability Survey and Vehicle Repair History Survey were much more tightly integrated. Below you’ll find my initial thoughts. Feedback would be appreciated.

Why haven’t these surveys been integrated all along? It’s a matter of history. The Vehicle Reliability Survey came first, and initially included no place to describe the repair. Nearly a year later the Vehicle Repair History Survey was added to do things the Vehicle Reliability Survey could not: include models below the minimum sample size, include repairs that occurred before someone joined, and include repair descriptions and post them to the site. Soon afterwards, the Vehicle Reliability Survey was revised to include a description, but to aid in error checking.

Problem is, while it is possible to participate in both surveys, the effort involved is needlessly redundant. And I fear that many people think that the Vehicle Repair History Survey is the main survey, and so don’t respond to the Vehicle Relibility Survey. Many others find the distinction confusing.

Much more closely integrating the two surveys will reduce or even eliminate these problems, while providing more informatioin to site visitors. I wish I’d thought of it sooner, but better late than never.

What will this integration entail?

First, the descriptions submitted with the Vehicle Reliability Survey will be posted along with the results to the Vehicle Repair History Survey. So people participating in the former will no longer have to enter their repairs a second time in the latter. If an entry is still made in the Vehicle Repair History Survey, it will override any entry for the same month in the Vehicle Reliability Survey. This will allow members to add additional detail or fix typos.

Second, members whose vehicles are not yet eligible for the Vehicle Reliability Survey will be much more strongly encouraged to participate in the Vehicle Repair History Survey, and this might be required of new members. A link to the survey will be provided in every third monthly update, with the expectation that the record for a vehicle be updated quarterly.

Some Vehicle Reliability Survey participants might not want their repairs posted to the site, even though there will be no way to identify the owners of the cars. It will be possible to opt out of this, but people who do not want their repairs posted to the site will also not be able to view those posted by others. Think of it as the vehicle reliability version of “I’ll show you mine if you show me yours.”

These changes will require many changes to the various programs involved, so they’ll probably take a month or two to fully implement. No doubt further wrinkles will be added along the way. I look forward to members’ suggestions. As the integration progresses, further details will be posted here.

In the end, we’ll have a much more useful, less confusing site. 

TrueDelta.com turned three at the end of September, and the third year was a good one.

We started providing Vehicle Reliability Survey results, and provided updates each quarter, as promised. Each time more models have been included, despite increases in the minimum sample sizes. In a growing number of cases TrueDelta has provided reliability information on a model before anyone else. Next month: the 2008 Buick Enclave. Over the course of the year, we’ve continued to refine the pricing and reliability information, most recently adding about 1,250 static price comparison pages. We also started this blog.

Site visits grew from 16,300 in September 2006 to 44,200 in September 2007–and might top 60,000 this month. (We’re having an extremely good October.) The number of vehicles in the panel grew from 7,000 to nearly 19,000–and will reach 20,000 before the end of October.

All of this said, there have been disappointments. There has been some press coverage of what we’re doing, including a page three article in industry bible Automotive News, but in general the press has ignored the site. Some members have been extremely helpful in telling others about the site, but in general word of mouth is weaker than it ought to be.

My largest disappointment: too many people sign up and then neglect to respond to the surveys. This is unfair to everyone who does participate, and at some point in the next few months changes will have to be made. I’m finding it hard to be a “glass 40 percent full” guy. Beyond fairness, the system can only work if those who want information help provide it.

So, what will the next year bring? First, the change I’ll cover in the next blog post: integration of the Vehicle Reliability Survey and the Vehicle Repair History Survey. Early next year we hope to add databases for the 2000 and 2001 model years, so that these can be included in the latter survey. We’ve also had another whole section planned, but won’t have time to implement it for another month or two.

Finally, we’ve started talking to potential partners. One thing is not on the table: TrueDelta will not be sold. I can’t imagine that anyone working for a salary, including myself, would operate this research the way it needs to be operated. And once a large corporation gets involved, the pressure for quick results would increase, while the speed at which improvements could be made would decrease. Not a good combination.

But some form of partnership could be very helpful. A partner might help inform more car owners about our research, and help provide the resulting information to more car buyers. Any such partnership, like just about everything else we do, will have as its primary goal better information on more cars.

Thank you for helping us provide better information on more cars during the past year. Now on to an even better fourth year.

When Toyota’s current reliability issues made headlines this past week, many people were quick to assume that manufacturing the cars in the U.S. was the source of the problem.

I’d avoid such a rush to judgment. I’ve noticed that Acura and Honda owners are especially likely to assume that Japanese-made Acuras and Hondas are much better built than American-made Acuras and Hondas. And yet the Japanese-made Acura RL is the least reliable car in either line, according to Consumer Reports.

In Toyota’s case, most Camry V6s were made in the U.S. But the problem is with the transmission, which was engineered and manufactured in Japan. And the Lexus GS, also recently singled out, is entirely manufactured in Japan.

Many people want clear, simple rules such as “American bad, Japanese good” on which to base their decisions. But reliability cannot be inferred from the country of manufacture or even the brand. As Consumer Reports now realizes in the case of Toyota (if not yet with Honda and Subaru), it’s necessary to go down to at least the model level when estimating how reliable a car is likely to be.

Each year Consumer Reports posts a list of models whose reliability has declined. But in some cases this “declining reliability” is a sign that they really didn’t have enough data a year ago. Many times with models that were introduced in the winter or spring they report a “better than average” result, only to report a year later that reliability has “declined.”

This affected the GMT900s this year. It affected the Honda Ridgeline last year. And, based on TrueDelta’s data, it’ll affect the 2007 Acadia and Outlook next year.

The source of the problem is that most of their surveys are returned in April, and at that time many of these late intro models have only been owned for a few weeks–or less. People haven’t had enough time to experience problems.

TrueDelta sees a similar effect in its own results when a model is very new. When the reported trips per year is under 20 per 100 vehicles, and the average odometer reading is under 3,000, there’s a very good chance that the reported repair rate is artificially low.

This is why in the notes to the August results you’ll find:

Any result under 20 repair trips per 100 vehicles is likely a result of the small sample size or cars too new to have experienced many problems, and should be expected to increase to at least 20 in future updates.

So, when looking at TrueDelta’s results, pay attention to the average odometer reading and the average number of months per vehicle. When these are very low, there’s a fair chance that future reliability will “decline.” Not because the cars have actually gotten worse, but because earlier owners had not yet had their cars long enough to experience problems.

For years, Consumer Reports has recommended new Honda and Toyota models even without reliability data on them because cars from these manufacturers had been consistently reliable.

However, this has changed in recent years as Honda’s and Toyota’s have become less consistent and the gap between all makes has narrowed. So to me the practice seemed questionable.

A couple months ago TrueDelta reported that the repair rates for the Camry V6 were running around double the average, easily high enough for a black dot from CR. Recently I began to wonder what impact this might have on CR’s recommendations. Not recommend a Camry? That’s unheard of.

Well, CR’s ratings are out, and the Camry V6 not only got the black dot TrueDelta predicted, but CR has announced that because of this and other problematic Toyotas the manufacturer’s products will no longer be recommended before their reliability is known.

Honda still gets this boost, but it’s just a matter of time before the practice is dropped altogether.

Over on Bimmerforums.com, someone recently asked, “If my vehicle had problems, how would it benefit me to report it? It would only cause my resale to drop later if I had problems.”

No doubt this question has crossed many people’s minds. Seems logical.

But there is a major flaw in the logic: TrueDelta isn’t the only one reporting vehicle reliability information. And others only provide vague dots, so people often imagine that “worse than average” models are really bad.

Truth is, such cars often aren’t all that much worse than the average car. Only TrueDelta makes this obvious by posting actual repair rates. By viewing our results, people might learn that the car others recommend against doesn’t require nearly as many repairs as they imagined.

So participating in TrueDelta’s research might actually improve rather than harm resale values.

Of course, for this to happen the results must be credible, and generally in line with those from other sources. So, somewhat paradoxically, you’re likely to help both yourself and others the most if you participate in TrueDelta’s research and accurately report repairs.

I’ve written before about what I call “hidden price increases,” where the manufacturer raises the invoice price paid by the dealer but leaves the sticker price that the public sees unchanged. Since the invoice price often determines the actual price you pay more than the sticker price does, car buyers end up paying more, but without realizing that anything has changed.

Audi recently applied such an increase across all of its models, to both base prices and options. For example, on an A6 4.2 sedan that lists for $56,300 (plus $775 destination), the invoice price went up from $51,861 to $52,361, an increase of $500. With options, invoice is now 93 percent of MSRP. It used to be 91 percent. So on a car with $5,000 in options the invoice price will now be another $100 higher, for a total increase of $600, even though the sticker remains the same.

It’s not hard to figure out why Audi is doing this: with the Euro worth more than $1.40, they may well be losing money in the United States lately. At a minimum they’re earning far less than they were when the dollar wasn’t so weak.

Why not just raise the sticker price? Probably because no one else is. The U.S. market remains high competitive, and this combined with the weakening dollar has made car prices much cheaper here than pretty much anywhere else in the world.