I wanted more up-to-date car reliability information that included actual repair rates. So in late 2005 I started getting people together to make this possible. TrueDelta now updates actual repair frequencies, not just dots, four times a year, to track cars closely as they age.

We've had excellent participation by Boxster owners, and have started to provide stats for that car. I'd like to do the same for the 911, so I'm grateful to rennlist for providing me with permission to post about the survey.

Participants simply report repairs the month after they occur on a one-page survey. When there are no repairs, they simply report an approximate odometer reading four times a year, at the end of each quarter.

To encourage participation, participants receive full access to all results, not just those for the 911, for free.

For the details, and to sign up to help out:

Car reliability research

 

Note: this survey does not randomly sample owners. Posters in a similar thread in the 996 forum have strongly criticized it for this reason.

I'm well aware of the risks involved with a non-random sample, and have carefully designed the research process to compensate. I've been pleasantly surprised by the quality of the data this survey has collected.

I invited the critics in the other thread to examine the results we have so far, but they had nothing to say about the results themselves, only the method used to get them.

Don't agree with what I'm doing? Then don't participate. I made getting results for Porsche models a priority recently because a number of Porsche owners asked me to, and conduct this survey for the many people who are interested.

 

The 2006 is now halfway to the minimum, and other model years are about one-third of the way.

With the Boxster we're seeing a definite improvement from the 986 to the 987. Looking forward to seeing whether the same is the case with the 911.

 

Enrolled...

 

Enrolled . . . '07 TT

 

Already enrolled. Just waiting for you to get enough to start. 09 C2

 

That's not quite true. What many people, myself included, said was that your results are irrelevant because your data is corrupt due to the flawed methodology. If more people took a critical look at test methodology, there would be far less snake-oil sold worldwide, and far less damage (physical and financial) as a result of people simply accepting conclusions of "surveys" without regard to the methodology.

Your survey fails not because it isn't random, but because you make no attempt to verify the data. You do not ask for proof that responders even own the car they are reporting, let alone proof that claimed failures/repairs actually happened. In short, your data is 100% hearsay, taken on faith alone that it is factual. Given the propensity for playing fast and loose with the truth, behind the cloak of anonymity on the internet, there is a real possibility that at least some unknown percentage of the data you collect is bogus, yet the conclusions drawn from that data will most likely be assumed to be accurate by people who read them. That can have a significant impact on the resale value of the vehicles, positive or negative. I strongly believe that you have an obligation to verify your data...which thus far you have denied based on a few weak arguments camouflaged with statisticians' jargon, the only one of which that rings true is that it would cost you too much money.

 

Great points Palmbeacher.
Basically, if someone had a grudge with a particular manufacturer, they could create as many accounts as they want and flood the true delta system with bad data to make a particular model look bad.
Again, no verification of the data or that the person even owns the car that they say.

 

Verification would cost far too much money, and it's a purely hypothetical problem. In the past four years I have come across absolutely no evidence that it is a real problem.

I run quarterly checks for duplicate cars. These tend to catch about a dozen cars, out of a total of 17,000 responses in the latest round. The reason for these appears to be user error, not any ill intentions.

If there were money involved, then perhaps this would be more of a problem. But there's no real payoff as is. So why take the time to muck up the work of thousands of other people?

A much larger problem than keeping people from participating with cars they don't have has been getting people to participate with the cars they do have. This one is quite real.

With the 997, the 2006 is now only ten cars short of the minimum, and the 2007 and 2009 are each 15 short.

Car reliability research

 

It's a nice try, but your methods of collecting this herein are inherently biased.

 

Yes, fact-checking costs money. But I'm sure we can all think of numerous examples where the failure to check facts has led to some serious consequences.

I'm not surprised you believe it's purely hypothetical, and have no evidence, when by your own admisson you don't require any.

Checking for duplicate VIN#s is good. However, given the tiny number of overall Porsche owners who respond, one could probably post numerous different VIN#s without duplicating that of another responder.

You don't think there is anything financially to be gained by anyone from survey results that "prove" a certain part or parts are prone to significant statistical failure? Or that cars over a certain age are significantly less reliable? (How many model years back does Porsche offer CPO? How many model years back do Porsche dealerships keep on their used lots vs wholesaling to indys?) Do not misunderstand, I'm not making accusations here, just attempting to show that there is significant financial motive for wanting the results of an internet survey to point a certain way.

 

I run a number of different checks on the data and personally review every single repair reported. I don't want to go into the details here for the same reason that banks don't publicly discuss their security procedures.

There is seriously no current problem in this area. But perhaps if we discuss it enough we can create one?

 

Flawed method..useless. Also, compared to what? You might find that your P car is "not reliable"..but Mercedes, Audi, and BMW are 10x worse....thus relative to the industry you have a very reliable car.
abe

 

What is the source of your information that the others are "10x worse"? What methods were used to gather this information?

 

Earlier you said you didn't verify data because it would cost too much. Which of your contradictory statements is the truth? And what do you mean by "review (every single repair reported")? Do you require the responder to provide documentation? Or do you call the repair shop and verify it? By what other means does your "review" satisfactorily authenticate the repair?

There's no need for you to divulge your top-secret security measures. All we need is to pose a question to anyone who has participated in the survey: have you been required to provide personal identification (driver license #, SS#, etc.), show proof of ownership (title, registration) of your car, or been asked to fax a copy of repair invoices?