chsu74

Will we be able to say the type of oil we use in this survey?

Wellardmac

Bob,
It seems to me that you've already made up your mind what the result should be. I'm sure that your opinion is equally valid as the data that the OP would collect because both are just that - opinion. He is collecting unstructured and statistically unsound data that is meaningless and misleading.

As a moderator you are using your clout to try and sway opinion in favor of a grossly flawed survey - respectfully, that is a disservice to this membership.

Dell and I are not doomsayers, we are two professionals qualified in statistical analysis that are stating that this survey is flawed. The OP is choosing to ignore our professional opinion. If we're counting the number and size of Ph.Ds, then I'd say that Dell and I add up to two to one against and I'm sure that the other Ph.Ds in the discussion are equally appalled by the methodology in play here.

jasper

I just tried to do the survey to see for myself if the questions asked were designed well enough to characterize my car.

** the engine was replaced in 2005, four years before I bought it. I don't know why the motor broke. How will this be reflected in the survey?**

Anyway - the email I received said the 2002 911 is not yet included in the reliability survey.

So...

Bob Rouleau

Wellardmac - we have a lack of understanding here. The fact is the 996 is seen as a risky buy. The resale prices prove that. You and Dell maintain that the car is reliable. I agree with you

I see this as an opportunity to collect positive opinions which might dispel the myth of 996 unreliability. In other words the same thing you and Dell have been trying to do. So why are we seemingly in opposition?

BruceP

Bob, somehow I've failed to make myself understood, here. I thought I was clear that this was a slightly rankling issue about transparency. I don't appreciate having my arguments reframed for me, but such is the Internet sometimes. A subject, I should add, that I've got pretty good bona fides for commenting on.

I'll pass on the survey, given the weirdness. But I do look forward to citing this episode in an upcoming speech. It's a great little story about the criticality of imputed motive in social media marketing.

Wellardmac

Bob,
Dell and I are data driven people. It's all about the facts for us.

Our opinions may be in agreement with regard to reliability. The perception of a 996 being a risky buy may be wrong and our assessment that it is wrong may both be equally incorrect. How would we know if we do not have valid data to prove that point? Only Porsche knows the correct answer.

My personal experience is that the 996 is the most reliable car I've ever been exposed to, but my data might be wrong. The methodology of the OP is flawed, so that doesn't make his data/opinion any more valid.

Just because something produces an answer that you might like does not make it a valid result. Data and truth matters.

The point being that we might have an opinion and it is just that - everyone is entitled to an opinion - doesn't make it correct. We don't represent that opinion as fact. The OP will represent the limited data he has as being a fact and general for all 996s. That is deceptive.

Graufuchs

I agree with bruceP and wellardmac 100%.

mkaresh

By design the survey gathers information going forward, so that past experience cannot potentially bias the results. So that engine replacement won't be reportable on the main reliability survey. The analysis always looks at the past twelve months. So it won't report the lifetime repair frequency, only the frequency in the past year.

There is a second form for those who want to post earlier repairs, but this only fills out the posted repair history. Repairs entered through this form are not included in the analysis.

I do not ask which type of oil people use. The typical car owner probably doesn't even know.

The reliability survey form used when submitting a repair is here, for those who want to see it:

http://www.truedelta.com/survey.php

mkaresh

"Opinions" probably isn't the best word. "Experiences" would be a better way to think of it.

The survey is designed to keep opinions out of the data as much as possible. Instead, it asks factual questions, focusing on whether or not the car was repaired. Either it was, or it wasn't.

While I doubt engines are failing at a high rate, it remains likely that the overall repair frequency for these cars is about average, and perhaps worse. With the Boxster, the reported repair frequencies have been very close to the average:

Porsche Boxster / Cayman reliability comparisons

My own opinion certainly has nothing to do with it. I periodically ask people whether or not the car required a repair, and tabulate the results. Unlike some other reliability surveys, I do not ask people if they feel the car is reliable. That would be opinion, and any opinion survey without a random sample is prone to distorted results.

I have never claimed that this survey is perfect. No survey in the real world is. Luckily, flawed isn't inherently the same as useless, any more than a flawed car is useless. In this case, as in many cases, a flawed tool can be made to work quite well. In any field you'll find people who are sticklers for the rules, and who would rather have no product than an imperfect product, and others who work with what is available to get things done. I am actually something of a perfectionist myself, too much so in the view of people who know me, but stop at the point where something is still doable.

I will readily claim that this survey provides valid, useful information, as attested to by the results. Look at those for the Boxster. I find them vastly preferable to no stats at all--which does seem to be the preference of some people here--and those from CR and JD Power, which are the only real alternatives at the present time.

Saying that only Porsche can really know does not help anyone outside Porsche, and simply isn't true. This survey could provide very good information on the 996. The key question is whether we can get enough owners involved. The more owners get involved, the better the information will be.

When I started posting here 105 911 owners had signed up. After the first thread and this one, despite the harsh criticism, we're now at 129. Still far too few, but moving in the right direction.

Palmbeacher

In the healthcare professions we're inundated with statistics on everything. I'm convinced that most of the researchers bank on us being too busy to look beyond the "conclusions" paragraph and take a critical look at the methodology. otherwise a whole host of things jump out that lead to suspect corrupt data.

Voluntary surveys are perhaps the least reliable, because there is absolutely no way to filter the variables, and because they assume and rely upon the veracity of the participant, unlike a controlled study where the subjects are closely monitored. That's even before we discuss cohort size. I've seen medical studies extrapolating from double-digit patient sampling.

The problem we have, and will be a problem here also, is that people tend to glance over the methodology and jump straight to the conclusions. There is nothing "wrong" with the proposed survey IF whoever reads the results will interpret them taking into account the limits of the methodology. However that doesn't happen frequently enough even with trained professionals, let alone the lay public.

mkaresh

Medical studies are a different case. There are far more significant variables involved (genetic background, diet, exercise, substance abuse, other drugs, etc.), the interactions among these variables are much more numerous and complex, and the subject being reported on is often inherently much more subjective--i.e. do you feel better after taking the pill? Add all of this together, and you generally need two very large random samples, with one to serve as a control group.

I always strongly encourage people to look at the methodology behind any survey results, including my own.

First thing to check: what questions were asked? If you don't know what was asked, then you don't know how to interpret the answer.

Here's a challenge: find out what questions Consumer Reports and JD Power ask to get their stats. Both firms release very little about their methodology, and neither publicly posts its survey form.

If people did look into the methodology employed by those two firms, and especially by CR, I'd have a much easier time of growing this survey.

My own critique of CR's survey, which I find not only flawed but very seriously flawed:

Consumer Reports survey critique

They ask people to report problems they "considered serious"--opening the door wide for many respondent biases. Many people report rattles, many others only report a problem if it keeps the car from being driven.

But I'm not only a critic--I'm doing my best to develop an alternative.

jasper

I think that was meant to be a joke. Oil discussions are as hotly debated as bunk reliability surveys on this forum.

Holy crap - that puts a *HUGE* onus on the owner to consistently and concientiously complete the survey. I signed up for the survey, but there's no way even I would go to the website and fill this out every time I went into my garage to work on my car. That's problem 1.

Problem 2 is that I do my own maintenance which includes more maintenance and tuning than the "average" owner. This would make my car appear very unreliable since I spend dozens of hours fiddling while the opposite is actually true.

I'm no statitician but I don't see this working. The system relies entirely on the discipline of the people responding to the survey, and this is inherently unreliable.

The survey is structured to require high quality input but there is simply no control over the reliability of the data. Participation is voluntary and with a weak incentive.

Sorry to pee in your cornflakes.

I was going to stay out of this too

Shark Attack

here is my take. I dont mind you making money. But I do mind when you try to do it passivly like you are trying to do everyone a favor. taking advantage of a situation like a guy selling caskets to a morning family and trying up up sell on the silk bedding.

Thats my take on it

ivangene

they are NOT ticking time bombs - they are cars that have a small set of problems that CAN be catistrophic if not fixed prior to the due date on the bomb mechanism.....

you might weant to ask some of those people that do not frequent the boards, who have had thier cars explode, and who had them fixed out of thier own pocket how they feel about the ticking time bomb analogy... the motor I just helped rebuid was a TIP with ~30k, the owner had it for a month, and he stopped to get gas....after filling, turned the key and WHAM!! - $18k

this thing is too much to read - so I am not going to, but for those who knoe Jake Raby, keep in mind, he is very busy fixing problems that people refuse to admit exsist and want to argue with him about the exsistance of said issues.......

NO AFFILIATION

Torontoworker

So what was wrong with making it simple?

1. Has your IMS failed and blown up your motor? Y/N
2. Has your engine been replaced? Y/N
3. Has your RMS leaded? Y/N
4. Indicate 6 speed ___ or Tip ___
5. Indicate year ___
6. Indicate Model ___

Thats all I really need to know.