Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.carno.net.au!harbinger.cc.monash.edu.au!munnari.OZ.AU!news.ecn.uoknor.edu!feed1.news.erols.com!news.maxwell.syr.edu!news.cis.ohio-state.edu!nntp.sei.cmu.edu!news.psc.edu!not-for-mail From: peterb@hoopoe.psc.edu (Peter Berger) Newsgroups: comp.unix.bsd.freebsd.misc,comp.unix.bsd.bsdi.misc,comp.sys.sgi.misc Subject: Re: no such thing as a "general user community" Date: 31 Mar 1997 19:28:22 -0500 Organization: Pittsburgh Supercomputing Center Lines: 119 Message-ID: <5hpkr6$32u@hoopoe.psc.edu> References: <331BB7DD.28EC@net5.net> <5hn00k$dio@fido.asd.sgi.com> <5hnam9$393@hoopoe.psc.edu> <5hp7p3$1qb@fido.asd.sgi.com> NNTP-Posting-Host: hoopoe.psc.edu Xref: euryale.cc.adfa.oz.au comp.unix.bsd.freebsd.misc:38284 comp.unix.bsd.bsdi.misc:6546 comp.sys.sgi.misc:29610 Larry, Larry, Larry. First, I apologize for quoting quite so much here, but I don't want to be accused of twisting Larry's words. Larry, let me make myself perfectly clear: you are not arguing against my position. You are arguing against your own demons. Let me clarify further: My problem with your USENIX paper was not with the conclusions. My problem with your USENIX paper was that you claimed to reach conclusions. Do you see the difference? No, I guess not. See, I could care less whether FreeBSD is faster, or Linux is more stable, or FreeBSD comes with chewy caramel in the center, or Linux has flavor crystals. I don't care about that horse race. YOU'RE the one who cares about that. I don't use either FreeBSD or Linux in my production work or at home anymore (although I've used both in the past). If I am standing up and saying "See, McVoy is a big fat idiot," I'm saying it not because your numbers were wrong (because, hey, who really cares?) but because your methodology was braindead. I think it was sloppy research. Let me say it again, louder, so maybe you'll hear me this time: YOUR RESEARCH METHODOLOGY WAS UTTERLY AND COMPLETELY BOGUS. I did not say "Your conclusions were bogus." I really don't care about your conclusions, but I do care about the integrity of USENIX as a place where I can count on the fact that if someone is presenting results to me, I can know that their methodology was rigorous. I don't think, after hearing your presentation, that I can count on that any more. If I dropped a MIPS chip in a standard PCI bus machine (with some help from the Hardware Faeries) and claimed that it was the same as an Indigo because "the chip is the same speed," I would be wrong, for the exact same reasons that you are wrong in your analysis below. Larry, the fact that your assumptions turned out to be close to reality doesn't really matter. Like I said, I am not concerned with whether Linux leaves you feeling minty fresh or FreeBSD has the power of scrubbing bubbles. I'm concerned when someone stands up in front of a group of his peers and pretends to be able to draw conclusions from hallucinatory research. -Peter In article <5hp7p3$1qb@fido.asd.sgi.com>, Larry McVoy <lm@slovax.engr.sgi.com> wrote: >Peter Berger (peterb@hoopoe.psc.edu) wrote: >: The point is that it doesn't matter which way -you- think the >: hardware differences are skewed. Frankly, I'm sure that Jordan >: would have objected to your conclusions even if FreeBSD won >: across the board: because your conclusions are completely, >: utterly, and didactically bogus. You can't compare apples to >: oranges and then claim that the oranges are apples because you >: like apple juice. > >Um, Peter. I hate to burst your bubble, but after Jordan flamed me I >went and tested FreeBSD vs Linux on the same machine, a P133. I didn't >think I needed to do so because the differences between a P120 and >a P133 are typically in the noise, especially when measuring the OS. >At any rate, the results came out exactly as I expected. The so called >hardware skew that you keep harping on made no difference. I'm pretty >sure I published those results on this list as part of the TCP latency >flamewar about a year ago. > >Don't you find it the slightest bit strange that even though the benchmark >is free software, even though it is part of the FreeBSD benchmarks, and >even though a couple of years have gone by, nobody from the FreeBSD team >(or anyone else, for that matter) has stood up, published results and said >"see, McVoy is a big fat idiot and here are the results that prove it". >I challenge you to go take the benchmark, get a machine, install Linux, >install FreeBSD and come back and tell me how the benchmark mislead us, >skewed our perception, or whatever your current favorite whine is. > >I challenge you to state what conclusions I drew that were "utterly bogus" >and prove that they were (or are) "utterly bogus". You won't, because >(a) you can't, and (b) you just want to whine. > >: The point is that saying "FreeBSD wins" or "Linux wins" when >: the underlying hardware you're making your "comparisons" on >: is different. If I compared two versions of IRIX, one running >: on an Indy and another running on a Personal Iris and then told >: the world that "IRIX X.X wins: pipe bw, system call, etc." the world >: would ask me what sort of crack I was smoking. And they would be >: right. > >Yup, they would. And if the hardware made any substantive difference, >you would be absolutely right. But in the lmbench paper, the P5 >that FreeBSD was on was actually slightly faster than the Linux P5. >It's hard to claim I was skewing the results against FreeBSD. And it >is also hard to claim that I was skewing the results against Linux. >I've measured lots of PCs and the difference between 120 & 133 is just >not enough to be an issue, it's in the noise. > >I'm happy to be proven wrong. I'm waiting.... >-- >--- >Larry McVoy lm@sgi.com http://reality.sgi.com/lm (415) 933-1804 -- Pete Berger, Esq. Coordinator, Regional Information Infrastructure Pittsburgh Supercomputing Center peterb@psc.edu http://www.psc.edu/~peterb I don't speak for my employers, nor they for me.