Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.carno.net.au!harbinger.cc.monash.edu.au!news.cs.su.oz.au!metro!metro!munnari.OZ.AU!news.ecn.uoknor.edu!feed1.news.erols.com!dispatch.news.demon.net!demon!cpk-news-hub1.bbnplanet.com!news.bbnplanet.com!ix.netcom.com!enews.sgi.com!fido.asd.sgi.com!neteng!lm From: lm@neteng.engr.sgi.com (Larry McVoy) Newsgroups: comp.unix.bsd.freebsd.misc,comp.unix.bsd.bsdi.misc,comp.sys.sgi.misc Subject: Re: no such thing as a "general user community" Followup-To: comp.unix.bsd.freebsd.misc,comp.unix.bsd.bsdi.misc,comp.sys.sgi.misc Date: 3 Apr 1997 18:48:21 GMT Organization: Silicon Graphics Inc., Mountain View, CA Message-ID: <5i0u1l$8nk@fido.asd.sgi.com> References: <331BB7DD.28EC@net5.net> <5hun4a$eav@fido.asd.sgi.com> <5hus36$3ev$1@bofh.noc.best.net> <5huvun$hlq@fido.asd.sgi.com> <5hv9e9$jgo$1@bofh.noc.best.net> Reply-To: lm@slovax.engr.sgi.com NNTP-Posting-Host: neteng.engr.sgi.com X-Newsreader: TIN [version 1.2 PL2] Lines: 99 Xref: euryale.cc.adfa.oz.au comp.unix.bsd.freebsd.misc:38562 comp.unix.bsd.bsdi.misc:6595 comp.sys.sgi.misc:29723 Ron Echeverri (rone@bofh.noc.best.net) wrote: : The common perception seems to be that, to make a comparison between : two operating systems which run on the same chipset, a fair comparison : would involve running the benchmark on both operating systems on : similar hardware. Yeah, if I had been doing that, that would be the thing to do. I've never argued against that, nor would I. The important thing to note is what you are suggesting is a completely different project from what I did. : Now, if your intention was not to run a fair comparison with your : benchmark, but just to generate some cool numbers, or whatever other : goal you might have had at the time, then you are 100% in the right. Those who have read the paper or listened to the talk, know that the primary focus was to provide information about a wide variety of platforms. At the time the paper was published, it had very up to date information about current processors. The platforms measured included Linux, FreeBSD, OSF1, IRIX, SunOS, Solaris, AIX, HPUX, Unixware, NetBSD running on Intel, Sparc, PA-RISC, MIPS, RIOS, Alpha, 88K, just to mention a few. Somehow, in the minds of the FreeBSD crowd, this turned into a FreeBSD vs Linux comparison. Which is just ridiculous. It was no such thing and they know it. If you read the paper or listen to the talk, you will see me giving credit to HP for their caches, Intel for the processor, FreeBSD for ethernet, SGI for HIPPI, etc. If someone wants to turn it into a FreeBSD/Linux comparison, a quick read actually shows FreeBSD in a pretty good light. But that is beside the point, the paper is about results on many platforms, not about FreeBSD vs Linux. : >Anyone can look at some work, claim it doesn't do what they want it : >to do and therefor the work is "bogus" or "unscientific". By that : >definition, _all_ published works are "bogus" and "unscientific". : Certainly. It lies with the creator of the work to explicitly mention : what he is trying to accomplish with the work. Allow me to quote from the paper: .NH 1 Introduction .PP \*[lmbench] provides a suite of benchmarks that attempt to measure the most commonly found performance bottlenecks in a wide range of system applications. These bottlenecks have been identified, isolated, and reproduced in a set of small micro-benchmarks, which measure system latency and bandwidth of data movement among the processor and memory, network, file system, and disk. .NH 1 Conclusion .PP \*[lmbench] is a useful, portable micro-benchmark suite designed to measure important aspects of system performance. We have found that a good memory subsystem is at least as important as the processor speed. As processors get faster and faster, more and more of the system design effort will need to move to the cache and memory subsystems. The paper doesn't claim to be comparing FreeBSD vs Linux. Nor does it claim to be doing an indepth comparison of any two operating systems. It does claim to be giving a long list of a results for a long list of hardware & software platforms, and it did exactly what it said it would. Perhaps you would like to read the paper? http://reality.sgi.com/lm/lmbench/lmbench-usenix.ps : Now, forgive me, as i was not at USENIX, but, from what i've You've been flaming my ass off in a public forum about me not doing good work and you didn't take the time to read the paper or listen to the talk. You, sir, are a poor excuse for a scientist. Shame on you for daring to stand up and pass judgement on work you haven't even seen. If I were you, I'd crawl under a rock and stay there. Or you could say "whoops, sorry, I guess I should have checked, please accept my apologies" and I'll think you are a really great guy :-) In all seriousness, this sort of shoddy behaviour is why serious researchers don't bother to interact with Usenix. You might think about that. : understood, you used lmbench to benchmark Linux on a machine which was : faster than the one on which you benchmarked FreeBSD. I am not : certain that this is 100% accurate; if it is not, then forget it. No, it isn't accurate at all. The FreeBSD guys would claim I compared Linux on a P6 to FreeBSD on a P5. Any idiot can see that that is an unreasonable comparison, and since I knew that people would compare numbers, I also included results for Linux on a P5. So you could get some idea of the relative performance of the two operating systems. The amusing thing is that the Linux was a P5@120mhz and the FreeBSD box was a P5@133, so the numbers were marginally in favor of the FreeBSD box. -- --- Larry McVoy lm@sgi.com http://reality.sgi.com/lm (415) 933-1804