*BSD News Article 73586


Return to BSD News archive

Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!nntp.coast.net!news.kei.com!newsfeed.internetmci.com!zdc!zdc-e!szdc-e!news
From: "John S. Dyson" <toor@dyson.iquest.net>
Newsgroups: comp.os.linux.networking,comp.unix.bsd.netbsd.misc,comp.unix.bsd.freebsd.misc
Subject: Re: TCP latency
Date: Sat, 13 Jul 1996 14:39:21 -0500
Organization: John S. Dyson's home machine
Lines: 26
Message-ID: <31E7FB69.167EB0E7@dyson.iquest.net>
References: <4paedl$4bm@engnews2.Eng.Sun.COM> <31E106AF.41C67EA6@dyson.iquest.net> <4rvmtf$ven@linux.cs.Helsinki.FI> <31E3D9E2.41C67EA6@dyson.iquest.net> <4s5bl2$qpg@linux.cs.Helsinki.FI> <31E664EB.167EB0E7@inuxs.att.com> <4s67sk$oa9@fido.asd.sgi.com> <31E6B8AB.3E6C@indy.celebration.net> <4s7j2r$blf@fido.asd.sgi.com> <31E79738.1D1E@cet.co.jp> <4s8piu$f6p@fido.asd.sgi.com>
NNTP-Posting-Host: dyson.iquest.net
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Mailer: Mozilla 3.0b5a (X11; I; FreeBSD 2.2-CURRENT i386)
Xref: euryale.cc.adfa.oz.au comp.os.linux.networking:45106 comp.unix.bsd.netbsd.misc:4026 comp.unix.bsd.freebsd.misc:23484

Larry McVoy wrote:

> 
> I should probably not even argue about this, you can go get the benchmark
> and reproduce the numbers yourself if you have any doubts.  I do take
> exception to people suggesting I fudged the numbers, so I'll ask you
> to produce data if you're going to go that route.  Or just shut up.
>
The biggest problem that I have is NOT with the numbers, but with the
conclusions associated with the numbers.  For example, you run a
test on a P6-200 with Linux and P5-133 with FreeBSD.  Because the
Linux runs are faster than FreeBSD, does NOT MEAN that the Linux kernel
is better (faster) than the FreeBSD kernel.  All you can say is that the
faster machine ran the BENCHMARKS on Linux faster than the slower
machine ran the BENCHMARKS on FreeBSD.

My position is that BENCHMARKS are good ONLY at measuring what they
measure, and to compare OSes, it is very important to understand what
is measured.  Since most people don't understand what is being mesasured,
because they most likely don't read the code, the conclusions made
by people who analyse the results are as influential as the benchmark
numbers themselves.  Just be careful and have integrity with
conclusions.

John