Return to BSD News archive
Xref: sserve comp.unix.aix:48686 comp.unix.bsd:15531 comp.unix.pc-clone.32bit:7703 comp.unix.solaris:28152 comp.unix.unixware:15059 Path: sserve!newshost.anu.edu.au!harbinger.cc.monash.edu.au!msunews!uwm.edu!cs.utexas.edu!swrinde!sgiblab!sgigate.sgi.com!fido.asd.sgi.com!slovax!lm From: lm@slovax.engr.sgi.com (Larry McVoy) Newsgroups: comp.unix.aix,comp.unix.bsd,comp.unix.pc-clone.32bit,comp.unix.solaris,comp.unix.unixware Subject: Re: Unix for PC Date: 10 Dec 1994 22:01:28 GMT Organization: Silicon Graphics Inc., Mountain View, CA Lines: 61 Distribution: inet Message-ID: <3cd8fo$ns5@fido.asd.sgi.com> References: <199411210319.TAA18133@nic.cerf.net> <D0E32G.3x8@news.cern.ch> <MICHAELV.94Dec10124723@MindBender.HeadCandy.com> Reply-To: lm@slovax.engr.sgi.com NNTP-Posting-Host: slovax.engr.sgi.com X-Newsreader: TIN [version 1.2 PL2] Michael L. VanLoon (michaelv@MindBender.HeadCandy.com) wrote: : In article <3c81c7$h1o@fido.asd.sgi.com> lm@fubar (Larry McVoy) writes: : Nate Williams (nate@bsd.coe.montana.edu) wrote: : : C'mon Dan. Commercial OS software testing is completely different than : : free software testing in general. The reason Linux and FreeBSD have : I hate to burst your bubble, but I worked at Sun in the systems group for : a few years (and then in the server group). They had *no* regression : test other than the binaries that shipped with the OS. Since 5.x, : they use the POSIX test suites but those (were) are pathetic and : certainly don't cover everything. : I've worked with a certain very large software company and it is not : done like this at all. They do extensive testing constantly during : the development cycle, before anyone outside the company even sees it. I think you are misunderstanding the point. Certainly Sun, and every other big or small company, will run the new release internally before shipping it and will "test" that the binaries "work". I think we are arguing about the definition of test and work. "It boots" is not the same as testing, nor is running bunch of makes and ftps, whatever. Certainly those are all good things but they are far from conclusive. If it was that easy to test the software and insure no bugs then every company would do so, they aren't stupid. The point is that there is very little in the way of formal regression testing. Even to the point of testing for old bugs. You would think that any company that fixes a bug would add a small test to a regression suite that tested for that bug as a part of the fix. Sun doesn't. I don't know of any major Unix vendor that does. Most of the testing falls under one of the following catagories: 1) run it as a server for builds and/or NFS service (the latter is a pretty hard thing to get people to do - internal organizations don't want their NFS servers going down either). 2) Run some sort of conformance test suite such as POSIX. This is painful and not done frequently. 3) Run it on the developer's desktops. This is the most common. I'd love to "raise the bar" for the industry as a whole by having someone out there describe their testing procedure that goes beyond what I described. I think that more testing is good. I think that no amount of testing finds everything. I think that people don't like hearing that their favorite OS might not be bullet proof but that is a fact of life. The original point was that Linux is less tested than the major vendor's releases. That may be true but the gap between Linux and a commercial release is much smaller than you seem to think. -- --- Larry McVoy (415) 390-1804 lm@sgi.com