*BSD News Article 50705


Return to BSD News archive

Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!simtel!news.kei.com!news.mathworks.com!tank.news.pipex.net!pipex!newsfeed.internetmci.com!news.sprintlink.net!in1.uu.net!news.ios.com!chelsea.ios.com!haven.ios.com!rashid
From: rashid@haven.ios.com (Rashid Karimov.)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: QUOTAs cause system lockup. URGENT !
Date: 6 Sep 1995 19:26:37 GMT
Organization: Internet Online Services
Lines: 100
Message-ID: <42ksld$p2k@chelsea.ios.com>
NNTP-Posting-Host: haven.ios.com
X-Newsreader: TIN [version 1.2 PL2]


		Hi there folx,

	looks like we're getting closer to the biggest ;) problem
	the FreeBSD faces with on the servers market:
	system locks. Random and confusing.

	A long time ago I was advised by Terry (Lambert) that
	the QUOTAs implementation in FreeBSD could cause the
	system lockup.

	Citation:

>From looking at the quota code, it looks like it may not take the
dev_t field into accoun when computing quotas, which means that it
doesn't use the right mount record to locate the quota file.  So
it locks entrancy across file systems, but doesn't compute transitive
closure over the directed graph which is the set of locks held in
all file systems.

Really, it should per-fs lock to guarantee reentrancy.

Probably this is a 10-15 line fix in the quota code alone, if someone
spends the time working it out (I've just taken a quick pass through
the code that does vonde-based I/O that the quota stuff uses, and
I didn't see any obvious bugs).       

	=-=-=-= End of citation =-=-=-=

	But it happened that starting with 205 we all got working QUOTAs
	and they still work with 2.1Stable  kind of fine. But:

	on certain servers here ( they all pretty much the same - 
	P90-120 PCI Adaptecs/SMC EtherPowers,4000 users) I get often locks.
	System literally dies. And having now DDB compiled I was
	able to see what's going on when the systems lock.

	A LOT of processes are sleeping on wmesg "ufslk2" with stat of 3
	and flag of 00004. Saying a lot I mean probably 100-200 of them.

	I tried to see what was on the caddr_t they were sleeping at ( that
	should be i_node struct) and I got dev_t = 400 and inode = 2 (!!?).

	Looks like those processes went to sleep from ufs_lock() func.
	("ufs_vnops.c" file).

	What makes me think that was QUOTAs code which  caused this is 
	that in the beginning there were a couple of processes "edquota"
	which I started to change the QUOTAs for a couple of users. 
	When I did ps ( and system was alive at that time) a bit later
	I saw those processes (only!)  were sleeping on the same "ufslk2"
	event.
	I decided to start extra "edquota" process - just for the heck of
	it and system locked up in a minute. When runned "ps" from DDB
	later - all those processes the system was running were sleeping
	on the same event ( well, almost all of them - probably 90%,
	but there were a few sleepeng on the same wmesg but with different
	wait channels).


	So that's about it .... I'm not sure  who was the author of the 
	QUOTAs code and where is the bug exactly. I also know that certain
	ppl here think that the currnet implementation of QUOTA mechanism
	sucks and aren't willing to change it, voting for rewriting the thing
	form the scratch.


	What should we do about it ? I don't have enough time to dedicate
	to this problem and frankly don't have enough kernel programming
	experience to work on it.
	The same time QUOTAs are a must for FBSD to be used as a user
	server.

	A bit more about that system:

	P90 ASUSP54TP4 motherboard , Adaptec 2940 + 2 SEAGATE BARRACUDAs,
	QUOTAs are on :

	/dev/sd0a           /       ufs rw 1 1
	/dev/sd0s1b         none        swap    sw 0 0
	proc                /proc       procfs  rw 0 0
	/dev/sd0s1e         /usr        ufs rw 1 1
	/dev/sd0s1f         /var        ufs rw,userquota 1 1
	/dev/sd1s1e         /u/u1       ufs rw,userquota 1 1
	/dev/sd1s1f         /u/u2       ufs rw,userquota 1 1
	/dev/sd1s1g         /u/u3       ufs rw,userquota 1 1
	/dev/sd1s1h         /u/u4       ufs rw,userquota 1 1

	Rashid


--
                             ------------------

               Beyond the horizon of the place we lived when we were young
               In the world of magnets and miracles
               Our thoughts strayed constantly and without boundary
               The ringing of the division bell had begun ...
                        -=PF, The Division Bell=-