Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!news.rmit.EDU.AU!news.unimelb.EDU.AU!munnari.OZ.AU!news.mel.connect.com.au!news.mira.net.au!inquo!bofh.dot!in-news.erinet.com!imci5!imci4!newsfeed.internetmci.com!in1.uu.net!EU.net!Austria.EU.net!siemens.at!not-for-mail
From: mingo@pc5829.hil.siemens.co.at (Ingo Molnar)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: FreeBSD as a router
Date: 21 May 1996 11:21:50 GMT
Organization: Siemens AG Austria
Lines: 119
Message-ID: <4ns90e$c67@news.siemens.at>
References: <4lfm8j$kn3@nuscc.nus.sg> <317CAABE.7DE14518@FreeBSD.org> <4lt098$erq@itchy.serv.net> <Pine.SUN.3.90.960427140735.3161C-100000@tulip.cs.odu.edu> <4mj7f2$mno@news.clinet.fi> <318E6BB1.6A71C39B@lambert.org> <4mtfsg$14l8@serra.unipi.it> <319407B4.32F4B8B6@lambert.org> <4nc6v9$jib@news.siemens.at> <319BD085.3EE5FAF9@lambert.org>
NNTP-Posting-Host: pc5829.hil.siemens-austria
X-Newsreader: TIN [UNIX 1.3 BETA-950824-color PL0]
Terry Lambert (terry@lambert.org) wrote:
<slight snip>
: ] There is an overhead of deciding which one it was.
: ] But if there is heavy traffic, then it's MUCH cheaper to have
: ] shared interrupts. Interrupt frequency is to be kept low.
:
: The only thing you are doing by checkig board 2 after processing
: board 1's interrupt is delaying the ack so that board 1 is
: unavailable to process data because its interrupt has not yet
: been ack'ed in the shared case.
trying to put together some factoids:
i assume that both boards can be acked separately. (by acking the
IRQ on the board itself, so it releases the line). The PIC gets
acked right after the interrupt is discovered, no matter which
board it was. I see no unpredictable delay there. Here is how
shared IRQ handling is done:
IRQ comes in, PIC interrupts the CPU and hands a vector
generic handler gets called
handler masks off this irq, and ACKs it on the PIC
generic handler discovers that this IRQ is configured as shared
generic handler calls shared irq handler
shared irq handler polls first card -> status indicates no IRQ
polls second card -> status indicates IRQ
shared irq handler does it's work
shared irq handler acks the IRQ on board 1, the board in turn
releases the line.
shared irq handler does a second pass, to detect interrupts
received during the first pass.
if no more irqs left, then the shared handler returns
generic handler masks on the line in the PIC and does an rti()
[or calls the scheduler, depending on other things]
this isnt exactly what should be done, there might be some races left.
Some comments:
- board 2 doesnt know about board 1 at all, it just puts blindly
its IRQ on the line.
- two passes are needed to avoid races due to board 1 sending an
IRQ again
- in a busy system, interrupts occur more often, and there is a
fair possibility that the second pass discovers other interrupts.
( and it might happen that the handler runs for a longer time, no
problem, other interrupts are enabled )
: ] Internal serialization is a clear loss. Interrupt latencies can
: ] be up to several tens of usecs. Checking for a board is a few PCI
: ] cycles, much faster.
:
: You are serializing latency by delaying the first board
: generating its next interrupt until you have processed the
: interrupt for the second board. This is nore harmful than
: having to handle two interrupts.
no, i ack the first board after i've processed it. (btw, i think
network cards are not halted till the IRQ is not acked, the
3C509 has a status stack for example, if i remember right. other
boards might differ [and surely they do, judging from your comment :)
])
< snip, ISPs, dont have enough hard numbers:) >
: IMO, the name of the game is concurrency. The processor has to
yep
: stay as busy as possible, not executing bus interaction
: instructions. This is especially true of, for instance, a P166,
: which runs a 33MHz PCI and take 5 CPU clocks for each bus clock
: to arbitrate interrupt sharing.
interrupt processing is expensive. Checking for a status bit in a
(relatively fast) card, not. I heard somewhere that on the PC,
hardware irq latency is 0-8 usecs (this is from the point on that
the card raises the line, to the point where the CPU starts the
first instruction in the handler. handler = the entry in the IDT).
And the CPU does nothing useful. (this is 0-800 cycles on a 100 MHz
system). And then comes the generic handler, which has to issue a
few ISA cycles again to ack the PIC and mask off the IRQ.
you repeat all this crap if you have separate interrupts.
You are right that shared IRQs dont scale well with the number of
cards. But i would say that up to 3-5 cards on one interrupt, in a
busy system, there is lower overhead than in a system with 3-5
separate interrupts.
And separate IRQs are right if you want to prioritize your cards,
since in a shared IRQ system the first card detected will be
serviced first.
True, the name of the game is concurrency. The shared IRQ system is
event driven too, with an overhead of (1-2 * N) bus cycles. (this is
a definite overhead). But this event driven system turns out to
"clusterise" interrupts when the system gets busy. Which is the
right thing IMHO ...
: Yes, this is low overhead if you have a crappy PCI implementation,
: but the whole point I was trying to make was: don't buy crappy
: hardware if you require performance. This should be intuitively
: obvious. 8-).
true ... but when you have limited resources (=no money), then a
dirty shared IRQ system does almost as much as heavy iron :)
[ i'm talking about putting 3 $50 cards into the box, instead of
using a one-irq, internal-sharing, $500 thingie ]
[ heh, 'anything' is infinitely better than 'nothing' :) ]
-- mingo