Return to BSD News archive
Xref: sserve comp.unix.bsd:12630 comp.os.linux:54760 comp.os.386bsd.misc:993 Newsgroups: comp.unix.bsd,comp.os.linux,comp.os.386bsd.misc Path: sserve!newshost.anu.edu.au!munnari.oz.au!news.Hawaii.Edu!ames!elroy.jpl.nasa.gov!swrinde!gatech!news.byu.edu!cwis.isu.edu!fcom.cc.utah.edu!cs.weber.edu!terry From: terry@cs.weber.edu (A Wizard of Earth C) Subject: Re: shared libraries (was BSD UNIX) Message-ID: <1993Sep13.203442.21808@fcom.cc.utah.edu> Sender: news@fcom.cc.utah.edu Organization: Weber State University, Ogden, UT References: <CCu0s1.29o@ssesco.com> <1993Sep7.162843.19294@fcom.cc.utah.edu> <33833@dog.ee.lbl.gov> Date: Mon, 13 Sep 93 20:34:42 GMT Lines: 95 In article <33833@dog.ee.lbl.gov> torek@horse.ee.lbl.gov (Chris Torek) writes: [ ... Static vs. Dynamic link tradeoffs ... ] >This makes the Utah approach (as described at the last USENIX) all the >more interesting. In this case, `executing' a binary invokes the >linker, which can choose either to run a cached `pre-linked' version or >to construct a new one. As Terry notes, most applications are run much >more often than they need re-linking (the shared libraries do not >change often). Hence, the same cached `post-fixup' version can be >reused (saving time) and shared (saving space). In effect, this is >the happy medium between `pure static' and `pure dynamic': resolved >on demand, then static until something forces a change. I thought that the numbers presented in the paper were, shall we say, optimistic, especially with regards to the relative frequency of cache hits. There is also the problem of converting the usage back to vanilla (or not so vanilla) C from the C++ required in their implementation. >Note that if this is done `right', the cached post-fixup binaries do >not need to contain the shared libraries. Rather, the dynamic linker >chooses an address for each shared library for each binary, and >attempts to keep this address constant per library. If/when this >attempt fails, this falls back to the equivalent of `pure dynamic' >linking; when it succeeds, it works much like `pure static' linking. >The only thing that the choice of per-library address `constants' >affects is the ratio of successful `static-like' links to failed >`dynamic-like' links. > >Assigning addresses to libraries would be the task of a program similar >to SunOS's `ldconfig'. A library that lacks a preselected address >would simply have one chosen dynamically. This would take more space, >to cache `preloaded' binaries and (at their preselected addresses) >libraries, but only those that are in fact used and/or only those that >meet some policy. Again, the fallback is, in essence, `pure dynamic' >linking; all else is merely optimization. The idea of attempting to place the tables at a known location in all binaries was the real interesting idea -- that way the post-fixup pages can be shared. The problem with assigned addresses remains, however... you still eat an address range for the library per version of the library. I don't think the work at the UofU went through very many generations of libraries, and so this problem didn't become evident. The dynamic fallback simply resolves the packing problem. Admittedly, this will alleviate the space issues somewhat, but with the excessively frequent revisions to libraries for systems undergoing active developement (like NetBSD/FreeBSD), this either implies a release authority with more frequent releases or a plethora of incompatable library images. I think it is just as possible to get around the problems with fixed GOT locations; note that global data referenced by the libraries, unless it is const data, must be pushed into the process at initial symbol resoloution time, even if the link is delayed (as it is in the Utah system). This means that data reference must be through the relocation table as well, and thus pushing the table out of the image will not be successful if there is external and internal symbol references taking place... this, in a nutshell, was the main impediment to using Pete's modification of Joerg's fixed address libraries to produce workable X (or termcap) libraries. I think the basic justification for the Utah code comes from the ability to launch or relaunch an application rather quickly; this is not the highest item on my list of needs, and paying the per application fixup penalty up front at exec time for the first instance of an application is acceptable, especially as a trade-off to incurring additional run-time overhead. I think a lot of the problem could be alleviated by copy-on-dirty and force-copy-on-text-busy to get the swap backing store into the swap area and off the file (getting rid of the antiquated "ETXTBUSY" we bought off on when we ate the MACH VM system whole). A subsequent LRU garbage collection in swap would buy us the pre-relocation benefits of the Utah system by allowing copy-on-write from a prerelocated image -- the "old" swap image being used to get the relocated GOT and read-only and non-dirty pages without recourse to the image. This would imply a need for a unification of the vnode LRUing similar to the SVR4 implementation to let us force-flush a swap image of a vnode when the vnode itself was flushed (this assumes that swap might not be the limiting resource). A secondary benefit is that the currently broken NFS model (how do we return ETXTBUSY when the file is executing on a client instead of locally?) will be 'magically' resolved. A tertiary benefit to the swap-store changes is that we will be able to get a real system dump without reserving a dump partition other than swap. It isn't necessary to buy off on the whole ball of wax to get most of the benefits. Terry Lambert terry@icarus.weber.edu --- Any opinions in this posting are my own and not those of my present or previous employers.