Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.carno.net.au!harbinger.cc.monash.edu.au!news.cs.su.oz.au!metro!metro!munnari.OZ.AU!news.ecn.uoknor.edu!news.wildstar.net!news.ececs.uc.edu!news.kei.com!news.mathworks.com!EU.net!usenet2.news.uk.psi.net!uknet!usenet1.news.uk.psi.net!uknet!dispatch.news.demon.net!demon!awfulhak.demon.co.uk!awfulhak.demon.co.uk!awfulhak.demon.co.uk!not-for-mail
From: brian@anorak.utell.net (Brian Somers)
Newsgroups: comp.unix.solaris,comp.unix.bsd.misc
Subject: Re: Solaris 2.6
Date: 27 Nov 1996 16:24:53 -0000
Organization: Coverform Ltd.
Lines: 229
Sender: brian@awfulhak.demon.co.uk
Distribution: inet
Message-ID: <57hq0l$3rp@anorak.utell.net>
References: <32986299.AC7@mail.esrin.esa.it>
<57djlg$bks@agate.berkeley.edu> <57dkbq$bsr@panix2.panix.com>
<casper.329abb76@mail.fwi.uva.nl> <57ej3a$7ij@panix2.panix.com>
<casper.329ae8f2@mail.fwi.uva.nl>
Reply-To: brian%anorak.coverform.lan@awfulhak.demon.co.uk
NNTP-Posting-Host: anorak.coverform.lan
X-NNTP-Posting-Host: awfulhak.demon.co.uk
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Newsreader: knews 0.9.8
Xref: euryale.cc.adfa.oz.au comp.unix.solaris:90671 comp.unix.bsd.misc:1611
First, sorry if I've missed any issues - I've only seen the last two
articles in this thread - the previous (four) references must have
been destined for comp.unix.solaris only ???
In article <casper.329ae8f2@mail.fwi.uva.nl>,
casper@fwi.uva.nl (Casper H.S. Dik) writes:
: tls@panix.com (Thor Lancelot Simon) writes:
:
: [ I *really* hate it when people send me copies of replies thru email;
: invariably this I end up replying twice ]
:
>>>tls@panix.com (Thor Lancelot Simon) writes:
:
>>Oh? And why can't you? This works just fine under 4.4BSD, without causing
>>any applications compiled for the old size to lose. See below.
:
: What about old .o files? Old dynamic libraries? Old archive libraries?
:
: How do you compile using those?
:
You can't compile old objects/libraries with new objects/libraries and
expect to get a working binary at the end. Think about it - your header
files aren't the same ! To do this sort of thing, you need to make
the compiler aware of the version number of the shared library with which
each function/variable prototype is resolved. The compiler would then
need to store that information with the external symbol inside the object
file. At link time (or run time if linked sharable), this info would be
used to figure out how to resolve the symbol.
: You *can't* do it in the way you describe. You need new external
: symbols.
A new "external symbol" is a new function - a completely different thing.
: I know that free-unixes don't really care that much about binary compatibility
: (you should only run applications you ahve source for, right?)
What ?
"free-unixes" (certainly FreeBSD) have better binary compatibility than
any comercial unixes I've seen !
>>As for newly compiled applications, this is *why* there are all of those
>>typedefs in the header files in the first place -- so that correct code which
>>uses off_t, dev_t, pid_t, &c & so forth won't lose when the type size is
>>changed.
:
>>And no, naive code won't lose -- that's what prototypes are for.
:
: No, but code that uses "long" to calculate with off_t's will need to
: be fixed; it won't compile or just gives a warning and an erronous
: computation.
:
: You can claim that such code has a bug, but you also have the
: C-standard claiming that lng is the largest integral type.
This "excuse" for badly written code is bogus. Passing the wrong "size"
argument is wrong - whether it's bigger or smaller. This is why compilers
(certianly gcc) do the "lint" job these days.
>>>All you really need is -D_FILE_OFFSET_BITS=64 on the compile
>>>command line. The new API is for internal use of libraries and include
>>>files only, except for code where a distinction between the two types
>>>of open's needs to be made, such as in libraries.
:
>>There shouldn't be two types of open, period. The 32-bit open (lseek, etc)
>>should remain implemented under its old syscall number, but should vanish
>>from all libraries and include files entirely. Problem solved -- *without*
>>two types of open and all the concomitant lossage, not to mention inelegance.
:
: The dynamic libraries will need to retain the old "open" entry point for
: use by old applications.
Yep. - or the old shared libararies that these old apps know about and
insist on having in order to run.
>>Basically, you guys are ignoring plenty of preexising real-world experience
>>with this issue, and doing something really dumb just because some other
>>vendors are doing so. Would your development organization jump off a bridge
>>if HP's did? I didn't think so. :-)
:
: I think you guys are being casual about supporting preexisting software
: in binary form; apart from a lot of broken code being sold that still must
: work after we go to large files, there's also a lot of the following
: around:
:
: - vendors shipping archive/dynamic libraries for users
: to link against (All major DB vendors, for one)
Then they are *wrong*. If you ship object code, it *must* be compatible
with the OS's reckoning. (otherwise, the it must be aware of the
syscall numbers or the shared library versions - symbols aren't enough).
: - libraries created/maintained locally
Rebuild them.
: You can't solve that without keeping the old symbols and sizes
: available.
:
: You can say, well just bump the libc.so. revision number?
: Well, it wouldn't work; you'd get your users into a complex mess
: requiring them to link either 32bit file only objects or
: 64bit file only objects. Since "open/lseek/stat" etc are already
: taken the "open64" etc calls are used (invisibly) as entry points
: for 64bit compiled objects.
Sounds like a microsoft answer. Dynamic libraries can only support
binaries that were linked against them - ie. the header files used to
build an object's external references and the dynamic library that
resolves that reference are tied. There's no workable way to break
that tie.
>>Mind you, the commercial vendors, as usual, didn't bother to examine actual
>>experience with the research systems before going their merry way; they
>>evidently preferred, as usual, to theorize about it -- incorrectly.
:
: I think that, as usual, free Unixes ignore an important part of
: what binary compatibility consists of *for commercial unixes*.
: The market is different, the requirements are different.
:
: Tell me, can you take 4.3 BSD .o files and link them to a 4.4 BSD program>
Why do you keep on at this ? The .o file is tied to the header file !
If you can guarantee that the entry into a given (old) .o file complies
with an existing header and that the (old) .o files entry
into any shared libraries is already specified (ie. the shared library
version is contained in the object file), then you can link new .o
files with old .o files - but the library version stuff is written by the
loader, not the compiler. This is because the compiler doesn't know if
the binary will be statically or dynamically linked.
>>>Anotehr important requirement is that non-64 bit offset aware programs
>>>should not be able to manipulate or modify files with sizes overflowing their
>>>idea of off_t. You can't tell in advance whether they're gong to seek or
>>>not, but we don't want them to corrupt data if they do.
:
>>I'm not following you here. POSIX indicates that lseek takes an off_t as an
>>argument. You seem to be suggesting either that the same syscall number's to
>>used for lseek() by old and new binaries, or that code which explicitly
>>casts the argument of lseek to long (usually by using the constant 0L, in my
>>experience) isn't broken.
:
: No, I'm suggesting that 32bit off_t programs may corrupt >32bit files because
: of truncation that happens when seek pointers are obtained and later used.
You're linking an object that effectively has the wrong "lseek" args. You'll
probably end up corrupting the whence argument by having the wrong size
offset argument.
:
: There's still the unsolvable problem of large fds inherited by non largefile
: savvy programs.
This isn't unsolvable (unless linking old and new objects). The old version
of the open syscall will only ever return the expected size descriptor.
>>Again, actual experience porting hundreds of large programs to 4.4BSD
>>indicates that this is essentially a nonissue -- the few erroneous casts are
>>easily enough taken care of, and correct prototypes in the header files
>>ensure that naive programs get lseek()'s seek argument promoted to 64 bits --
>>and anyone writing new code which explicitly casts lseek's argument to long
>>should, frankly, be taken out back and shot.
:
: Sure, but we'd have gone to some traumatic source transformations before.
:
: I don't believe we can sell "sure you must rebuild all your .o files and
: libraries" to customers upgrading to 2.6.
Why not ? Either you have a binary & old shared libs and nothing needs to
be done or you've got source (and you can type "make").
The only bad situation is if you've got proprietry shared libraries and want
to build new code using them. This can't work unless the objects that are
built into said libraries know what versions of what shared libraries they
require symbols from (this is currently unknown as I said above).
I *would* say that it's only a matter of getting the owner of the shared
library to rebuild you a new version - but I've been there before - it's
not that easy :(
>>And the idea that programs should ever _need_ to be "64 bit offset aware" is
>>an abomination all of its own. What happens when you want 128 bit offsets? I
>>guess you go through this all *again*?
:
: That's really only needed for the components of the ssytem that need to
: distinguish, such as the kernel and the libraries.
: It's not for use in programs. It's not something application
: developers have to go through (and again, and again, etc)
Which necessitates the inability to link old and new objects without
storing the shared library info in the object.
>>*Sigh* once upon a time, Sun was one of the "good" Unix vendors, actually
>>paying attention to the results which came out of research systems and acting
>>accordingly. Unfortunately, these days, though Sun does plenty of good
>>research themselves, they seem to suffer from severe (if typical)
>>commercial-vendor NIH.
:
:
: Nothing you said has convinced me one bit that it is possible to
: redefine open() and get the backward compatibility exepected of a commercial
: OS vendor.
Are we saying that solaris can link new objects with old shared libraries ?
[I missed the first four articles in this discussion]
If this is the case, how are they achieving this ?
: There's one simple option to recompile your code (if it doesn't
: mixup off_t's and longs and if it doesn't define it's own types/
: prototypes) to get largefiles. I don't see a way to do it differently,
: given the constraints commercial OS vendors operate under.
Correct. It's not easy to write code that uses the correct types, but
it's getting easier. gcc now distinguishes between different enums. You
can switch on "-Wall" and get complaints about assigning from a "bigger"
type - it's just that this results in piles of warnings about things that
are actually ok.
What we *really* need in gcc is the ability to complain about assignment
between different types - *even* if they're typedefs of eachother. That
way the long/off_t problems will be seen when they are written, not
10 years later when someone increases the size of an off_t.
--
Brian <brian%anorak.coverform.lan@awfulhak.demon.co.uk>
<http://www.awfulhak.demon.co.uk/>
Don't _EVER_ lose your sense of humour....
.