Return to BSD News archive
Newsgroups: comp.os.386bsd.apps Path: sserve!newshost.anu.edu.au!munnari.oz.au!cs.mu.OZ.AU!summer From: summer@ee.mu.OZ.AU (Mark Summerfield) Subject: TCX -- transparent [un]compression of executables Message-ID: <9308509.26865@mulga.cs.mu.OZ.AU> Sender: news@cs.mu.OZ.AU Organization: Dept of Electrical & Electronic Engineering, University of Melbourne Date: Thu, 25 Mar 1993 23:02:02 GMT Lines: 154 A package called "tcx" has just been posted to alt.sources. It was actually developed by someone here at the University of Melbourne, but I make the disclaimer that I have never met them or had any contact (it's quite a big University!), nor do I have any personal experience with this package (yet). What this package does is to allow you to keep any (or all) of your executables in compressed form. They are transparently uncompressed when run, and automatically recompressed when not in use. This seems to me to be a potential boon for 386bsd'ers with limited disk space! Think of all those applications taking up heaps of space that you rarely use... Anyway, I haven't got my 386bsd system up and running yet, so I can't play with tcx -- I'm just posting this pointer to bring it to your attention. If anyone has a go at compiling/porting this package before I do (probably within a couple of weeks if all goes well) I'd be interested to hear how it goes. As I said, look in alt.sources. I have attached the README file. Mark. ---- TCX - Transparently Compressed Executables (For Unix) by Stewart Forster, University Of Melbourne, 1993 TCX is a system designed for the transparent decompression, execution and recompression of executables under Unix. It allows configuration options such as the type of compression system used (compress(1), gzip(1), your own local system etc), timeouts between recompressions, and emergency directories in case a decompression fails from shortage of disk space. The system is designed with a reasonable amount of robustness in mind, such as in the event of system crashes, or races on trying to uncompress, compress or execute something. Currently TCX is only proven to run on Irix 4.*, Ultrix 4.2 or higher, and SunOs 4.1.1 or higher. It may run on other releases of these OS's but has not been tested. It should also be relatively easy to port to other OS's as much of the code is fairly straight forward. The system is used here at the Department of Computer Science University Of Melbourne, and in other departments on campus. This is the first official release of TCX to the world, so please feel free to comment or help in bug fixes and machine reports by sending email to slf@cs.mu.OZ.AU --------------------------------------------------------------------- INSTALLATION TCX is a small package with just the following files. If you don't have these three file, get a complete distribution. config.h tcx.c untcx.c First read the "FURTHER INFO" section below before configuring TCX for your system, since it may influence some of your decisions. To configure TCX, edit the config.h file. There are enough directions therein to help you configure for your system. TCX relies on the compression system you use to barf when the filesystem gets full and return a fail exit code. If the compression system you use doesn't do this, you run the risk of losing files. The GNU gzip package is known to do the correct safety checks and exit accordingly. Compress may not under certain systems. Once done editing the config file, edit the Makefile to set the C compiler and libraries for your system, and compile. Don't worry if your optimizer complains of no exit points in the procedures called "check_tcxd_mode", "untcx_and_exec_local", and "untcx_and_exec_nfs" when compiling untcx.c, these are normal. The tcx, and untcx executables must be placed at the pathnames you specified for them in config.h, if they are not there, the system will NOT work. eg. cp tcx /usr/local/bin/tcx cp untcx /usr/local/bin/untcx Untcx must also be installed setuid, so issue the following command. chmod 4711 /usr/local/bin/untcx (or path you chose) and copy the manual pages to your regular man directory and read them. The installation is now complete. ---------------------------------------------------------------------- TESTING YOUR INSTALLATION You can now try running "tcx" on an innocuous executable to transform it, and then try running it. Wait for the timeout interval you specified (after disuse, which means not even reading the file) and it should recompress. If it doesn't recompress after about twice the interval specified, check the following. 1) Access times on the file. MAKE SURE it wasn't touched before suspecting something is wrong. 2) That the daemon mode for untcx started correctly. The pid should be stored in ENFSDIR/.lock. Assuming ENFSDIR is /tmp/exec as in the distributed config file, issue ps `cat /tmp/exec/.lock` and check that the daemon started. If it didn't, there is a minor problem with permissions, or maybe you specified a read-only file system for ENFSDIR. 3) Check if you installed tcx where PATHTCX in the config file said where it was. 4) Check that the uncompressor is where PATHUNPACK in the the config file says it is. 5) If you're using the compress/uncompress system, it may be returning bad exit codes. Try getting the GNU gzip compression package. If all else comes to nought, look into the source code, or mail your system configuration and config file to slf@cs.mu.OZ.AU, and I'll try and see what can be done. ---------------------------------------------------------------------- FUTHER INFO / POTENTIAL BUGS The compress(1) system had problems on Ultrix in that is was returning failures when trying to decompress a tcx(1) executable. I was unable to pinpoint the source of the problem, and this problem may exist on other systems too. I STRONGLY recommend using the "gzip" package from GNU as your compression system of choice, or at least for TCX, since it has much stronger error checking, and returns reliable exit codes. Gzip will compress from 10 to 40% better than compress, and is faster on uncompression, which is where the system becomes noticable to the user in the effect of delays when an executable is run. Users may experience a slight delay when a TCX executable is started up due the decompression phase. If this becomes a problem for large executables, you should probably discourage use of TCX on these huge binaries. The "gzip" system decompresses at around 500-700K/sec on the systems I tested it on, which are all around the 25 SPECmark rating. ---------------------------------------------------------------------- THANKS Special thanks go to Robert Elz, who provided invaluable input, suggestions and clarifications on the whole idea. Thanks also to Jeff Shultz who provided some potential "sticky" cases which gave me some more work to do (plus made tcx more robust). Stewart.