Return to BSD News archive
Path: sserve!newshost.anu.edu.au!munnari.oz.au!constellation!aardvark.ucs.uoknor.edu!ns1.nodak.edu!netnews.nwnet.net!ogicse!hp-cv!sdd.hp.com!math.ohio-state.edu! magnus.acs.ohio-state.edu!usenet.ins.cwru.edu!agate!dog.ee.lbl.gov!hellgate.utah.edu!fcom.cc.utah.edu!cs.weber.edu!terry Newsgroups: comp.os.386bsd.development Subject: Re: Compressing file system ? Message-ID: <1993Aug10.222057.20600@fcom.cc.utah.edu> From: terry@cs.weber.edu (A Wizard of Earth C) Date: 10 Aug 93 22:20:57 GMT Sender: news@fcom.cc.utah.edu References: <EgN6R_K00WB5JOZpRc@andrew.cmu.edu> <1993Aug9.233341.20631@fcom.cc.utah.edu> <johnh.744999634@ficus.cs.ucla.edu> Organization: Weber State University, Ogden, UT Article-I.D.: fcom.1993Aug10.222057.20600 Lines: 81 In article <johnh.744999634@ficus.cs.ucla.edu> johnh@ficus.cs.ucla.edu (John Heidemann) writes: >>Perhaps the best implementation of a compressed file scheme I have seen was >>the one in the UCLA Ficus implementation in BSD4.4; it has many of the >>attributes I have tagged as "desirable" (noting that the aesthetics are >>dictated by teeny, tiny disks rather than clean kernel implementation or >>fast access). > >Here I am somewhat confused. Ficus (which is only the replicated file >system) is not part of BSD 4.4; only the UCLA stackable layers >interface is included. Unfortunately, 4.4 does not include a >compression layer, just a null layer and a user-id mapping layer. > >The SunOS implementation of UCLA stacking has at one time included a >prototype compression layer. This layer was done as part of a class >project by Geoff Kuenning and Steve Whitney. Unfortunately, as a >class project, though, it lacked some of the...reliability...one might >want before putting it in production use. (BTW, to relative to the >problems described above, it encoded the compression attribute in the >file name and compressed on block boundaries.) This is interesting; I was under the impression from UCB that the entirety of your work was included in 4.4, and not just the stackable layering. I am disappointed on two fronts; the first is, that they should have taken everything; the second is, if they weren't going to use preexisting modules, Rosenthal makes some good points for reforming the vops interface and the vnode itself above and beyond what your project has done. In the process of completing my work, I have been doing name space and attribution in a monolithic (UFS) file system as a set of extensions to the ordinary semantics. Like the EFS and Kuenning/Whitney work, this encodes the selectors as part of the name information -- reducing the set of available file names. It's clear that there are major problems in the UFS inode caching and the POSIX atime semantics that a vops reform would either eliminate completely or at least ameliorate. I was also under the impression from the paper that the compression layer was much more real. 8-). Compression on block boundries seems to be the only mechanism that could easily guarantee fast random access without a full decompression and timeout recompression of the data. My entire world view has become somewhat slanted toward the "generic object store" model since realizing the directory heirarchy and file storage models were hopelessly polluting each other in working out the directory issues. >>I'd encourage anyone who wants to get into building file systems to do so; >>there are very few live projects of this type out there, and I can count the >>number of commercial file system vendors (that's all they do) on one hand. > >I'd have to agree with this. I'd love to see a good Unix >compression layer, among other things. Is there any way that the same UCLA stacking code could be donated to the *BSD projects on the net (under the same restrictions given to CSRG, of course) so that there is not a SVR4 source license requirement to see it (given that the BSD 4.4 Lite release seems to be in danger of not happening for the near future)? What parts, exactly, did CSRG get? The UFS implementation in the paper would be a nice excuse to throw out the existing monolithic UFS in *BSD and to start down the right road (IMO) for new file system work. This would provide a better base than what is currently available to the casual developer on these systems, which is basically module loading of file systems as functional units (per my LKM package) without the ability to apply a layering model. If you have seen the "Filesystem Survival Kit" for SVR4, you are already aware that research developement in the *BSD arena is quickly shared into more commercial areas, and to the benefit of many users. The first candidate for this could easily be seperation of the RockRidge extensions from the ISO9960 FS as a seperate functional layer. This would provide a good proof-of-concept test for seperation of directory services from storage services. Regards, Terry Lambert terry@icarus.weber.edu --- Any opinions in this posting are my own and not those of my present or previous employers.