Return to BSD News archive
Path: euryale.cc.adfa.oz.au!newshost.anu.edu.au!harbinger.cc.monash.edu.au!news.uwa.edu.au!disco.iinet.net.au!news.uoregon.edu!vixen.cso.uiuc.edu!uwm.edu!cs.utexas.edu!howland.reston.ans.net!EU.net!usenet2.news.uk.psi.net!uknet!usenet1.news.uk.psi.net!uknet!dispatch.news.demon.net!demon!mail2news.demon.co.uk!pierrot.demon.co.uk From: Terrance Richard Boyes <tez@pierrot.demon.co.uk> Newsgroups: demon.ip.support,demon.tech.unix,comp.unix.bsd.freebsd.misc Subject: Re: Batch FTP and Web Pages Followup-To: demon.ip.support,demon.tech.unix,comp.unix.bsd.freebsd.misc Date: Sat, 29 Jun 96 19:09:41 +0100 Organization: ECL.net Lines: 29 Message-ID: <9606292009.1z0b@pierrot.demon.co.uk> References: <31c2e7bd.14691630@news.demon.co.uk> <834878464snz@pair.com> <834921960snz@michaels.demon.co.uk> <199606261821.SAA02204@mauve.demon.co.uk> <4qtcee$n1h@alfie.demon.co.uk> <31D42221.58C3@www.play-hookey.com> X-NNTP-Posting-Host: pierrot.demon.co.uk X-Newsreader: TIN [AMIGA 1.3 950726BETA PL0] X-NaMLServ: $VER: v0.9b <tez@pierrot.demon.co.uk> X-Mail2News-Path: relay-4.mail.demon.net!post.demon.co.uk!pierrot.demon.co.uk Ken Bigelow (kbigelow@www.play-hookey.com) wrote: > > The problem with that idea is that a Web page is not a single file. The > main HTML document is (usually) not that big, and consists entirely of > ASCII text. > > There seems to be no good way to retrieve such dynamic files in pieces > over time. There are scripts available on at least one platform that allow you to do some of this, eg ask for the entirety of one page including all immediate links, or all links down to two levels, or whatever. Last time I saw it the author was talking about allowing you to specify things such as, no gifs/jpegs. As for changes to files, that's only a small extension, just use the "normal" caching facilities, ie check the datestamps... Some caching progs will also allow you to work in either offline or online mode. In offline you can only view what you've already downloaded but requests to any links to things you don't currently have held locally are cached until you go into online mode... Whether such things are available on _other_ platforms I leave as an exercise for the readers/coders out there. -- <URL:http://www.geocities.com/BourbonStreet/1666> Team AMIGA Fort Wayne is not the headquarters of F troop.