Message-Id:	<199611100335.QAA28548@papaioea.manawatu.gen.nz>
Comments:	Authenticated sender is <malcolm AT mail DOT manawatu DOT gen DOT nz>
From:	"Malcolm Taylor" <malcolm AT manawatu DOT gen DOT nz>
Organization:	Grafik Software
To:	Mark Habersack <grendel AT ananke DOT amu DOT edu DOT pl>, grendel AT ananke DOT amu DOT edu DOT pl,
	djgpp AT delorie DOT com
Date:	Sun, 10 Nov 1996 16:34:14 +1200
Subject:	Re: Why not to use 'tar' before packing DJGPP?
Reply-to:	malcolm AT manawatu DOT gen DOT nz

> Malcolm Taylor wrote:
> 
> > size using RKIVE or ACB. This is an extreme case though as RKIVE
> > requires 8Mb and ACB 16Mb to work well, so aren't really suited to
> > the 'everyone can use' category, but there are other more generally
> > usable compressors (eg the new JAR from the maker of ARJ).
> I keep my DJGPP distribution in RAR 2.0 format and it's about 35%
> smaller. RKIVE is excellent but unfortunately slow (I imagin it uses
> some sophisticated compression algorithm, maybe fractal one?) and I
> haven't seen the two others you mentioned. But I agree, PKZIP's
> algorithms are quite dinosaur today...

RAR only tracks around 64K of data, try using WinRAR2.0 with a 1M 
dictionary and you'll get much better ratios - eg. I compressed all the 
DJGPP exes in my Bin dir with PKzip and got 4008690byte zip, with 
WinRAR2.0 it goes down to 2483578. RKIVE gets 2022798 (with some 
extra options). This is close to half the size with PKzip, and that 
situation may well improve on the source zips. Note that RAR with a 
1M dictionary is slower than RKIVE though.
I wrote RKIVE so I'm a little biased, but compared with zip it is 
slow. It does use some fairly sofisticated algorithms, actually 
modern variants of PPM (PPMZ) and some modified string matching. ACB 
is about the best known archiver (in compression terms) but it is 
best run on a Pentium as it is around 2x slower than RKIVE. JAR is a 
new archiver (still in beta) by the author of ARJ. It has better 
compression than RAR and is faster.

> > You could put the compression straight into the installer itself, and
> > hence enable the use of tar or something else without anyone having
> > to be too troubled by it.
> In fact, the installer encapsulates the ZIP Deflate/Inflate algorithm
> and it would be fairly easy to add the tar support, as well as GZIP
> (well, GZIP is there for free, the installer uses zlib).

I thought that would be the case :)
 
> > One of the biggest problems with this is that it would require a
> > complete switch to the new archive format, or doubling up of archives
> > (old and new format) which would probably not sit well with DJ or the
> > Simtel.Net maintainers.
> That's right. Such a switch would take some time. Besides, I think DJ is
> compressing DJGPP on Unix, and AFAIK there's no other common compressor
> than ZIP or TAR on DOS and Unixes.

He does it on DOS. If there is a great need for smaller distribution 
archives then I can throw together a good archiver for the job in 
less than a week. It would not be as fast as pkzip, but should get 
significantly better ratios without too much more time. Is there any 
interest in this?

I just did a quick test in the background while writing this on the 
source files (basicly libc.a sources). PKZIP gets 1395854 in 3m33s, 
RKIVE gets 605076 in 12m55s (using fast mode). The times aren't 
accurate as it was executing in the background, but it gives an idea. 
I could write an archiver getting ratios close to RKIVE while being 
quite a lot faster.

Does your installer use cwsdpmi.exe or PMODE?

Malcolm

- Raw text -