Release files - Squashfs2.0-ALPHA

2.0-ALPHA       21 MAY 2004     Filesystem changes and compression improvements

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>

5 files changed, 176 insertions, 19 deletions

diff --git a/ACKNOWLEDGEMENTS b/ACKNOWLEDGEMENTS
index 597fc4b..72d47ce 100644
--- a/ACKNOWLEDGEMENTS
+++ b/ACKNOWLEDGEMENTS
@@ -58,3 +58,8 @@ squashfs-tools package) suggested the new mksquashfs -ef option, and the
 standalone build for mksquashfs.
 
 Mike Schaudies made a donation.
+
+A lot of people have emailed patches, suggestions etc. since
+the release of 1.3r3.  I have not got time to properly
+mention names here.  I'll try and get this done for the BETA
+or later versions of Squashfs 2.0.
diff --git a/CHANGES b/CHANGES
index 3e4da32..2e9ecb2 100644
--- a/CHANGES
+++ b/CHANGES
@@ -84,3 +84,19 @@
 	   on an Apple G5,
 	5. New patch for Linux 2.4.24,
 	6. New patch for Linux 2.6.1, this replaces the patch for 2.6.0-test7.
+
+2.0	21 MAY 2004	Filesystem changes and compression improvements
+
+	1. Squashfs 2.0 has added the concept of fragment blocks.
+           Files smaller than the file block size and optionally the
+	   remainder of files that do not fit fully into a block (i.e. the
+	   last 32K in a 96K file) are packed into shared fragments and
+	   compressed together.  This achieves on average 5 - 20% better
+	   compression than Squashfs 1.x.
+	2. The maximum block size has been increased to 64K (in the ALPHA version of
+	   Squashfs 2.0).
+	3. The maximum number of UIDs has been increased to 256 (from 48 in 1.x).
+	4. The maximum number of GIDs has been increased to 256 (from 15 in 1.x).
+	5. Removal of sleep_on() function call in 2.6.x patch, to allow Squashfs
+	   to work on the Fedora rc2 kernel.
+	6. Numerous small bug fixes have been made.
diff --git a/INSTALL b/INSTALL
index df2e7e4..b98ab6f 100644
--- a/INSTALL
+++ b/INSTALL
@@ -1,32 +1,31 @@
 			INSTALLING SQUASHFS
 
-The squashfs1.3r3.tar.gz file contains this file, a README file,
-an ACKNOWLEDGEMENTS file, a CHANGES file, the squashfs patch directories/files, and
-the squashfs-tools directory (mksquashfs).
+The squashfs2.0-ALPHA.tar.gz file contains this file, a README file,
+an ACKNOWLEDGEMENTS file, a CHANGES file, the squashfs patch
+directories/files, and the squashfs-tools directory (mksquashfs).
 
 1. Patching the kernel
 ----------------------
 
-There are six kernel patch directories depending on your linux kernel version.
-In each directory there are patches to provide squashfs1.3 release 3 support
-from a base kernel, and an incremental upgrade patch that upgrades a
-squashfs1.3r2 patched kernel to a squashfs1.3r3 kernel (except for 2.4.24 and
-2.6.1).
+There are seven kernel patch directories depending on your linux kernel version.
+In each directory there are patches to provide squashfs2.0 alpha release
+from a base kernel.  If your kernel version isn't listed then try the patch for
+the nearest kernel listed.
 
 To patch your kernel, cd into the top level directory, and run the "patch"
 comand, e.g. assuming linux-2.4.20
 
 %cd /usr/src/linux-2.4.20
-%patch -p1 < location-of-squashfs/linux-2.4.20/squashfs1.3r3-patch
+%patch -p1 < location-of-squashfs/linux-2.4.20/squashfs2.0-patch
 
-Where "location-of-squashfs" is the path to the squashfs1.3r3 source directory.
+Where "location-of-squashfs" is the path to the squashfs2.0-ALPHA source directory.
 
 The squashfs patches patch the relevant kernel files to add configure support,
 initrd support, include files, and the squashfs directory under linux/fs/.
 Once patched, the kernel must be reconfigured, with squashfs support turned on
 (either Y/M) to ensure that inflate support is built into the kernel.  The
 squashfs kernel option can be found in the filesystems submenu of the
-configure menus.  In the 2.6.1 kernel, the squashfs option is hiding in
+configure menus.  In the 2.6.x kernels, the squashfs option is hiding in
 the new miscellaneous filesystems submenu near the bottom of the filesystems
 submenu.
 
diff --git a/RELEASE-README b/RELEASE-README
index b919316..27b6bac 100644
--- a/RELEASE-README
+++ b/RELEASE-README
@@ -1,17 +1,18 @@
-	SQUASHFS 1.3r3 - A squashed read-only filesystem for Linux
+	SQUASHFS 2.0-ALPHA - A squashed read-only filesystem for Linux
 
-	Copyright 2004 Phillip Lougher (phillip@lougher.demon.co.uk)
+	Copyright 2004 Phillip Lougher (plougher@users.sourceforge.net)
 
 	Released under the GPL licence (version 2 or later).
 
-Squashfs is currently at version 1.3 release 3.  Please see the CHANGES file
-for recent changes to squashfs.
+Welcome to the alpha release of Squashfs version 2.0!  A lot of changes
+to the filesystem have been made under the bonnet (hood).  Please see
+the README-2.0 and CHANGES files for further details.
 
 Squashfs is a highly compressed read-only filesystem for Linux.
 It uses zlib compression to compress both files, inodes and directories.
 Inodes in the system are very small and all blocks are packed to minimise
 data overhead. Block sizes greater than 4K are supported up to a maximum
-of 32K.
+of 64K.
 
 Squashfs is intended for general read-only filesystem use, for archival
 use (i.e. in cases where a .tar.gz file may be used), and in constrained
@@ -39,8 +40,8 @@ devices and as normal files using the loopback device.
    (the exact length varies on file type, i.e. regular file, directory,
    symbolic link, and block/char device inodes have different sizes).
 
-5. Squashfs can use block sizes up to 32K (the default size is 32K).
-   Using 32K blocks achieves greater compression ratios than the normal
+5. Squashfs can use block sizes up to 64K (the default size is 64K).
+   Using 64K blocks achieves greater compression ratios than the normal
    4K block size.
 
 6. File duplicates are detected and removed.
@@ -385,7 +386,7 @@ cache.
 5. Author info
 --------------
 
-Squashfs was written by Phillip Lougher, email phillip@lougher.demon.co.uk,
+Squashfs was written by Phillip Lougher, email plougher@users.sourceforge.net,
 in Chepstow, Wales, UK.   If you like the program, or have any problems,
 then please email me, as it's nice to get feedback!
 
diff --git a/RELEASE-READMEs/README-2.0 b/RELEASE-READMEs/README-2.0
new file mode 100644
index 0000000..0ca1582
--- /dev/null
+++ b/RELEASE-READMEs/README-2.0
@@ -0,0 +1,136 @@
+	SQUASHFS 2.0-ALPHA - A squashed read-only filesystem for Linux
+
+	Copyright 2004 Phillip Lougher (plougher@users.sourceforge.net)
+
+	Released under the GPL licence (version 2 or later).
+
+Welcome to the alpha release of Squashfs version 2.0!  A lot of changes to the
+filesystem have been made under the bonnet (hood).  Squashfs 2.0 uses fragment
+blocks and larger blocks (64K) to improve compression ratio by about 5 - 20%
+over Squashfs 1.0 depending on the files being compressed.  Using fragment
+blocks allows Squashfs 2.0 to achieve better compression than cloop and similar
+compression to tgz files while retaining the I/O efficiency of a compressed
+filesystem.
+
+Detailed changes:
+
+1. Squashfs 2.0 has added the concept of fragment blocks (see later discussion).
+   Files smaller than the file block size (64K in Squashfs 2.0) and optionally the
+   remainder of files that do not fit fully into a block (i.e. the last 32K in
+   a 96K file) are packed into shared fragments and compressed together.  This
+   achieves on average 5 - 20% better compression than Squashfs 1.x.
+
+2. The maximum block size has been increased to 64K (in the ALPHA version of
+   Squashfs 2.0).
+
+3. The maximum number of UIDs has been increased to 256 (from 48 in 1.x).
+
+4. The maximum number of GIDs has been increased to 256 (from 15 in 1.x).
+
+5. Removal of sleep_on() function call in 2.6.x patch, to allow Squashfs
+   to work on the Fedora rc2 kernel.
+
+6. Numerous small bug fixes have been made.
+
+
+Compression improvements example
+--------------------------------
+
+The following is the compression results obtained compressing the 2.6.6
+linux kernel source using CRAMFS, Cloop (with iso filesystem), Squashfs 1.3 and
+Squashfs 2.0 (results generated using big-endian filesystems).
+
+In decreasing order of size:
+
+	CRAMFS		62791680 bytes (59.9M)
+	Squashfs 1.x	51351552 bytes (48.9M)
+	Cloop		46118681 bytes (44.0M)
+	Squashfs 2.0	45604854 bytes (43.5M)
+
+
+The Squashfs 1.x filesystem is 12.6% larger than the new 2.0 filesystem.
+The cloop filesystem is 1.1% larger than then Squashfs 2.0 filesystem.
+
+
+Fragment blocks in Squashfs 2.0
+-------------------------------
+
+Squashfs like all other compressed filesystems compresses files individually
+on a block by block basis.  This is performed to allow mounting and
+de-compression of files on a block by block basis without requiring the entire
+filesystem to be decompressed.  This is in contrast to data-based compression
+schemes which compress without understanding the underlying filesystem (i.e.
+cloop and tgz files) and which, therefore, do not compress files individually.
+Each approach has advantages and disadvantages, data-based systems have better
+compression because compression is always performed at the maximum block size
+(64K in cloop) irrespective of the size of each file (which could be less than
+the block size).  Compressed filesystems tend to be faster at I/O because
+they understand the filesystem and therefore employ better caching stategies
+and read less un-needed data from the filesystem.
+
+Fragment blocks in Squashfs 2.0 solves this problem by packing files (and
+optionally the ends of files) which are smaller than the block size into
+shared blocks, which are compressed together.  For example five files each of
+10K will be packed into one shared fragment of 50K and compressed together,
+rather than being compressed in five 10K blocks.
+
+This scheme produces a hybrid filesystem, retaining the I/O efficiency
+of a compressed filesystem, while obtaining the compression efficiency
+of data-based schemes by compressing small files together.
+
+
+New Squashfs 2.0 options
+------------------------
+
+-noF or -noFragmentCompression
+
+	Do not compress the fragments.  Added for compatibility with noI and
+	noD, probably not that useful.
+
+-no-fragments
+
+	Do not use fragment blocks, and rather generate a filesystem
+	similar to a Squashfs 1.x filesystem.  It will of course still
+	be a Squashfs 2.0 filesystem but without fragments, and so
+	it won't be mountable on a Squashfs 1.x system.
+
+-always-use-fragments
+
+	By default only small files less than the block size are packed into
+	fragment blocks.  The ends of files which do not fit fully into a block,
+	are NOT by default packed into fragments.  To illustrate this, a
+	100K file has an initial 64K block and a 36K remainder.  This
+	36K remainder is not packed into a fragment by default.  This is
+	because to do so leads to a 10 - 20% drop in sequential I/O
+	performance, as a disk head seek is needed to seek to the initial
+	file data and another disk seek is need to seek to the fragment
+	block.
+
+	Specify this option if you want file remainders to be packed into
+	fragment blocks.  Doing so may increase the compression obtained
+	BUT at the expense of I/O speed.
+
+-no-duplicates
+
+	Do not detect duplicate files.  Nothing to do with fragments but it is new
+	in 2.0 :)
+
+	
+Squashfs 1.x and Squashfs 2.0 compatibility
+-------------------------------------------
+
+Appending to Squashfs 1.x filesystems is not supported.  If you wish to append
+to 1.x filesystems, then either use the original mksquashfs, or convert them
+to Squashfs 2.0 by mounting the filesystem and running the 2.0 mksquashfs
+on the mounted filesystem.
+
+Mounting Squashfs 1.x filesystems IS supported by the 2.0 kernel patch.
+
+
+Limitations of the APLHA release
+--------------------------------
+
+The block size cannot be changed from 64K - because other block sizes
+have not been tested...
+
+