New issue
Advanced search Search tips

Issue 755201 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Oct 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature

Blocking:
issue 717785



Sign in to add a comment

Generate file map for a squashfs image

Project Member Reported by ahass...@chromium.org, Aug 14 2017

Issue description

We need to be able to identify the location of files and fragments inside a squashfs image along with the size of the compressed blocks for each file and fragment. This is necessary for the update engine to create efficient delta payloads for android container.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/3066535e641c8db874feea4a18eb4f989eced843

commit 3066535e641c8db874feea4a18eb4f989eced843
Author: Amin Hassani <ahassani@google.com>
Date: Fri Aug 18 01:42:26 2017

squashfs-tools: Generate squashfs image file map using unsquashfs

In order to generate a delta update for a squashfs file, we need to be
able to locate the files inside the image along with location of
compressed blocks. Generally squashfs breaks a file into same size
chunks and compresses them. It keeps the location of these compressed
blocks inside its metadata. This CL modifies unsquashfs to generate a
file map when given the flag -m or -map. It does not show the metadata
of the image. The metadata (including the superblock) can be inferred by
subtracting all files locations from the image itself.

The structure of the file map is a list of per-line following:

path start compressed_block_size_1 ... compressed_block_size_N

where:
- 'path' is the full path to the file.
- 'start' is the start byte address of the file.
- 'compressed_block_size_i' is a the length of the i'th compressed block
in bytes. If the block is uncompressed its 25th LSB is set.

It also first prints the list of all fragments along with their address
and block sizes at the beginning for the aforementioned list.

The usage:
unsquashfs -m(ap) <path_to_file_map> <squashfs_image>

BUG= chromium:755201 
TEST=unsquashfs -m filemap.txt 25-4k-gzip.img

Change-Id: I4898661bdfc776ac43dd3665f0ca109909000f9d
Reviewed-on: https://chromium-review.googlesource.com/619307
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/3066535e641c8db874feea4a18eb4f989eced843/sys-fs/squashfs-tools/files/squashfs-tools-4.3-file-map.patch
[rename] https://crrev.com/3066535e641c8db874feea4a18eb4f989eced843/sys-fs/squashfs-tools/squashfs-tools-4.3.ebuild
[add] https://crrev.com/3066535e641c8db874feea4a18eb4f989eced843/sys-fs/squashfs-tools/squashfs-tools-4.3-r3.ebuild

Comment 2 by zork@chromium.org, Aug 24 2017

Components: Internals>Installer
Project Member

Comment 3 by bugdroid1@chromium.org, Sep 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/b2b78226d325846b2569acc5e9cf8ad038aac812

commit b2b78226d325846b2569acc5e9cf8ad038aac812
Author: Amin Hassani <ahassani@google.com>
Date: Tue Sep 05 23:04:29 2017

generate_au_zip: Add unsquashfs

We are adding squashfs update support for android container and we need
our custom unsquashfs to generate the file map of the squashfs
image. This patch adds it to generate_au_zip.py.

BUG= 755201 
TEST=looked into the au-generator.zip for existence of unsquashfs.

Change-Id: I1d95d8ed3ddfa9645865422d101e5998e41cd5dd
Reviewed-on: https://chromium-review.googlesource.com/650518
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/b2b78226d325846b2569acc5e9cf8ad038aac812/build_library/generate_au_zip.py

Status: Verified (was: Started)

Sign in to add a comment