For the Android delta updater we need to support parsing squashfs container into individual files before creating install operations. Currently we are treading the entire squashfs container as one file.
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/5421ba9104027d9cfa66da1f2bb4dd65c0f710b1 commit 5421ba9104027d9cfa66da1f2bb4dd65c0f710b1 Author: Amin Hassani <ahassani@google.com> Date: Tue Sep 26 20:36:08 2017 update_engine: Adds dependency to puffin BUG= chromium:767120 TEST=sudo emerge update_engine CQ-DEPEND=CL:451881 Change-Id: I66c43d9618c53af62d4a9ee76810678bf62fa00a Reviewed-on: https://chromium-review.googlesource.com/683157 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/5421ba9104027d9cfa66da1f2bb4dd65c0f710b1/chromeos-base/update_engine/update_engine-9999.ebuild
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d3a3079874c08a3c630f8ee46c5471cc8b800736 commit d3a3079874c08a3c630f8ee46c5471cc8b800736 Author: Amin Hassani <ahassani@google.com> Date: Sat Oct 07 04:31:04 2017 update_engine: Add dependency to sys-fs/squashfs-tools This patch adds squashfs-tools dependency to update_engine because we need unsquashfs binary for payload generation and also unittests. Also since we don't plan to include squashfs-tools/squashfs_fs.h anymore, we can just not export it. BUG= chromium:767120 TEST=precq passes CQ-DEPEND=CL:698948 Change-Id: I4e8f6a4e73f7ff481b109abfcb1d7b4673b9345e Reviewed-on: https://chromium-review.googlesource.com/686034 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/d3a3079874c08a3c630f8ee46c5471cc8b800736/chromeos-base/update_engine/update_engine-9999.ebuild [rename] https://crrev.com/d3a3079874c08a3c630f8ee46c5471cc8b800736/sys-fs/squashfs-tools/squashfs-tools-4.3-r6.ebuild [modify] https://crrev.com/d3a3079874c08a3c630f8ee46c5471cc8b800736/sys-fs/squashfs-tools/squashfs-tools-4.3.ebuild
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/d7da8f4118c10d00d02b6274edd2f88971a6cb22 commit d7da8f4118c10d00d02b6274edd2f88971a6cb22 Author: Amin Hassani <ahassani@google.com> Date: Sat Oct 07 04:31:04 2017 update_engine: Add Squashfs (phase 1) Currently the Android container in CrOS is a Squashfs image (around 500MB) and the update engine treats it as one single file. This patch uses -map flag in 'unsquashfs' to get the location of files inside the Squashfs image. and creates a list of files that can be used in payload generator to create a delta. This patch adds support for both Squashfs file system and its appropriate unittests. However, we do not activate it in this patch, further patches will activate and use it in the payload generator. BUG= chromium:767120 TEST=cros_workon_make --board=amd64-generic --test update_engine; CQ-DEPEND=CL:650518,CL:686034 Change-Id: I02179727cd736661704d0ca2d9f881754b332b36 Reviewed-on: https://chromium-review.googlesource.com/636105 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> [modify] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/sample_images/sample_images.tar.bz2 [add] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/payload_generator/squashfs_filesystem.cc [modify] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/update_engine.gyp [modify] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/Android.mk [add] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/payload_generator/squashfs_filesystem.h [modify] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/sample_images/generate_images.sh [add] https://crrev.com/d7da8f4118c10d00d02b6274edd2f88971a6cb22/payload_generator/squashfs_filesystem_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/924183b4ec3a66a36e681b2737472cf7bc6b6d68 commit 924183b4ec3a66a36e681b2737472cf7bc6b6d68 Author: Amin Hassani <ahassani@google.com> Date: Mon Oct 16 21:14:02 2017 update_engine: Add Squashfs (phase 2) This patch replaces the Android container related Squashfs images with their files in the update engine. The location of each new file is defined as: 'Squashfs image file path' + / + 'the file path inside the Squashfs' This works even if the files inside the Squashfs are not 4k aligned. But we need 4k alignment before we add Puffin support. In some experiments this could even reduce the delta payload size by like 1MB. BUG= chromium:767120 TEST=cros_workon_make --board=amd64-generic --test update_engine; brillo_update_payload generate --payload=payload.delta --source_image=veyron_minnie-R61-9765.13.0.bin --target_image=veyron-minnie-R62-9804.0.0.bin Change-Id: I3fbcc5cd0d724435955ed1bd1204dee0e41944be Reviewed-on: https://chromium-review.googlesource.com/636106 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> [add] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/payload_generator/deflate_utils_unittest.cc [add] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/payload_generator/deflate_utils.cc [add] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/payload_generator/deflate_utils.h [modify] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/Android.mk [modify] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/update_engine.gyp [modify] https://crrev.com/924183b4ec3a66a36e681b2737472cf7bc6b6d68/payload_generator/delta_diff_utils.cc
The following revision refers to this bug: https://chromium.googlesource.com/aosp/platform/system/update_engine/+/3cd4df127c29eb90c0e203373edfbc9534c79169 commit 3cd4df127c29eb90c0e203373edfbc9534c79169 Author: Amin Hassani <ahassani@google.com> Date: Tue Nov 07 01:26:17 2017 update_engine: Generate PUFFDIFF operation This patch adds functionality for generating PUFFDIFF operation, but since the minor verion has not been increase, it actually does not generate the operation itself. PUFFDIFF is used for patching deflate based files. BUG= chromium:767120 TEST=unittest passes; brillo_update_payload {generate|verify} passes; CQ-DEPEND=CL:726945 Change-Id: I4ffdac8dce3740ef9fc2156cb0dad83a8364c8b5 Reviewed-on: https://chromium-review.googlesource.com/658298 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/filesystem_interface.h [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/squashfs_filesystem.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/delta_diff_utils.h [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/deflate_utils_unittest.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/deflate_utils.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/deflate_utils.h [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/Android.mk [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/delta_diff_utils_unittest.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/update_engine.gyp [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/squashfs_filesystem.h [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/squashfs_filesystem_unittest.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/inplace_generator.cc [modify] https://crrev.com/3cd4df127c29eb90c0e203373edfbc9534c79169/payload_generator/delta_diff_utils.cc
Comment 1 by ahass...@chromium.org
, Sep 26 2017