summaryrefslogtreecommitdiff
path: root/scripts/update_payload/common.py
AgeCommit message (Collapse)Author
2019-11-19update_payload: Port scripts to python3Andrew Lassalle
Update the update_payload scripts to be compatible with python2 and python3. Python2 compatibility is needed since the repo is shared with Android. BUG=chromium:1011631 TEST=Executed aosp/system/update_engine/scripts/run_unittests and cros_generate_update_payload Cq-Depend: chromium:1904837, chromium:1911499 Change-Id: Ie450b80b5f7550051b38d320173ccc0c915f65e7 Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1904310 Commit-Queue: Andrew Lassalle <andrewlassalle@chromium.org> Tested-by: Andrew Lassalle <andrewlassalle@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Auto-Submit: Andrew Lassalle <andrewlassalle@chromium.org>
2019-10-29update_engine: Deprecate major version 1Amin Hassani
We have moved away from major version 1 in Chrome OS and already have a stepping stone for it in M53. So this cleanup makes the code much easier to understand. BUG=chromium:1008553 TEST=FEATURES="test" sudo emerge update_engine update_payload TEST=cros_generate_update_payload --image chromiumos_test_image.bin --check --output delta.bin Change-Id: I01815dfa5fdf395f8214ef162e01ecca2d42f7fc Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1857459 Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org> Commit-Queue: Amin Hassani <ahassani@chromium.org>
2019-10-03update_engine: Deprecate minor version 1Amin Hassani
Minor version 1 was for the old days where we rewrite the signle partition with an update (no A/B partitions). But those days are long over and we don't think there is any device out that has this capability anymore. Even if there is, we can always serve full payloads along with the stepping stone we have in M53. So this is safe to go. BUG=chromium:1008553 TEST=sudo FEATURES=test emerge update_engine TEST=ran cros flash two times. Change-Id: Ib928ade36af5136cd4013a30dfb39ee7fd5b07b1 Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1829160 Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org> Commit-Queue: Amin Hassani <ahassani@chromium.org>
2018-07-11update_payload: Make paycheck take "root" instead of "rootfs"Tudor Brindus
This commit makes paycheck take "root" as the rootfs partition label instead of "rootfs", which makes it consistent with other tooling like delta_generator. BUG=chromium:794404 TEST=test_paycheck.sh and run_unittests Change-Id: I20835e93adbcde459bc86d832e695b38bca55e38 Reviewed-on: https://chromium-review.googlesource.com/1132843 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-06-21update_payload: De-duplicate paycheck partition handling for major version 2Tudor Brindus
This commit makes handling of major version 2 rootfs/kernel partitions more generic, with the goal of reusing the same code paths for major version 2 (which has an arbitrary number of partitions per payload). BUG=b:794404 TEST=no errors during run_unittests and test_paycheck.sh Change-Id: Ic6e8cdaae557c10dfbd302dba0e52ff8b2870c9f Reviewed-on: https://chromium-review.googlesource.com/1101500 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-02-08update_engine: Disable puffin in minor version 4Amin Hassani
This patch disables puffin for minor version 4, but keeps BROTLI_BSDIFF in the same minor version. Also adds puffin for minor version 5, but it does not enable it. BUG=b:72815313 BUG=chromium:810184 TEST=update_engine unittests pass TEST=update_payload unittests pass TEST=test_paycheck.sh on two images with minor version 4 pass TEST=brill_update_payload generate/verify pass Change-Id: I814fc126d67207501d419cff9c575cd13d877b48 Reviewed-on: https://chromium-review.googlesource.com/907223 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-01-30update_payload: migrate the license headers to AOSPAmin Hassani
update_engine/scripts/* have CROS license but the rest of UE have android license. This cl just migrates all CROS licenses to AOSP. BUG=None TEST=repo upload did not complain about the mismatched licenses. Change-Id: Id947be3e860baf7a523163965336f47d268c6808 Reviewed-on: https://chromium-review.googlesource.com/891688 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-01-05update_payload: Fix most of lint styling issues.Amin Hassani
This patch fixes a lot of pylint issues in the update_engine scripts. Majority of this changes are based on recommendation found in: https://www.chromium.org/chromium-os/python-style-guidelines It is a good idea to do these changes now, because if there are many pylint errors when performing 'repo upload', serious problems can be overshadowed by a lot of noise and eventually cause problems. These fixes include: - Fixing executable shebangs to /usr/bin/python2. - Fixing import-error problems by disabiling them. - Removing pylint disables that are not valid anymore. - Changing all imports to proper absolute import format. - Change the import of PayloadError from update_payload.PayloadError for simplicity. - Add pydoc strings for functions and classes that were missing. The remaining unchanged pylint problmes include: - The header files of these scripts are in CrOS copyright format, but the the cros lint hook is configured to AoSP copyright format. - The test* functions in unittests are not compatible with CamelCase format. BUG=chromium:796338 TEST=unittests pass TEST=start_devserver TEST=cros flash TEST=scripts/paycheck.py Change-Id: I7eed4d1625eb7c510c7949fada120de5a6a26c7b Reviewed-on: https://chromium-review.googlesource.com/834875 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-11-22update_engine: Adds BROTLI_BSDIFF operationAmin Hassani
Brotli compression creates on average 10%-20% smaller output than bzip2 in addition to having faster decompressor. With recent changes in bsdiff to compress the its patch with brotli, we can use it in the update_engine as a new operation BROTLI_BSDIFF. This operation will be turned on in minor version 4. However, this CL only adds support for it in the client. It will not generate BROTLI_BSDIFF operations yet. BUG=chromium:783437 TEST=unittests pass for both update_engine and update_payload; 'brillo_update_payload {generate|verify}' passes; 'scripts/paycheck.py payload.delta' passes; Change-Id: Ie791ba5431561c95de6fbc031a8196dbfd912288 Reviewed-on: https://chromium-review.googlesource.com/764791 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-08-09update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatchAmin Hassani
Currently we never create imgdiff operation as it was never activated. In near future we will be adding puffdiff and puffpatch operations which replaces the imgdiff. This CL does not actually adds puffdiff, but adds a placeholder for it. BUG=none TEST=cros_workon_make --board=amd64-generic --test update_engine Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242 Reviewed-on: https://chromium-review.googlesource.com/602733 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2016-06-30paycheck: Support minor version 4.Sen Jiang
From https://chromium-review.googlesource.com/#/c/333160/ Bug: 27156473 Test: ./scripts/update_payload/checker_unittest.py Change-Id: I940debd5c878f622970e214fce75588f96d78407
2016-04-04update_payload: Regenerate protobuf to include IMGDIFF.Sen Jiang
Generated from update_metadata.proto using protoc 2.5.0 BUG=b:26456666 TEST=./checker_unittest.py Change-Id: I9dc7db00be8afb2f78e80bec910f9ca83a55a9a1 Reviewed-on: https://chromium-review.googlesource.com/329237 Commit-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org>
2015-12-12paycheck: Allow minor version 3 in applier.Sen Jiang
Also fixed nits in checker. BUG=chromium:568473 TEST=./checker_unittest.py Change-Id: Ia791f4dc636e9c3a2921aeaa5f9dd01c247bf5bb Reviewed-on: https://chromium-review.googlesource.com/317780 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Sen Jiang <senj@chromium.org>
2015-10-15Parse Payload v2 header.Alex Deymo
The update payload v2 contains an extra field in the header with the size of the metadata signatures and the metadata signatures stored right after the metadata. This patch parses the new payload format. BUG=b:22024447 TEST=cros payload show payload-v2.bin; served a payload v2 with devserver.py Change-Id: I8ce85af1df505f82f62a9d1cd57910cee6921f84 Reviewed-on: https://chromium-review.googlesource.com/306090 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-09-14paycheck: Update generated protobuf code.Alex Deymo
This patch updates the generated update_metadata_pb2.py file with the latest changes in the udpate_metadata.proto file. Some other changes in the update_payload library were required to match the changes in the .proto file. BUG=b:23179128 TEST=paycheck unittests Change-Id: I482d67d4a35f69438a26395eea77286994108b7a Reviewed-on: https://chromium-review.googlesource.com/299498 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Fix printing of operation index while tracing.Gilad Arnold
The block tracer is meant to scan operations in reverse order, to discover the latest operation that writes to a block. Strangely, it only reversed the operation indexes but scanned the actual operations in the original order, which is both incorrect in the general case, but even when it works the printed results are confusing (operations shown with the wrong index). This fixes it. Also some cosmetic changes to pacify the linter. BUG=chromium:510909 TEST=paycheck -B now prints the correct operation indexes. Change-Id: I65c44eeb450c229a2d5251737a0953716e124687 Reviewed-on: https://chromium-review.googlesource.com/286220 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-06-13update_payload: Remove ctypes dependency.Alex Deymo
ctypes requires libffi.so to be installed in the system, but old test_images don't have it, preventing "cros flash" to work there. This patch removes the need for ctypes from common.py so cros flash can work. BUG=None TEST=cros flash from R29 on link. Change-Id: Idd0732660780081c26375a5214167b53f625e3ed Reviewed-on: https://chromium-review.googlesource.com/277070 Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
2015-04-15update_payload: Add SOURCE operations to applier.Allie Wood
Add support for SOURCE_COPY and SOURCE_BSDIFF to paycheck by adding functions to apply these operations in applier. Also remove the source to destination partition copy in the applier when minor version is 2. Adds constants for source and inplace minor versions to common.py. BUG=chromium:461635 TEST=unit tests and ./test_paycheck.sh with sample payloads. CQ-DEPEND=CL:263747 Change-Id: I72d354d0609d205aab374dbdca6f30eb4de6a819 Reviewed-on: https://chromium-review.googlesource.com/264931 Tested-by: Allie Wood <alliewood@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org>
2015-02-20update_payload: Regenerate proto file with new operation codes.Allie Wood
Regenerate update_metadata_pb2.py to include SOURCE_COPY and SOURCE_BSDIFF, the new a to b operations for delta minor version 2. BUG=chromium:459363 TEST=unit tests and `cbuildbot --remote link-release --hwtest` Change-Id: I3631c7af97f57d48f86a34c37f97ec8c3cef088d Reviewed-on: https://chromium-review.googlesource.com/250954 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-11-12paycheck: strip newlines off of hash digest stringsGilad Arnold
This is an annoyance I'd like to get rid of. BUG=None TEST=None Change-Id: I6119163ffc4944dd2f857bad055822b37229a692 Reviewed-on: https://chromium-review.googlesource.com/176478 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-07paycheck: enforce physical partition size correctlyGilad Arnold
During payload checking, payload has wrongly interpreted the size reported in the update payload to be the physical partition size, whereas this is in fact the size of the filesystem portion only (a misnomer). This sometimes caused it to emit errors on out-of-bounds operations, which are otherwise harmless in real-world scenarios. This CL makes a clear distinction between the two, with the following semantics: - The payload's embedded filesystem size must by <= the physical partition sizes. - Reading/writing from/to the new partition must be within the physical partition size boundaries, and not the filesystem ones. - Reading from the old partition is only allowed from filesystem boundaries; this is unchanged from current behavior and appears to be consistent with how we perform delta updates. - Old/new SHA256 verification during payload application is now limited to the allotted filesystem portion only (and not the full partition size). This is consistent with the update engine's semantics. - Other than that, this change currently has no further effect on payload application, which remains more permissive wrt to partition sizes. This also means that the sizes of partitions resulting from a payload application will not necessarily abide by the predetermined physical partition sizes. This is in line with the prevailing division of responsibilities between payload checking (strict) and application (relaxed). BUG=chromium:221847 TEST=Payload checking respects partition size override TEST=Unit tests pass TEST=Integration tests pass Change-Id: I0dbc88d538c0cc53b7551f4dfa8f543bcf480cd5 Reviewed-on: https://gerrit.chromium.org/gerrit/50103 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: David James <davidjames@chromium.org>
2013-04-05paycheck: unit tests + fixes to checker moduleGilad Arnold
This adds missing unit tests for the checker module, bundled with fixes to some bugs that surfaced due to unit tests. This includes: * A fake extent (signified by start_block == UINT64_MAX) that accompanies a signature data blob bears different requirements than previously implemented. Specifically, the extent sequence must have exactly one extent; and the number of blocks is not necessarily one, rather it is the correct number that corresponds to the actual length of the signature blob. * REPLACE/REPLACE_BZ operations must contain data. * MOVE operation validation must ensure that all of the actual message extents are being used. * BSDIFF operation must contain data (the diff). * Signature pseudo-operation should be a REPLACE. BUG=chromium-os:34911,chromium-os:33607,chromium-os:7597 TEST=Passes unittests (upcoming); works with actual payloads. Change-Id: I4d839d1d4da1fbb4a493b208958a139368e2c8ca Reviewed-on: https://gerrit.chromium.org/gerrit/45429 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-03-29Modify paycheck to print out hashes in base64.Chris Sosa
Both the update engine and sha256_partitions.sh print out hashes in base64 encoding. Let's stay consistent with ourselves so we can diagnose hash mismatches correctly. BUG=None TEST=Ran it with a delta payload and compared hashes with sha256_partitions.sh and logs from applying it in the update_engine. Change-Id: I90c511b936792a073fbe069065ff53c24f47041c Reviewed-on: https://gerrit.chromium.org/gerrit/46910 Commit-Queue: Chris Sosa <sosa@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Tested-by: Chris Sosa <sosa@chromium.org>
2013-03-08Update payload library + command-line toolGilad Arnold
An initial implementation of a Python module for parsing, checking and applying a Chrome OS update payload. Comes with a command-line tool (paycheck.py) for applying such operations on payload files, and a test script (test_paycheck.sh) for ensuring that the library and tool are working correctly. Since update_payload is introduced as a package, we're moving some previously merged utilities into the package's directory. (Unit testing for this code will be uploaded on a separate CL; see chromium-os:39663) BUG=chromium-os:34911,chromium-os:33607,chromium-os:7597 TEST=test_paycheck.sh successful on MP-signed payloads CQ-DEPEND=I5746a1d80e822a575f0d96f94d0b4e765fc64507 Change-Id: I77123a1fffbb2059c239b7145c6922968fdffb6a Reviewed-on: https://gerrit.chromium.org/gerrit/43041 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Jay Srinivasan <jaysri@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-03 14:48:45 +0000