summaryrefslogtreecommitdiff
path: root/scripts/update_payload/checker.py
AgeCommit message (Collapse)Author
2020-10-29Merge remote-tracking branch 'remotes/aosp/upstream-master' into merge-crosAmin Hassani
Merge back the recent update_engine changes back to Android. Created by: $ git merge remotes/aosp/upstream-master --commit -s recursive No special conflict to resolve. Bug: 163153182 Test: None Change-Id: I4c65eb9c57448847857e2339935a5d47c8cb690a
2020-09-29update_payload: Add support for minor version 7Amin Hassani
Although, we have not upreved to minor version 7, some of the current canary images have been moved to minor version 7. So temporarily add support for minor version 7. BUG=b:169526824 TEST=sudo FEATURES=test emerge update_payload TEST=cros_generate_update_payload --debug --tgt-image chromiumos_test_image.bin --src-image chromiumos_test_image.bin --output delta.bin --check --work-dir workdir Change-Id: Ifb32307ecee7814190028f5ee7033f67fc5c40db Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/2437004 Reviewed-by: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org>
2020-09-28update_payload: Remove signature version checkingAmin Hassani
aosp/1130669 deprecated the 'version' field from the Signatures message. But the update_payload code wasn't updated, so this CL removes that check. BUG=b:169526824 TEST=cros_generate_update_payload --debug --tgt-image dlc.img --src-image dlc.img --output delta.bin --private-key ~/.ssh/testing_rsa --check Change-Id: I20aca13eeb33c5c1d2aa72513649198868ab00be Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/2434552 Tested-by: Amin Hassani <ahassani@chromium.org> Auto-Submit: Amin Hassani <ahassani@chromium.org> Reviewed-by: Jae Hoon Kim <kimjae@chromium.org> Commit-Queue: Jae Hoon Kim <kimjae@chromium.org>
2019-11-19update_payload: Port scripts to python3Andrew Lassalle
Update the update_payload scripts to be compatible with python2 and python3. Python2 compatibility is needed since the repo is shared with Android. BUG=chromium:1011631 TEST=Executed aosp/system/update_engine/scripts/run_unittests and cros_generate_update_payload Cq-Depend: chromium:1904837, chromium:1911499 Change-Id: Ie450b80b5f7550051b38d320173ccc0c915f65e7 Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1904310 Commit-Queue: Andrew Lassalle <andrewlassalle@chromium.org> Tested-by: Andrew Lassalle <andrewlassalle@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Auto-Submit: Andrew Lassalle <andrewlassalle@chromium.org>
2019-10-29update_engine: Deprecate major version 1Amin Hassani
We have moved away from major version 1 in Chrome OS and already have a stepping stone for it in M53. So this cleanup makes the code much easier to understand. BUG=chromium:1008553 TEST=FEATURES="test" sudo emerge update_engine update_payload TEST=cros_generate_update_payload --image chromiumos_test_image.bin --check --output delta.bin Change-Id: I01815dfa5fdf395f8214ef162e01ecca2d42f7fc Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1857459 Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org> Commit-Queue: Amin Hassani <ahassani@chromium.org>
2019-10-03update_engine: Deprecate minor version 1Amin Hassani
Minor version 1 was for the old days where we rewrite the signle partition with an update (no A/B partitions). But those days are long over and we don't think there is any device out that has this capability anymore. Even if there is, we can always serve full payloads along with the stepping stone we have in M53. So this is safe to go. BUG=chromium:1008553 TEST=sudo FEATURES=test emerge update_engine TEST=ran cros flash two times. Change-Id: Ib928ade36af5136cd4013a30dfb39ee7fd5b07b1 Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/1829160 Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org> Commit-Queue: Amin Hassani <ahassani@chromium.org>
2019-03-20update_payload: deprecate unused flags from paycheck.pyXiaochu Liu
Some temporary workaround are placed to hack the parameters to some default value that the rest of the code is happy with. This CL removes them. part_sizes should be default None (it was default [None, None]). when part_sizes is None, part sizes are not checked (same behavior as [None, None]). src_part_paths should be default None (it was default [None, None]). when src_part_paths is None, we set args.assert_type=_TYPE_FULL (same behavior as [None, None]). dst_part_paths (out_dst_part_paths) should be default None (it was default [None, None]). when dst_part_paths (and out_dst_part_paths) is None, dargs is not set (same behavior as [None, None]). out_dst_part_paths is rarely used and we often chose to use dst_part_path. Old flags (deprecated) are removed as well. BUG=chromium:926045 TEST=test_paycheck.sh chromeos_11844.0.0_kevin-arcnext_canary-channel_full_kevin-mp.bin-6f7f58b3c9a1a84ea15ab67f84cd5387.signed chromeos_11844.0.0-11869.0.0_kevin-arcnext_canary-channel_delta_kevin-mp.bin-fc6014025415e0b5e780c0739a3b0461.signed chromeos_11869.0.0_kevin-arcnext_canary-channel_full_kevin-mp.bin-1be744e0723534e02084c762dea316c0.signed Change-Id: Ifdf872ddfa03d5759a8ee4021e296e4dc4571d9c Reviewed-on: https://chromium-review.googlesource.com/1525289 Commit-Ready: Xiaochu Liu <xiaochu@chromium.org> Tested-by: Xiaochu Liu <xiaochu@chromium.org> Reviewed-by: Nicolas Norvez <norvez@chromium.org>
2019-01-10update_payload: uprev to minor version 6Amin Hassani
BUG=chromium:920699 TEST=unittest TEST=check_update_payload --check # with a delta payload Change-Id: I40bc1e535ed241b74c6d889757a8dcf046bac0de
2018-12-15update_payload: Fix problem with signature size on unsigned payloadsAmin Hassani
If the payload is unsigned (e.g. for test image), then the checker.sigs_file would not exist. So check for that variable before using it. BUG=chromium:794404 BUG=chromium:914705 TEST=manually signing the payload ran update_payload_check with signed and unsigned image TEST=unittests Change-Id: I871137eadef00d012ee926d12fd4eee36a454487 Reviewed-on: https://chromium-review.googlesource.com/1375023 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2018-12-10update_payload: Add the remaining major version 2 signature supportsAmin Hassani
This patch fixes the issues with signatures (sizes) in major version 2 and a few minor issues with the payload and metadata sizes. BUG=chromium:862679 TEST=manually signing the payload and running update_payload_check TEST=unittests Change-Id: I9b431379b0574a150474a913f1ec4a11e86288ae Reviewed-on: https://chromium-review.googlesource.com/1363339 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2018-07-11update_payload: Make paycheck take "root" instead of "rootfs"Tudor Brindus
This commit makes paycheck take "root" as the rootfs partition label instead of "rootfs", which makes it consistent with other tooling like delta_generator. BUG=chromium:794404 TEST=test_paycheck.sh and run_unittests Change-Id: I20835e93adbcde459bc86d832e695b38bca55e38 Reviewed-on: https://chromium-review.googlesource.com/1132843 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-06-26update_payload: Implement checking for major version 2 payloadsTudor Brindus
This commit adds payload major version 2 support to paycheck.py --check. For consistency, report messages for affected fields are kept the same across both major version 1 and 2 checks, even if the particular field name does not exist in one version. BUG=b:794404 TEST=no errors during run_unittests and paycheck.py --check <major version 2 payload> (./test_paycheck.sh does not pass for major version 2 payloads since applying is not implemented yet; no regressions when running on major version 1 payloads) Change-Id: I3c5d0cbca3336c8136326ca52b19f659c7c741c9 Reviewed-on: https://chromium-review.googlesource.com/1105610 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-06-25update_payload: Allow specifying partition options for major version 2Tudor Brindus
This commit adds the ability to specify partition options for more than just kernel/rootfs. This supersedes -p/--root-part-size, -P/--kern-part-size, --dst_kern, --dst_root, --src_kern, --src_root, --out_dst_kern, and --out_dst_root. They are replaced by --part_names used in conjunction with --part_sizes, --dst_part_paths, --src_part_paths, and --out_dst_part_paths. Backwards-compatibility with the old flags is kept, so long as they are not used alongside the new flags. BUG=b:794404 TEST=no errors during run_unittests and test_paycheck.sh Change-Id: Icc1118abbf89dd268be3eafe41723657c5178197 Reviewed-on: https://chromium-review.googlesource.com/1103063 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-06-21update_payload: De-duplicate paycheck partition handling for major version 2Tudor Brindus
This commit makes handling of major version 2 rootfs/kernel partitions more generic, with the goal of reusing the same code paths for major version 2 (which has an arbitrary number of partitions per payload). BUG=b:794404 TEST=no errors during run_unittests and test_paycheck.sh Change-Id: Ic6e8cdaae557c10dfbd302dba0e52ff8b2870c9f Reviewed-on: https://chromium-review.googlesource.com/1101500 Commit-Ready: Tudor Brindus <tbrindus@chromium.org> Tested-by: Tudor Brindus <tbrindus@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-06-04Merge remote-tracking branch 'cros/upstream' into cros/masterAmin Hassani
BUG=chromium:815356 TEST=unittest TEST=precq TEST=cros flash Change-Id: I8d9c37411708d0cae61613b285123a090ce6deb1
2018-04-04Correct some spellings.Sen Jiang
Test: mma Change-Id: Icc49a4ee76f12d302ed18982d334f2f70b7263a8
2018-03-23update_payload: Allow check for given metadata sizeAmin Hassani
Allow passing metadata size to check_update_payload so we can verify the metadata size in omaha equals to the one in the payload. BUG=chromium:820243 TEST=run paycheck.py with both valid and invalid metadata sizes reports as expected TEST=unittests Change-Id: Ib41ce77af77636fffec6752201c363e7fbbf868d Reviewed-on: https://chromium-review.googlesource.com/955679 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2018-02-08update_engine: Disable puffin in minor version 4Amin Hassani
This patch disables puffin for minor version 4, but keeps BROTLI_BSDIFF in the same minor version. Also adds puffin for minor version 5, but it does not enable it. BUG=b:72815313 BUG=chromium:810184 TEST=update_engine unittests pass TEST=update_payload unittests pass TEST=test_paycheck.sh on two images with minor version 4 pass TEST=brill_update_payload generate/verify pass Change-Id: I814fc126d67207501d419cff9c575cd13d877b48 Reviewed-on: https://chromium-review.googlesource.com/907223 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-01-30update_payload: migrate the license headers to AOSPAmin Hassani
update_engine/scripts/* have CROS license but the rest of UE have android license. This cl just migrates all CROS licenses to AOSP. BUG=None TEST=repo upload did not complain about the mismatched licenses. Change-Id: Id947be3e860baf7a523163965336f47d268c6808 Reviewed-on: https://chromium-review.googlesource.com/891688 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2018-01-23Relanding 'update_payload: Add XZ compression support'Amin Hassani
This patch adds support for checking a payload that has REPLACE_XZ operations. REPLACE_XZ was added in minor version 3. BUG=chromium:758792 TEST=unittests pass; paycheck.py with a xz generated payload pass; CQ-DEPEND=CL:823234 Change-Id: I6ec8068e233f2d595fda93a985923d85c59f150e Reviewed-on: https://chromium-review.googlesource.com/872124 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2018-01-05update_payload: Fix most of lint styling issues.Amin Hassani
This patch fixes a lot of pylint issues in the update_engine scripts. Majority of this changes are based on recommendation found in: https://www.chromium.org/chromium-os/python-style-guidelines It is a good idea to do these changes now, because if there are many pylint errors when performing 'repo upload', serious problems can be overshadowed by a lot of noise and eventually cause problems. These fixes include: - Fixing executable shebangs to /usr/bin/python2. - Fixing import-error problems by disabiling them. - Removing pylint disables that are not valid anymore. - Changing all imports to proper absolute import format. - Change the import of PayloadError from update_payload.PayloadError for simplicity. - Add pydoc strings for functions and classes that were missing. The remaining unchanged pylint problmes include: - The header files of these scripts are in CrOS copyright format, but the the cros lint hook is configured to AoSP copyright format. - The test* functions in unittests are not compatible with CamelCase format. BUG=chromium:796338 TEST=unittests pass TEST=start_devserver TEST=cros flash TEST=scripts/paycheck.py Change-Id: I7eed4d1625eb7c510c7949fada120de5a6a26c7b Reviewed-on: https://chromium-review.googlesource.com/834875 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-12-15Revert "update_payload: Add XZ compression support"Amin Hassani
This reverts commit f1d6ceaedbcf81044d6af6b716c63ed79996f0a4. Reason for revert: <canaries failing> Original change's description: > update_payload: Add XZ compression support > > This patch adds support for checking a payload that has REPLACE_XZ > operations. REPLACE_XZ was added in minor version 3. > > BUG=chromium:758792 > TEST=unittests pass; paycheck.py with a xz generated payload pass; > CQ-DEPEND=CL:823234 > > Change-Id: If82c767a201b189e464f459d5c19485e5278d9b1 > Reviewed-on: https://chromium-review.googlesource.com/823227 > Commit-Ready: Amin Hassani <ahassani@chromium.org> > Tested-by: Amin Hassani <ahassani@chromium.org> > Reviewed-by: Ben Chan <benchan@chromium.org> > Reviewed-by: Sen Jiang <senj@chromium.org> TBR=benchan@chromium.org,senj@chromium.org,ahassani@chromium.org Change-Id: I931ff3be081a41fe5fceef0e049ba4165c6acb49 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:795313 Reviewed-on: https://chromium-review.googlesource.com/830054 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-12-14update_payload: Add XZ compression supportAmin Hassani
This patch adds support for checking a payload that has REPLACE_XZ operations. REPLACE_XZ was added in minor version 3. BUG=chromium:758792 TEST=unittests pass; paycheck.py with a xz generated payload pass; CQ-DEPEND=CL:823234 Change-Id: If82c767a201b189e464f459d5c19485e5278d9b1 Reviewed-on: https://chromium-review.googlesource.com/823227 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-11-22update_engine: Adds BROTLI_BSDIFF operationAmin Hassani
Brotli compression creates on average 10%-20% smaller output than bzip2 in addition to having faster decompressor. With recent changes in bsdiff to compress the its patch with brotli, we can use it in the update_engine as a new operation BROTLI_BSDIFF. This operation will be turned on in minor version 4. However, this CL only adds support for it in the client. It will not generate BROTLI_BSDIFF operations yet. BUG=chromium:783437 TEST=unittests pass for both update_engine and update_payload; 'brillo_update_payload {generate|verify}' passes; 'scripts/paycheck.py payload.delta' passes; Change-Id: Ie791ba5431561c95de6fbc031a8196dbfd912288 Reviewed-on: https://chromium-review.googlesource.com/764791 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-10-24update_payload: Add zero operation supportAmin Hassani
Currenlty the applier.py and checker.py does not support ZERO operation. This patch adds support for it. In addition, when generating ZERO operation, we were not clearing the source length and extent properties out of the protobuf. ZERO operation has no source length or extent. BUG=chromium:768461 TEST=unittest pass; scripts/paycheck.py --check payload.delta; Change-Id: I766deaca4380686797893c2686036d59525546f4 Reviewed-on: https://chromium-review.googlesource.com/713539 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-10-16update_payload: Fix larger source issueAmin Hassani
the _CheckOperations() function passes the new usable partition size as the old usable partition size. If the source image is larger than the target image, this will be errornous as some checks fail. BUG=chromium:773336 TEST=scripts/paycheck.py --check delta.bin (10002.0.0 reef -> 10019.0.0 reef) Change-Id: I8c30129831daff8e70df3dcb9639ff240e4a37ba Reviewed-on: https://chromium-review.googlesource.com/713463 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2017-08-09update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatchAmin Hassani
Currently we never create imgdiff operation as it was never activated. In near future we will be adding puffdiff and puffpatch operations which replaces the imgdiff. This CL does not actually adds puffdiff, but adds a placeholder for it. BUG=none TEST=cros_workon_make --board=amd64-generic --test update_engine Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242 Reviewed-on: https://chromium-review.googlesource.com/602733 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2016-06-30paycheck: Support minor version 4.Sen Jiang
From https://chromium-review.googlesource.com/#/c/333160/ Bug: 27156473 Test: ./scripts/update_payload/checker_unittest.py Change-Id: I940debd5c878f622970e214fce75588f96d78407
2015-12-12paycheck: Allow minor version 3 in applier.Sen Jiang
Also fixed nits in checker. BUG=chromium:568473 TEST=./checker_unittest.py Change-Id: Ia791f4dc636e9c3a2921aeaa5f9dd01c247bf5bb Reviewed-on: https://chromium-review.googlesource.com/317780 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Sen Jiang <senj@chromium.org>
2015-12-11paycheck: support minor version 3.Sen Jiang
Added check for mandatory source operation hash in minor version 3. Fixed unittest. BUG=chromium:568473 TEST=./paycheck.py <minor v3 payload>; ./checker_unittest.py Change-Id: Id96ddce8c59f28b3449524b786c54a6c69ca798f Reviewed-on: https://chromium-review.googlesource.com/317573 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
2015-07-17paycheck: Properly infer usable target partition size.Gilad Arnold
The payload checker used to restrict read/write block indexes to the reported target filesystem size, unless explicitly given a partition size value to use instead. So far this value was easy for clients (like paygen) to come up with, because it was constant at 2GB for all known boards. However this is no longer the case, and there is no an easy way for clients to know the actual target partition size after the payload has been generated (nor is it encoded in the payload). This adds logic for inferring the usable target partition size into PayloadChecker() itself, as follows: 1) If a partition size was given, use that. 2) Else, if this is an old delta (minor version < 2), use the aforementioned default. This is necessary because older deltas may actually read/write data beyond the filesystem size. It is also sufficient because any old delta payload we generate should write to a 2GB target partition. 3) In all other cases, just use the new filesystem size, as encoded in the payload. This is a safe choice for full updates and newer deltas. The command-line tool is updated accordingly. Note that the usable kernel partition size inference remains unaffected. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: I987f28fdfe1d82d0f6f565ae9852b7b11bce13e8 Reviewed-on: https://chromium-review.googlesource.com/285447 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Improve minor_version checking.Gilad Arnold
1) We explicitly catch whether this field is not set. This means we might fail payloads generated by an old delta_generator, but ensures that we catch such a failure in current payload generation. Test logic slightly restructured to reduce duplication. 2) Slight changes to the checker method signature, for better uniformity with the rest of the code. This also lets us test that we actually read the minor_version field. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: Ib2d1999964ba892ef778ffc16bd1ca1c7d02bcd5 Reviewed-on: https://chromium-review.googlesource.com/285446 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-04-27paycheck: Check MOVE operations for start block 0.Allie Wood
paycheck now blocks payloads that read or write to block 0 in a MOVE operation. It must do this because old versions of PReadAll and PWriteAll could not seek to 0. BUG=chromium:480751 TEST=unit tests, test_paycheck.sh Change-Id: I042de444c92896496795af235e9386dca6c77be2 Reviewed-on: https://chromium-review.googlesource.com/267088 Reviewed-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-27update_payload: Move minor version checks to CheckOperation.Allie Wood
Various small fixes. Move checks for operation and minor version compatibility to CheckOperation. BUG=none TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I6abd0ec200c1d4d885b09dfb84ee7e6cfad4172c Reviewed-on: https://chromium-review.googlesource.com/254345 Reviewed-by: Allie Wood <alliewood@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-26update_payload: Add checks for new operations and minor version.Allie Wood
Paycheck now checks to make sure delta payloads with minor version 1 do not have SOURCE_COPY or SOURCE_BSDIFF operations and that payloads with minor version 2 do not have MOVE or BSDIFF operations. It also checks that the minor version given matches the payload type (delta or full). BUG=chromium:459799 TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I2a61e44760ae2b672015acdf8683501327b5d197 Reviewed-on: https://chromium-review.googlesource.com/253050 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-06paycheck: Allow payloads with no operations.Gilad Arnold
Due to a fix to the delta generator (CL:246670) we may actually end up with delta payloads that contain no operations (e.g. a delta from an image to itself). This is a good thing and we want paycheck to not rule out trivial deltas such as these. BUG=chromium:453659 TEST=Empty delta verification passes Change-Id: I461f48bba743a5b96b77aa3f55697ffc5cb8ea0e Reviewed-on: https://chromium-review.googlesource.com/246673 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-02-06paycheck: Fix format compliance errors.Gilad Arnold
BUG=None TEST=None Change-Id: I2d810bae961387a63495ba5bbe85b7222b5f86ef Reviewed-on: https://chromium-review.googlesource.com/246672 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-10-07paycheck: fix errors around percentage / formatting of zero sizesGilad Arnold
This fixes two problems, both having to do with histogram generation: * When the total number of elements is zero, paycheck would crash due to a division by zero; further, even if the crash is fixed (returning, say, None) the histogram will contain a meaningless value in parenthesis, which we might as well drop. Both are fixed here. * When some size (say, bytes) is zero, its formatter (bytes-to-human-readable) returns None, which shows up as is in the final report. This should be checked and avoided. BUG=None TEST=Crash fixed; None percentage/formatted value omitted. Change-Id: I8bb5fbc47e1cde9dcbee7f7b96bcb63ef3a0935e Reviewed-on: https://chromium-review.googlesource.com/172046 Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org>
2013-09-12paycheck: fixes to checker module (readability review)Gilad Arnold
BUG=None TEST=Passes unit tests TEST=Passes integration tests Change-Id: Ifd502af9d2755b2c23805cd03857ebbf0e633732 Reviewed-on: https://chromium-review.googlesource.com/59752 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-07-23paycheck: accommodate > 255 block reads/writesGilad Arnold
It turns out that, in some delta payloads, we're read/writing specific blocks as many as 270 times. Therefore, the unsigned char counters we've used for recording these accesses overflow and cause the execution to die unexpectedly (also failing paygen when that happens). This change increases the counter size to an unsigned short. It should be noted that the fact that we have such frequently accessed blocks through the course of an update is somewhat troubling by itself, and should be investigated. BUG=chromium:261417 TEST=Checking a previously failed payload now passes Change-Id: I47903cde9971c142cd33382b485b22a88a06b80f Reviewed-on: https://gerrit.chromium.org/gerrit/63040 Commit-Queue: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-06-10update_payload library: TODO + bug referencesGilad Arnold
BUG=chromium:243559 TEST=None Change-Id: I6837b87ff6f0b845ebbca97fdfa4190dc2d1282a Reviewed-on: https://gerrit.chromium.org/gerrit/57179 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org>
2013-05-22paycheck: move default pubkey handling inside the libraryGilad Arnold
This is a more sensible choice given that the pubkey ships within the library directory and hence should not be specified explicitly by an outside entity (like paycheck). From the practical standpoint, it makes this useful feature available to clients who use the library directly. BUG=chromium:241283 TEST=Unit + integration tests Change-Id: I059302326af1e0e394829466ee97ad2f60de4986 Reviewed-on: https://gerrit.chromium.org/gerrit/56335 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-14paycheck: allow src_extents to span the whole new partition sizeGilad Arnold
In a previous fix, we allowed paycheck to consider the whole new partition (beyond just the filesystem) as valid range for write operations. However, since updates are done in place, they should also be considered valid for read operations. This is now fixed. BUG=chromium:240726 TEST=Previously failing delta update now verified successfully Change-Id: Ib904c0265e2fd61ab0768169b3257c4131441888 Reviewed-on: https://gerrit.chromium.org/gerrit/51138 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-13paycheck: support for in-place BSDIFF operationsGilad Arnold
When applying BSDIFF operations in payloads, we used to extract the block sequences from the old partition and serialize it into a temporary file, and vice versa for the new partition. This worked but did not allow us to test bspatch as it is actually being used by the update engine. This CLs allows paycheck to invoke bspatch so that it reads/writes block extents directly from/to the partition file, in the same way it's being used by the update engine. Since performance is the same, this is the new default behavior; users can opt for the aforementioned old behavior using a command-line flag (-x or --extract-bsdiff). BUG=chromium:229705 TEST=bspatch invoked differently depending on the -x flag TEST=Passes unit/integration tests Change-Id: I8821754e1163b357617ece6befa42d1c2e575930 Reviewed-on: https://gerrit.chromium.org/gerrit/50486 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Darin Petkov <petkov@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-07paycheck: use base64 encoding for SHA256 hashes throughoutGilad Arnold
Turns out we were using base64 when adding SHA256 data to a payload report, but intermittently using hex encoding in error messages. This is now fixed. BUG=None TEST=No mention of hex encoding in the code TEST=gpylinted correctly TEST=Passes unit tests Change-Id: Id2dc2fcd154d4647cc1a076579dde7a789c09e40 Reviewed-on: https://gerrit.chromium.org/gerrit/50104 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: David James <davidjames@chromium.org>
2013-05-07paycheck: enforce physical partition size correctlyGilad Arnold
During payload checking, payload has wrongly interpreted the size reported in the update payload to be the physical partition size, whereas this is in fact the size of the filesystem portion only (a misnomer). This sometimes caused it to emit errors on out-of-bounds operations, which are otherwise harmless in real-world scenarios. This CL makes a clear distinction between the two, with the following semantics: - The payload's embedded filesystem size must by <= the physical partition sizes. - Reading/writing from/to the new partition must be within the physical partition size boundaries, and not the filesystem ones. - Reading from the old partition is only allowed from filesystem boundaries; this is unchanged from current behavior and appears to be consistent with how we perform delta updates. - Old/new SHA256 verification during payload application is now limited to the allotted filesystem portion only (and not the full partition size). This is consistent with the update engine's semantics. - Other than that, this change currently has no further effect on payload application, which remains more permissive wrt to partition sizes. This also means that the sizes of partitions resulting from a payload application will not necessarily abide by the predetermined physical partition sizes. This is in line with the prevailing division of responsibilities between payload checking (strict) and application (relaxed). BUG=chromium:221847 TEST=Payload checking respects partition size override TEST=Unit tests pass TEST=Integration tests pass Change-Id: I0dbc88d538c0cc53b7551f4dfa8f543bcf480cd5 Reviewed-on: https://gerrit.chromium.org/gerrit/50103 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: David James <davidjames@chromium.org>
2013-05-02paycheck: allow to disable specific checksGilad Arnold
This became necessary as the delta generator appears to generate payloads that fail certain checks (e.g. during update_engine unit testing). BUG=None TEST=Disabled checks not being triggered Change-Id: I4491e0cb32ef44f85e11ffb0402b40d1371525ae Reviewed-on: https://gerrit.chromium.org/gerrit/49676 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-04-05paycheck: unit tests + fixes to checker moduleGilad Arnold
This adds missing unit tests for the checker module, bundled with fixes to some bugs that surfaced due to unit tests. This includes: * A fake extent (signified by start_block == UINT64_MAX) that accompanies a signature data blob bears different requirements than previously implemented. Specifically, the extent sequence must have exactly one extent; and the number of blocks is not necessarily one, rather it is the correct number that corresponds to the actual length of the signature blob. * REPLACE/REPLACE_BZ operations must contain data. * MOVE operation validation must ensure that all of the actual message extents are being used. * BSDIFF operation must contain data (the diff). * Signature pseudo-operation should be a REPLACE. BUG=chromium-os:34911,chromium-os:33607,chromium-os:7597 TEST=Passes unittests (upcoming); works with actual payloads. Change-Id: I4d839d1d4da1fbb4a493b208958a139368e2c8ca Reviewed-on: https://gerrit.chromium.org/gerrit/45429 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-03-15Fix update out-of-bounds check for partition boundary.Gilad Arnold
This is an off-by-one error caught during the implementation of unit tests. BUG=None TEST=Unit tests (upcoming) pass as expected. Change-Id: I9a5da10588f0ac4314e9a35280ff1f38c34c88ac Reviewed-on: https://gerrit.chromium.org/gerrit/44953 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-03-08Update payload library + command-line toolGilad Arnold
An initial implementation of a Python module for parsing, checking and applying a Chrome OS update payload. Comes with a command-line tool (paycheck.py) for applying such operations on payload files, and a test script (test_paycheck.sh) for ensuring that the library and tool are working correctly. Since update_payload is introduced as a package, we're moving some previously merged utilities into the package's directory. (Unit testing for this code will be uploaded on a separate CL; see chromium-os:39663) BUG=chromium-os:34911,chromium-os:33607,chromium-os:7597 TEST=test_paycheck.sh successful on MP-signed payloads CQ-DEPEND=I5746a1d80e822a575f0d96f94d0b4e765fc64507 Change-Id: I77123a1fffbb2059c239b7145c6922968fdffb6a Reviewed-on: https://gerrit.chromium.org/gerrit/43041 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Jay Srinivasan <jaysri@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-04 05:26:23 +0000