summaryrefslogtreecommitdiff
path: root/scripts/update_payload/checker_unittest.py
AgeCommit message (Collapse)Author
2017-11-22update_engine: Adds BROTLI_BSDIFF operationAmin Hassani
Brotli compression creates on average 10%-20% smaller output than bzip2 in addition to having faster decompressor. With recent changes in bsdiff to compress the its patch with brotli, we can use it in the update_engine as a new operation BROTLI_BSDIFF. This operation will be turned on in minor version 4. However, this CL only adds support for it in the client. It will not generate BROTLI_BSDIFF operations yet. BUG=chromium:783437 TEST=unittests pass for both update_engine and update_payload; 'brillo_update_payload {generate|verify}' passes; 'scripts/paycheck.py payload.delta' passes; Change-Id: Ie791ba5431561c95de6fbc031a8196dbfd912288 Reviewed-on: https://chromium-review.googlesource.com/764791 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-10-31update_payload: Add puffdiff support.Amin Hassani
This patch adds support for applier.py to be able to apply PUFFDIFF operation using puffin binary. It also fixes minor deficiencies for the PUFFDIFF operation. BUG=chromium:768461 TEST=unittests pass; TEST=scripts/paycheck.py payload-puffin.delta new_kern.dat new_root.dat old_kern.dat old_root.dat Change-Id: I5612ead8d8b8176e7263cfcb15403e8d36319642 Reviewed-on: https://chromium-review.googlesource.com/713540 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-10-24update_payload: Add zero operation supportAmin Hassani
Currenlty the applier.py and checker.py does not support ZERO operation. This patch adds support for it. In addition, when generating ZERO operation, we were not clearing the source length and extent properties out of the protobuf. ZERO operation has no source length or extent. BUG=chromium:768461 TEST=unittest pass; scripts/paycheck.py --check payload.delta; Change-Id: I766deaca4380686797893c2686036d59525546f4 Reviewed-on: https://chromium-review.googlesource.com/713539 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-10-16update_payload: Fix larger source issueAmin Hassani
the _CheckOperations() function passes the new usable partition size as the old usable partition size. If the source image is larger than the target image, this will be errornous as some checks fail. BUG=chromium:773336 TEST=scripts/paycheck.py --check delta.bin (10002.0.0 reef -> 10019.0.0 reef) Change-Id: I8c30129831daff8e70df3dcb9639ff240e4a37ba Reviewed-on: https://chromium-review.googlesource.com/713463 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2017-08-09update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatchAmin Hassani
Currently we never create imgdiff operation as it was never activated. In near future we will be adding puffdiff and puffpatch operations which replaces the imgdiff. This CL does not actually adds puffdiff, but adds a placeholder for it. BUG=none TEST=cros_workon_make --board=amd64-generic --test update_engine Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242 Reviewed-on: https://chromium-review.googlesource.com/602733 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2016-06-30paycheck: Support minor version 4.Sen Jiang
From https://chromium-review.googlesource.com/#/c/333160/ Bug: 27156473 Test: ./scripts/update_payload/checker_unittest.py Change-Id: I940debd5c878f622970e214fce75588f96d78407
2016-04-04update_payload: Regenerate protobuf to include IMGDIFF.Sen Jiang
Generated from update_metadata.proto using protoc 2.5.0 BUG=b:26456666 TEST=./checker_unittest.py Change-Id: I9dc7db00be8afb2f78e80bec910f9ca83a55a9a1 Reviewed-on: https://chromium-review.googlesource.com/329237 Commit-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org>
2015-12-11paycheck: support minor version 3.Sen Jiang
Added check for mandatory source operation hash in minor version 3. Fixed unittest. BUG=chromium:568473 TEST=./paycheck.py <minor v3 payload>; ./checker_unittest.py Change-Id: Id96ddce8c59f28b3449524b786c54a6c69ca798f Reviewed-on: https://chromium-review.googlesource.com/317573 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
2015-09-14paycheck: Update generated protobuf code.Alex Deymo
This patch updates the generated update_metadata_pb2.py file with the latest changes in the udpate_metadata.proto file. Some other changes in the update_payload library were required to match the changes in the .proto file. BUG=b:23179128 TEST=paycheck unittests Change-Id: I482d67d4a35f69438a26395eea77286994108b7a Reviewed-on: https://chromium-review.googlesource.com/299498 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Properly infer usable target partition size.Gilad Arnold
The payload checker used to restrict read/write block indexes to the reported target filesystem size, unless explicitly given a partition size value to use instead. So far this value was easy for clients (like paygen) to come up with, because it was constant at 2GB for all known boards. However this is no longer the case, and there is no an easy way for clients to know the actual target partition size after the payload has been generated (nor is it encoded in the payload). This adds logic for inferring the usable target partition size into PayloadChecker() itself, as follows: 1) If a partition size was given, use that. 2) Else, if this is an old delta (minor version < 2), use the aforementioned default. This is necessary because older deltas may actually read/write data beyond the filesystem size. It is also sufficient because any old delta payload we generate should write to a 2GB target partition. 3) In all other cases, just use the new filesystem size, as encoded in the payload. This is a safe choice for full updates and newer deltas. The command-line tool is updated accordingly. Note that the usable kernel partition size inference remains unaffected. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: I987f28fdfe1d82d0f6f565ae9852b7b11bce13e8 Reviewed-on: https://chromium-review.googlesource.com/285447 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Improve minor_version checking.Gilad Arnold
1) We explicitly catch whether this field is not set. This means we might fail payloads generated by an old delta_generator, but ensures that we catch such a failure in current payload generation. Test logic slightly restructured to reduce duplication. 2) Slight changes to the checker method signature, for better uniformity with the rest of the code. This also lets us test that we actually read the minor_version field. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: Ib2d1999964ba892ef778ffc16bd1ca1c7d02bcd5 Reviewed-on: https://chromium-review.googlesource.com/285446 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-04-27paycheck: Check MOVE operations for start block 0.Allie Wood
paycheck now blocks payloads that read or write to block 0 in a MOVE operation. It must do this because old versions of PReadAll and PWriteAll could not seek to 0. BUG=chromium:480751 TEST=unit tests, test_paycheck.sh Change-Id: I042de444c92896496795af235e9386dca6c77be2 Reviewed-on: https://chromium-review.googlesource.com/267088 Reviewed-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-04-06update_payload: Fix checker unittests.Allie Wood
Checker unittests failed when fail_bad_type=True in DoCheckOperationsTest. When the operation was added (in AddOperation), a ValueError was raised because an invalid enum value (for op.type) was given. For now, fail_bad_type tests have been removed. BUG=None TEST=`./checker_unittest.py` Change-Id: I19220e295c576b4ddc6f427dedd4bc037908483d Reviewed-on: https://chromium-review.googlesource.com/264005 Reviewed-by: Allie Wood <alliewood@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-27update_payload: Move minor version checks to CheckOperation.Allie Wood
Various small fixes. Move checks for operation and minor version compatibility to CheckOperation. BUG=none TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I6abd0ec200c1d4d885b09dfb84ee7e6cfad4172c Reviewed-on: https://chromium-review.googlesource.com/254345 Reviewed-by: Allie Wood <alliewood@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-26update_payload: Add checks for new operations and minor version.Allie Wood
Paycheck now checks to make sure delta payloads with minor version 1 do not have SOURCE_COPY or SOURCE_BSDIFF operations and that payloads with minor version 2 do not have MOVE or BSDIFF operations. It also checks that the minor version given matches the payload type (delta or full). BUG=chromium:459799 TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I2a61e44760ae2b672015acdf8683501327b5d197 Reviewed-on: https://chromium-review.googlesource.com/253050 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-26update_payload: Remove assertIsNone.Alex Deymo
unittest.assertIsNone was introduced in python 2.7 but at the time our servers were running an older python version. We have migrated to python 2.7 so we can drop the local wrapper now. BUG=None TEST=ran unittest. Change-Id: I1c43d027361ab7f8f5fad83cc900361e5a6ee137 Reviewed-on: https://chromium-review.googlesource.com/253610 Commit-Queue: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-02-06paycheck: Allow payloads with no operations.Gilad Arnold
Due to a fix to the delta generator (CL:246670) we may actually end up with delta payloads that contain no operations (e.g. a delta from an image to itself). This is a good thing and we want paycheck to not rule out trivial deltas such as these. BUG=chromium:453659 TEST=Empty delta verification passes Change-Id: I461f48bba743a5b96b77aa3f55697ffc5cb8ea0e Reviewed-on: https://chromium-review.googlesource.com/246673 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-02-06paycheck: Fix format compliance errors.Gilad Arnold
BUG=None TEST=None Change-Id: I2d810bae961387a63495ba5bbe85b7222b5f86ef Reviewed-on: https://chromium-review.googlesource.com/246672 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-09-12paycheck: fixes to checker module (readability review)Gilad Arnold
BUG=None TEST=Passes unit tests TEST=Passes integration tests Change-Id: Ifd502af9d2755b2c23805cd03857ebbf0e633732 Reviewed-on: https://chromium-review.googlesource.com/59752 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-22paycheck: move stuff from checker_unittest.py to test_utils.pyGilad Arnold
This is all stuff that'll come in handy when testing other modules. BUG=chromium:217039,chromium:215780,chromium:189855 TEST=Passes unittests Change-Id: I57d1fae97330f22748885f8028352f07b5058396 Reviewed-on: https://gerrit.chromium.org/gerrit/47615 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-07paycheck: enforce physical partition size correctlyGilad Arnold
During payload checking, payload has wrongly interpreted the size reported in the update payload to be the physical partition size, whereas this is in fact the size of the filesystem portion only (a misnomer). This sometimes caused it to emit errors on out-of-bounds operations, which are otherwise harmless in real-world scenarios. This CL makes a clear distinction between the two, with the following semantics: - The payload's embedded filesystem size must by <= the physical partition sizes. - Reading/writing from/to the new partition must be within the physical partition size boundaries, and not the filesystem ones. - Reading from the old partition is only allowed from filesystem boundaries; this is unchanged from current behavior and appears to be consistent with how we perform delta updates. - Old/new SHA256 verification during payload application is now limited to the allotted filesystem portion only (and not the full partition size). This is consistent with the update engine's semantics. - Other than that, this change currently has no further effect on payload application, which remains more permissive wrt to partition sizes. This also means that the sizes of partitions resulting from a payload application will not necessarily abide by the predetermined physical partition sizes. This is in line with the prevailing division of responsibilities between payload checking (strict) and application (relaxed). BUG=chromium:221847 TEST=Payload checking respects partition size override TEST=Unit tests pass TEST=Integration tests pass Change-Id: I0dbc88d538c0cc53b7551f4dfa8f543bcf480cd5 Reviewed-on: https://gerrit.chromium.org/gerrit/50103 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: David James <davidjames@chromium.org>
2013-05-02paycheck: allow to disable specific checksGilad Arnold
This became necessary as the delta generator appears to generate payloads that fail certain checks (e.g. during update_engine unit testing). BUG=None TEST=Disabled checks not being triggered Change-Id: I4491e0cb32ef44f85e11ffb0402b40d1371525ae Reviewed-on: https://gerrit.chromium.org/gerrit/49676 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-04-05paycheck: unit tests + fixes to checker moduleGilad Arnold
This adds missing unit tests for the checker module, bundled with fixes to some bugs that surfaced due to unit tests. This includes: * A fake extent (signified by start_block == UINT64_MAX) that accompanies a signature data blob bears different requirements than previously implemented. Specifically, the extent sequence must have exactly one extent; and the number of blocks is not necessarily one, rather it is the correct number that corresponds to the actual length of the signature blob. * REPLACE/REPLACE_BZ operations must contain data. * MOVE operation validation must ensure that all of the actual message extents are being used. * BSDIFF operation must contain data (the diff). * Signature pseudo-operation should be a REPLACE. BUG=chromium-os:34911,chromium-os:33607,chromium-os:7597 TEST=Passes unittests (upcoming); works with actual payloads. Change-Id: I4d839d1d4da1fbb4a493b208958a139368e2c8ca Reviewed-on: https://gerrit.chromium.org/gerrit/45429 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-04 14:04:02 +0000