summaryrefslogtreecommitdiff
path: root/scripts/update_payload
AgeCommit message (Collapse)Author
2017-12-15Revert "update_payload: Add XZ compression support"Amin Hassani
This reverts commit f1d6ceaedbcf81044d6af6b716c63ed79996f0a4. Reason for revert: <canaries failing> Original change's description: > update_payload: Add XZ compression support > > This patch adds support for checking a payload that has REPLACE_XZ > operations. REPLACE_XZ was added in minor version 3. > > BUG=chromium:758792 > TEST=unittests pass; paycheck.py with a xz generated payload pass; > CQ-DEPEND=CL:823234 > > Change-Id: If82c767a201b189e464f459d5c19485e5278d9b1 > Reviewed-on: https://chromium-review.googlesource.com/823227 > Commit-Ready: Amin Hassani <ahassani@chromium.org> > Tested-by: Amin Hassani <ahassani@chromium.org> > Reviewed-by: Ben Chan <benchan@chromium.org> > Reviewed-by: Sen Jiang <senj@chromium.org> TBR=benchan@chromium.org,senj@chromium.org,ahassani@chromium.org Change-Id: I931ff3be081a41fe5fceef0e049ba4165c6acb49 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:795313 Reviewed-on: https://chromium-review.googlesource.com/830054 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-12-14update_payload: Add XZ compression supportAmin Hassani
This patch adds support for checking a payload that has REPLACE_XZ operations. REPLACE_XZ was added in minor version 3. BUG=chromium:758792 TEST=unittests pass; paycheck.py with a xz generated payload pass; CQ-DEPEND=CL:823234 Change-Id: If82c767a201b189e464f459d5c19485e5278d9b1 Reviewed-on: https://chromium-review.googlesource.com/823227 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-12-01update_payload: Add puffin binary path to paycheck.pyAmin Hassani
We forgot to add the path to the puffin binary to paycheck.py and payload.py and this caused the canary builds fail for minor version 4. This patch fixes it. BUG=chromium:791079 TEST=tryjob Change-Id: I194ab9d9d0b93fc9d456b994f73424d7dedbf181 Reviewed-on: https://chromium-review.googlesource.com/803656 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
2017-11-22update_engine: Adds BROTLI_BSDIFF operationAmin Hassani
Brotli compression creates on average 10%-20% smaller output than bzip2 in addition to having faster decompressor. With recent changes in bsdiff to compress the its patch with brotli, we can use it in the update_engine as a new operation BROTLI_BSDIFF. This operation will be turned on in minor version 4. However, this CL only adds support for it in the client. It will not generate BROTLI_BSDIFF operations yet. BUG=chromium:783437 TEST=unittests pass for both update_engine and update_payload; 'brillo_update_payload {generate|verify}' passes; 'scripts/paycheck.py payload.delta' passes; Change-Id: Ie791ba5431561c95de6fbc031a8196dbfd912288 Reviewed-on: https://chromium-review.googlesource.com/764791 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org>
2017-10-31update_payload: Add puffdiff support.Amin Hassani
This patch adds support for applier.py to be able to apply PUFFDIFF operation using puffin binary. It also fixes minor deficiencies for the PUFFDIFF operation. BUG=chromium:768461 TEST=unittests pass; TEST=scripts/paycheck.py payload-puffin.delta new_kern.dat new_root.dat old_kern.dat old_root.dat Change-Id: I5612ead8d8b8176e7263cfcb15403e8d36319642 Reviewed-on: https://chromium-review.googlesource.com/713540 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-10-24update_payload: Add zero operation supportAmin Hassani
Currenlty the applier.py and checker.py does not support ZERO operation. This patch adds support for it. In addition, when generating ZERO operation, we were not clearing the source length and extent properties out of the protobuf. ZERO operation has no source length or extent. BUG=chromium:768461 TEST=unittest pass; scripts/paycheck.py --check payload.delta; Change-Id: I766deaca4380686797893c2686036d59525546f4 Reviewed-on: https://chromium-review.googlesource.com/713539 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-10-16update_payload: Fix larger source issueAmin Hassani
the _CheckOperations() function passes the new usable partition size as the old usable partition size. If the source image is larger than the target image, this will be errornous as some checks fail. BUG=chromium:773336 TEST=scripts/paycheck.py --check delta.bin (10002.0.0 reef -> 10019.0.0 reef) Change-Id: I8c30129831daff8e70df3dcb9639ff240e4a37ba Reviewed-on: https://chromium-review.googlesource.com/713463 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org>
2017-09-30update_device.py: support OTA zip if use omaha too.Sen Jiang
Pass the offset and size to UpdateHandler. Also added support for payload offset in update_payload.Payload. Test: applied a local OTA Change-Id: Ib116ef2c23a11e298118f203814c4ea8dd1629af
2017-09-30Fix pylint issue in payload.pySen Jiang
************* Module update_payload.payload W: 12, 0: Relative import 'applier', should be 'update_payload.applier' (relative-import) W: 13, 0: Relative import 'block_tracer', should be 'update_payload.block_tracer' (relative-import) W: 14, 0: Relative import 'checker', should be 'update_payload.checker' (relative-import) W: 15, 0: Relative import 'common', should be 'update_payload.common' (relative-import) W: 16, 0: Relative import 'error', should be 'update_payload.error' (relative-import) W: 17, 0: Relative import 'update_metadata_pb2', should be 'update_payload.update_metadata_pb2' (relative-import) C:222, 4: Missing function docstring (missing-docstring) E:261,33: Instance of 'DeltaArchiveManifest' has no 'partitions' member (no-member) Test: repo upload Change-Id: Ib3e99c6a3a5ea4575b442e672a871adc7b985289
2017-08-23update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatchAmin Hassani
Currently we never create imgdiff operation as it was never activated. In near future we will be adding puffdiff and puffpatch operations which replaces the imgdiff. This CL does not actually adds puffdiff, but adds a placeholder for it. BUG=none TEST=cros_workon_make --board=amd64-generic --test update_engine Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242 Reviewed-on: https://chromium-review.googlesource.com/602733 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2017-08-09update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatchAmin Hassani
Currently we never create imgdiff operation as it was never activated. In near future we will be adding puffdiff and puffpatch operations which replaces the imgdiff. This CL does not actually adds puffdiff, but adds a placeholder for it. BUG=none TEST=cros_workon_make --board=amd64-generic --test update_engine Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242 Reviewed-on: https://chromium-review.googlesource.com/602733 Commit-Ready: Amin Hassani <ahassani@chromium.org> Tested-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Ben Chan <benchan@chromium.org> Reviewed-by: Sen Jiang <senj@chromium.org>
2016-06-30paycheck: Support minor version 4.Sen Jiang
From https://chromium-review.googlesource.com/#/c/333160/ Bug: 27156473 Test: ./scripts/update_payload/checker_unittest.py Change-Id: I940debd5c878f622970e214fce75588f96d78407
2016-04-04update_payload: Regenerate protobuf to include IMGDIFF.Sen Jiang
Generated from update_metadata.proto using protoc 2.5.0 BUG=b:26456666 TEST=./checker_unittest.py Change-Id: I9dc7db00be8afb2f78e80bec910f9ca83a55a9a1 Reviewed-on: https://chromium-review.googlesource.com/329237 Commit-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org>
2015-12-12paycheck: Allow minor version 3 in applier.Sen Jiang
Also fixed nits in checker. BUG=chromium:568473 TEST=./checker_unittest.py Change-Id: Ia791f4dc636e9c3a2921aeaa5f9dd01c247bf5bb Reviewed-on: https://chromium-review.googlesource.com/317780 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Sen Jiang <senj@chromium.org>
2015-12-11paycheck: support minor version 3.Sen Jiang
Added check for mandatory source operation hash in minor version 3. Fixed unittest. BUG=chromium:568473 TEST=./paycheck.py <minor v3 payload>; ./checker_unittest.py Change-Id: Id96ddce8c59f28b3449524b786c54a6c69ca798f Reviewed-on: https://chromium-review.googlesource.com/317573 Trybot-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
2015-11-17update_payload: Fix IsDelta to support payload v2.Sen Jiang
Added check for the old_partition_info field which is used by major version 2 payload. Other changes are needed in order to pass presubmit lint check. BUG=None TEST=Run devserver with a v2 delta payload. Change-Id: I3d7057fde44bd24d768177d5ed5c36bc2d4ca94b Reviewed-on: https://chromium-review.googlesource.com/312952 Commit-Ready: Sen Jiang <senj@chromium.org> Tested-by: Sen Jiang <senj@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-10-15Parse Payload v2 header.Alex Deymo
The update payload v2 contains an extra field in the header with the size of the metadata signatures and the metadata signatures stored right after the metadata. This patch parses the new payload format. BUG=b:22024447 TEST=cros payload show payload-v2.bin; served a payload v2 with devserver.py Change-Id: I8ce85af1df505f82f62a9d1cd57910cee6921f84 Reviewed-on: https://chromium-review.googlesource.com/306090 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-09-24paycheck: Update generated protobuf code.Alex Deymo
This patch updates the generated update_metadata_pb2.py file with the latest changes in the udpate_metadata.proto file. BUG=None TEST=paycheck unittests Change-Id: Ibba40eb94535496654447c23895677cacb69f26b Reviewed-on: https://chromium-review.googlesource.com/302191 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-09-15update_payload: Regenerate protobuf with protoc 2.5Alex Deymo
Goobuntu trusty uses protobuf 2.5.0 python modules, but the chroot uses version 2.6.1. Protobufs compiled with protoc 2.6.1 (inside the chroot) are not compatible with the python modules installed in trusty, so they can only run inside the chroot. Nevertheless, protobufs compiled with protoc 2.5.0 work with the new library version. BUG=chromium:531884 TEST=`./paycheck.py` inside and outside the chroot with libprotobuf 2.5.0 Change-Id: Ifeaf203a9c891156dd487249673a8264c74fd7e3 Reviewed-on: https://chromium-review.googlesource.com/299814 Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org>
2015-09-14paycheck: Update generated protobuf code.Alex Deymo
This patch updates the generated update_metadata_pb2.py file with the latest changes in the udpate_metadata.proto file. Some other changes in the update_payload library were required to match the changes in the .proto file. BUG=b:23179128 TEST=paycheck unittests Change-Id: I482d67d4a35f69438a26395eea77286994108b7a Reviewed-on: https://chromium-review.googlesource.com/299498 Commit-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2015-07-22paycheck: Small improvements to the block tracer utility.Gilad Arnold
This prepends the current block number to the output, simplifies some logic, and tightens argument validation in the command-line parser. BUG=None TEST=paycheck -B/-b works. Change-Id: I90d5cdf721612cdd12e49f4e4181849fc699807f Reviewed-on: https://chromium-review.googlesource.com/286547 Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-07-17paycheck: Fix printing of operation index while tracing.Gilad Arnold
The block tracer is meant to scan operations in reverse order, to discover the latest operation that writes to a block. Strangely, it only reversed the operation indexes but scanned the actual operations in the original order, which is both incorrect in the general case, but even when it works the printed results are confusing (operations shown with the wrong index). This fixes it. Also some cosmetic changes to pacify the linter. BUG=chromium:510909 TEST=paycheck -B now prints the correct operation indexes. Change-Id: I65c44eeb450c229a2d5251737a0953716e124687 Reviewed-on: https://chromium-review.googlesource.com/286220 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-07-17paycheck: Disambiguate hash error messages.Gilad Arnold
Before we've been emitting the exact same error message when failing to verify the hash of either pre- or post-update partition. This prefixes the partition name with "old" or "new", respectively. BUG=chromium:510909 TEST=None Change-Id: Icf302d0a3b26d3f9860c4ccb03dba38493296794 Reviewed-on: https://chromium-review.googlesource.com/286143 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Properly infer usable target partition size.Gilad Arnold
The payload checker used to restrict read/write block indexes to the reported target filesystem size, unless explicitly given a partition size value to use instead. So far this value was easy for clients (like paygen) to come up with, because it was constant at 2GB for all known boards. However this is no longer the case, and there is no an easy way for clients to know the actual target partition size after the payload has been generated (nor is it encoded in the payload). This adds logic for inferring the usable target partition size into PayloadChecker() itself, as follows: 1) If a partition size was given, use that. 2) Else, if this is an old delta (minor version < 2), use the aforementioned default. This is necessary because older deltas may actually read/write data beyond the filesystem size. It is also sufficient because any old delta payload we generate should write to a 2GB target partition. 3) In all other cases, just use the new filesystem size, as encoded in the payload. This is a safe choice for full updates and newer deltas. The command-line tool is updated accordingly. Note that the usable kernel partition size inference remains unaffected. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: I987f28fdfe1d82d0f6f565ae9852b7b11bce13e8 Reviewed-on: https://chromium-review.googlesource.com/285447 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Improve minor_version checking.Gilad Arnold
1) We explicitly catch whether this field is not set. This means we might fail payloads generated by an old delta_generator, but ensures that we catch such a failure in current payload generation. Test logic slightly restructured to reduce duplication. 2) Slight changes to the checker method signature, for better uniformity with the rest of the code. This also lets us test that we actually read the minor_version field. BUG=chromium:508566 TEST=Unit tests (revised) Change-Id: Ib2d1999964ba892ef778ffc16bd1ca1c7d02bcd5 Reviewed-on: https://chromium-review.googlesource.com/285446 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-07-17paycheck: Fix linter errors.Gilad Arnold
All whitespace related. BUG=None TEST=None Change-Id: I18694b7f3f86b3970de31fa0d366560234767632 Reviewed-on: https://chromium-review.googlesource.com/286142 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-07-15Fix linter errors.Gilad Arnold
BUG=None TEST=None Change-Id: I65434c5e7d00006b89d71aadb5a4721ee16ff289 Reviewed-on: https://chromium-review.googlesource.com/285445 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-06-13update_payload: Remove ctypes dependency.Alex Deymo
ctypes requires libffi.so to be installed in the system, but old test_images don't have it, preventing "cros flash" to work there. This patch removes the need for ctypes from common.py so cros flash can work. BUG=None TEST=cros flash from R29 on link. Change-Id: Idd0732660780081c26375a5214167b53f625e3ed Reviewed-on: https://chromium-review.googlesource.com/277070 Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
2015-04-27paycheck: Check MOVE operations for start block 0.Allie Wood
paycheck now blocks payloads that read or write to block 0 in a MOVE operation. It must do this because old versions of PReadAll and PWriteAll could not seek to 0. BUG=chromium:480751 TEST=unit tests, test_paycheck.sh Change-Id: I042de444c92896496795af235e9386dca6c77be2 Reviewed-on: https://chromium-review.googlesource.com/267088 Reviewed-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-04-15update_payload: Add SOURCE operations to applier.Allie Wood
Add support for SOURCE_COPY and SOURCE_BSDIFF to paycheck by adding functions to apply these operations in applier. Also remove the source to destination partition copy in the applier when minor version is 2. Adds constants for source and inplace minor versions to common.py. BUG=chromium:461635 TEST=unit tests and ./test_paycheck.sh with sample payloads. CQ-DEPEND=CL:263747 Change-Id: I72d354d0609d205aab374dbdca6f30eb4de6a819 Reviewed-on: https://chromium-review.googlesource.com/264931 Tested-by: Allie Wood <alliewood@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org>
2015-04-06update_payload: Fix checker unittests.Allie Wood
Checker unittests failed when fail_bad_type=True in DoCheckOperationsTest. When the operation was added (in AddOperation), a ValueError was raised because an invalid enum value (for op.type) was given. For now, fail_bad_type tests have been removed. BUG=None TEST=`./checker_unittest.py` Change-Id: I19220e295c576b4ddc6f427dedd4bc037908483d Reviewed-on: https://chromium-review.googlesource.com/264005 Reviewed-by: Allie Wood <alliewood@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-27update_payload: Move minor version checks to CheckOperation.Allie Wood
Various small fixes. Move checks for operation and minor version compatibility to CheckOperation. BUG=none TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I6abd0ec200c1d4d885b09dfb84ee7e6cfad4172c Reviewed-on: https://chromium-review.googlesource.com/254345 Reviewed-by: Allie Wood <alliewood@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-26update_payload: Add checks for new operations and minor version.Allie Wood
Paycheck now checks to make sure delta payloads with minor version 1 do not have SOURCE_COPY or SOURCE_BSDIFF operations and that payloads with minor version 2 do not have MOVE or BSDIFF operations. It also checks that the minor version given matches the payload type (delta or full). BUG=chromium:459799 TEST=`./checker_unittest.py` and running paycheck on payloads. Change-Id: I2a61e44760ae2b672015acdf8683501327b5d197 Reviewed-on: https://chromium-review.googlesource.com/253050 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Trybot-Ready: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-26update_payload: Remove assertIsNone.Alex Deymo
unittest.assertIsNone was introduced in python 2.7 but at the time our servers were running an older python version. We have migrated to python 2.7 so we can drop the local wrapper now. BUG=None TEST=ran unittest. Change-Id: I1c43d027361ab7f8f5fad83cc900361e5a6ee137 Reviewed-on: https://chromium-review.googlesource.com/253610 Commit-Queue: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-02-20update_payload: Regenerate proto file with new operation codes.Allie Wood
Regenerate update_metadata_pb2.py to include SOURCE_COPY and SOURCE_BSDIFF, the new a to b operations for delta minor version 2. BUG=chromium:459363 TEST=unit tests and `cbuildbot --remote link-release --hwtest` Change-Id: I3631c7af97f57d48f86a34c37f97ec8c3cef088d Reviewed-on: https://chromium-review.googlesource.com/250954 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2015-02-09update_payload: Regenerate protobuf file.Allie Wood
Regenerate update_metadata_pb2.py so that it includes minor_version field. BUG=None TEST=unit tests and `cbuildbot --remote link-release --hwtest` Change-Id: I05a09f2e9afa424aeeb7791982040211c0152957 Reviewed-on: https://chromium-review.googlesource.com/247162 Reviewed-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Allie Wood <alliewood@chromium.org> Tested-by: Allie Wood <alliewood@chromium.org>
2015-02-06paycheck: Allow payloads with no operations.Gilad Arnold
Due to a fix to the delta generator (CL:246670) we may actually end up with delta payloads that contain no operations (e.g. a delta from an image to itself). This is a good thing and we want paycheck to not rule out trivial deltas such as these. BUG=chromium:453659 TEST=Empty delta verification passes Change-Id: I461f48bba743a5b96b77aa3f55697ffc5cb8ea0e Reviewed-on: https://chromium-review.googlesource.com/246673 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2015-02-06paycheck: Fix format compliance errors.Gilad Arnold
BUG=None TEST=None Change-Id: I2d810bae961387a63495ba5bbe85b7222b5f86ef Reviewed-on: https://chromium-review.googlesource.com/246672 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-11-12paycheck: strip newlines off of hash digest stringsGilad Arnold
This is an annoyance I'd like to get rid of. BUG=None TEST=None Change-Id: I6119163ffc4944dd2f857bad055822b37229a692 Reviewed-on: https://chromium-review.googlesource.com/176478 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-10-07paycheck: fix errors around percentage / formatting of zero sizesGilad Arnold
This fixes two problems, both having to do with histogram generation: * When the total number of elements is zero, paycheck would crash due to a division by zero; further, even if the crash is fixed (returning, say, None) the histogram will contain a meaningless value in parenthesis, which we might as well drop. Both are fixed here. * When some size (say, bytes) is zero, its formatter (bytes-to-human-readable) returns None, which shows up as is in the final report. This should be checked and avoided. BUG=None TEST=Crash fixed; None percentage/formatted value omitted. Change-Id: I8bb5fbc47e1cde9dcbee7f7b96bcb63ef3a0935e Reviewed-on: https://chromium-review.googlesource.com/172046 Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org>
2013-09-12paycheck: fixes to checker module (readability review)Gilad Arnold
BUG=None TEST=Passes unit tests TEST=Passes integration tests Change-Id: Ifd502af9d2755b2c23805cd03857ebbf0e633732 Reviewed-on: https://chromium-review.googlesource.com/59752 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-08-26paycheck: allow to pass an explicit path to bspatchGilad Arnold
The bspatch binary is used when applying update payloads. By default, we were using whatever bspatch that was found via path expansion in os.execvp, however there are cases where we want to be specific about where the bspatch binary is that we need to be using (such as during paygen runs). BUG=chromium:277072 TEST=Non-default bspatch binary used Change-Id: I85ffd28aeb26938cbf5ea428fa97d29af0353a7d Reviewed-on: https://gerrit.chromium.org/gerrit/66736 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-07-23paycheck: accommodate > 255 block reads/writesGilad Arnold
It turns out that, in some delta payloads, we're read/writing specific blocks as many as 270 times. Therefore, the unsigned char counters we've used for recording these accesses overflow and cause the execution to die unexpectedly (also failing paygen when that happens). This change increases the counter size to an unsigned short. It should be noted that the fact that we have such frequently accessed blocks through the course of an update is somewhat troubling by itself, and should be investigated. BUG=chromium:261417 TEST=Checking a previously failed payload now passes Change-Id: I47903cde9971c142cd33382b485b22a88a06b80f Reviewed-on: https://gerrit.chromium.org/gerrit/63040 Commit-Queue: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
2013-06-10update_payload library: TODO + bug referencesGilad Arnold
BUG=chromium:243559 TEST=None Change-Id: I6837b87ff6f0b845ebbca97fdfa4190dc2d1282a Reviewed-on: https://gerrit.chromium.org/gerrit/57179 Commit-Queue: Gilad Arnold <garnold@chromium.org> Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org>
2013-06-03paycheck/applier: stop using fallocateGilad Arnold
It appears that fallocate is not supported on certain filesystems (e.g. failing on paygen genati instances). It also appears that we can do just fine without it when applying a full update, without significant impact on performance. In the interest of making the update_payload library more portable, we therefore drop the fallocate call and replace it with a simple create/truncate code (all Python standard lib calls). BUG=chromium:244091 TEST=Integration tests Change-Id: I57f7dec19e8131050c1d56d891e486714d4f7128 Reviewed-on: https://gerrit.chromium.org/gerrit/56762 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Matt Tennant <mtennant@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-24paycheck: truncate partitions resulting from applying a payloadGilad Arnold
Since the correctness of the result only encompasses the filesystem (or otherwise "meaningful data") on the target partition, it is desirable to actually get rid of whatever is past that point. There are different reasons for the presence of such extra space in delta updates, including remnants from a source partition that served as baseline for a delta update, or scratch space used by MOVE operations for breaking cycles. We make truncation the default behavior, although it can be suppressed by passing the right flag (truncate_to_expected_size=False). Note that this change is necessary for comparing the results of applying a payload to the partitions as they are extracted from a target image, which is to be attempted during payload generation. This also fixes tiny gpylint complaints. BUG=chromium:241283 TEST=Emitted partition files truncated as expected Change-Id: Ibb71e4f2305ec41224afdc503168ae02c312f6fe Reviewed-on: https://gerrit.chromium.org/gerrit/56532 Tested-by: Gilad Arnold <garnold@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org> Reviewed-by: Gilad Arnold <garnold@chromium.org>
2013-05-22paycheck: move stuff from checker_unittest.py to test_utils.pyGilad Arnold
This is all stuff that'll come in handy when testing other modules. BUG=chromium:217039,chromium:215780,chromium:189855 TEST=Passes unittests Change-Id: I57d1fae97330f22748885f8028352f07b5058396 Reviewed-on: https://gerrit.chromium.org/gerrit/47615 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-22paycheck: move default pubkey handling inside the libraryGilad Arnold
This is a more sensible choice given that the pubkey ships within the library directory and hence should not be specified explicitly by an outside entity (like paycheck). From the practical standpoint, it makes this useful feature available to clients who use the library directly. BUG=chromium:241283 TEST=Unit + integration tests Change-Id: I059302326af1e0e394829466ee97ad2f60de4986 Reviewed-on: https://gerrit.chromium.org/gerrit/56335 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
2013-05-15Update paycheck to understand the updated update_metadata fields.Don Garrett
When checking a payload, always start by printing a description of the payload being checked. Recompiled (with protoc) the updated update_metadata.proto from the update_engine. BUG=chromium:226310 TEST=Manual Run CQ-DEPEND=CL:47347 Change-Id: Ib60c6e6978d30444db7b65ef6d09540c9ffacdb8 Reviewed-on: https://gerrit.chromium.org/gerrit/50899 Commit-Queue: Don Garrett <dgarrett@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Tested-by: Don Garrett <dgarrett@chromium.org>
2013-05-14paycheck: allow src_extents to span the whole new partition sizeGilad Arnold
In a previous fix, we allowed paycheck to consider the whole new partition (beyond just the filesystem) as valid range for write operations. However, since updates are done in place, they should also be considered valid for read operations. This is now fixed. BUG=chromium:240726 TEST=Previously failing delta update now verified successfully Change-Id: Ib904c0265e2fd61ab0768169b3257c4131441888 Reviewed-on: https://gerrit.chromium.org/gerrit/51138 Tested-by: Gilad Arnold <garnold@chromium.org> Reviewed-by: Chris Sosa <sosa@chromium.org> Commit-Queue: Gilad Arnold <garnold@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-03 20:10:51 +0000