summaryrefslogtreecommitdiff
path: root/payload_consumer
AgeCommit message (Collapse)Author
2021-07-27Don't check for FEC offset if fec is disabledKelvin Zhang
Test: th Bug: 194661607 Bug: 194779085 Change-Id: I0e37136313914f1ee9a4eae0e5db59807adc7dc5 Merged-In: I0e37136313914f1ee9a4eae0e5db59807adc7dc5
2021-05-13Fix some lint/style errorKelvin Zhang
Test: th Change-Id: I57a36f8f6254d3e57a4787f1a7d3cc3368bbd7a7
2021-05-12Write verity first, then do fs verificationKelvin Zhang
Old behavior: Read partition, for each block: Update hasher Update verity writer before reading hashtree/verity: write hashtree/verity to disk Read the last verity blocks. Finalize hasher, verity hashes. The old bahvior tries to minimize fs read by only read once and feed data to hasher and verity writer. However, in VABC, reading/writing are handled very differently. Read can be done via regular fd, but writes must go through special COW API. As we have seen in b/186196758, using COW API in filesystem hashing can lead to inconsistent read and boot failure. Therefore, we've decided to write verity first using COW API, then read/hash partition using regular fd. This does mean that we need to read everything twice, but we think this is a worth while tradeoff. As verity writes can take 5 minutes, but reading the entire partition again only takes <10 seconds. New behavior: Read partition, for each block: Update verity writer Finalize verity writer, write verity to disk launch snapuserd, open a regular fd. Read partition, for each block: Update hasher Finaliaze hasher, verity hashes. Test: th Test: Manual testing on pixel of the following scenario: 1. Verity enabled, VABC enabled, pause/resume multiple times 2. Verity disabled, VABC enabled, pause/resume multiple times 3. Verity Enabled, VABC enabled, pause/resume multiple times Bug: 186196758 Change-Id: I2477c2dc4da5b921e84b48a54d0d8a877c1a52ef
2021-05-11Refactor get partition path and IsVABC into separate functionKelvin Zhang
Test: th Change-Id: Ifb6efa1e63f5d5a047d18ac0876a1ecd860ba869
2021-05-11Add a case to cover repeatedelly running fs verificationKelvin Zhang
b/186196758 could be detected by keep running fs verification. The 2nd attempt will cause verity data to be discarded, but since data is still visible during the 2nd attempt, bug is only visible after device reboot, or running fs verification for the 3rd time. Bug: 186196758 Test: th Change-Id: I7415665fa030b68acc3903499750702d8df5626e
2021-05-11Add more unittest for fs verification VABC behaviorKelvin Zhang
Test: th Change-Id: I7db1874cffdacf93bbee8243dc45bb1bcc8b04ee
2021-05-06Create a minimal testcase to reproduce silent verity corruptionKelvin Zhang
b/186196758 is triggered by the following sequence of events: 1. update_engine finish writing all install ops, emits kEndOfInstall label 2. update_engine opens cow in append mode, invokes InitialiazeAppend(kEndOfInstall) 3. update_engine writes verity data, invokes SnapshotWriter::Finalize() 4. update_engine repeats step 2, but does not write any data after opening SnapshotWriter. Instead, it reads verity and make sure the hash matches what's specified in OTA payload. 5. Reboot device, verity data corrupted, device rollback to slot _a. This is because, during step 4, when calling InitializeAppend(kEndOfInstall), the SnapshotWriter only reads up to the given label. But OpenReader() completely disregards the resume label and reads all ops. Therefore, update_engine sees the verity data, and determines that everything is fine. However, when calling SnapshotWriter::Finalize(), data after resume label are discarded, therefore verity data is gone. Test: th Bug: 186196758 Change-Id: I0166271b64eb7b574434d617ce730f345ca93ff1
2021-05-06Fix verity discarded bugKelvin Zhang
If update_engine opens CowWriterFileDescriptor w/o writing anything, data past the resume label is readable while fd is open, but will be discarded once the fd is closed. Such "phantom read" causes inconsistency. This CL contains two changes to address the above bug: 1. When device reboots after update, all I/O are served by snapuserd. update_engine should use snapuserd for verification to emulate bahvior of device after reboot. 2. When a CowWriterFd is opened, don't call Finalize() if no verity is written. Since past-the-end data is discarded when we call Finalize() Test: th Bug: 186196758 Change-Id: Ia1d31b671c16fded7319677fe0397f1288457201
2021-05-05Rename postinstall_mount_device to readonly_target_pathKelvin Zhang
When postinstall_mount_device is initially introduced, it's only intended to be used by postinstall action, hence the name. Now we plan to use it for fs verification purpose as well, rename for better clarity. Test: th Change-Id: Iff996f2f513bb44694e39d758a69851793b9a565
2021-04-27Unmmap all partitions after postinstallKelvin Zhang
Previous code will only unmap if postinstall finishes successfully, move call to Unmap() to top of cleanup function. Test: th Change-Id: Ibb8b5cd82dbed46da1ed32a3ec370f1acbb42890
2021-04-20Refactor extent writer to take filedescriptor in constructorKelvin Zhang
Functions which receive an instance of extent writer need to manually pass fd to ExtentWriter via Init() call, which breaks separation of concerns. It makes it hard for us to decouple InstallOp execution from writing of data, as the execution unit must be aware of which fd to pass to extent writer. In addition, many extents writer, such as snapshot extent writer, simply ignores the fd parameter, which is a indication of poor code structure. To address the above issue, we pass FileDescriptorPtr via constructor if needed. This way, whoever is "executing" InstallOps don't need to care about where the output data is going, and whoever's writing the data would be responsible for initializing an ExtentWriter. Test: th Change-Id: I6d1eabde085eefd55da9ecc0352d4a16ae458698
2021-03-31Add a unittest for read-after-write pattern in cow writerKelvin Zhang
Test: th Change-Id: I4e461b03d4008d484eafe601d3de2f4b06bf585d
2021-03-30Allow /postinstall files to have custom contextsAlex Light
We were mounting /postinstall with a 'context=...' option. This forces all files within /postinstall to have a single selinux context, limiting the possible granularity of our policies. Here we change it to simply default to the 'postinstall_file' context for the 'system' partition but allow individual files to have their own custom contexts defined by /system/sepolicy. Other partitions retain the single 'postinstall_file' context. The sample_images were updated to manually add a selinux label for testing FS contexts. Test: Manual OTA of blueline Test: atest update_engine_unittests Bug: 181182967 Change-Id: I0b8c2b2228fa08afecb64da9c276737eb9ae3631 Merged-In: I0b8c2b2228fa08afecb64da9c276737eb9ae3631
2021-03-28Support verity writes in VABCKelvin Zhang
Test: generate an OTA with verity enabled, install it Bug: 173432386 Change-Id: I14cccb1bb339c9824a95b8e42ac5144cb8b75c3f
2021-03-24Make dynamic partition control android return a writable fdKelvin Zhang
We can return a FileDescriptor object, which encapsulates logic needed to write to a COW. This way, filesystem verfication action can use the turend value directly for computing verity and hash the partition. Test: th Change-Id: Iafe9699ef0cc15961641fc94f8ad2820230a56e1
2021-03-24Add CowWriterFileDescriptorKelvin Zhang
Add a wrapper to use CowWriter as a file descriptor, this will be used by FileSystemVerification stage when writing FEC Test: treehugger Bug: 173432386 Change-Id: Iea42ca1081cd6836c7700db172d6987f770d52c0
2021-03-22Add checks before writing hashtree/verityKelvin Zhang
Verity that we read everything in hashtree_data_extent before writing hash tree. Bug: 173432386 Test: th Change-Id: I00ab8053de71b13991adaa243b6cb6c7efd6e60f
2021-03-22Unmap all partitions after postinstall finishesKelvin Zhang
If we don't unmap partitions, snapuserd will keep running in the background and do nothing. Unmap partitions so that memory associated with daemon can be freed. Test: th, apply an OTA, verify that _b partitions in /dev/block/mapper are gone Change-Id: I4a75f4abe3e69f9ad2a67a78472b61cfbd77b0eb Change-Id: Icacf1699a32bf045c2b832f7c1f021d90805d676
2021-03-17Refactor verity reads/writes to a separate fucntionKelvin Zhang
Test: th & serve an OTA with veity enabled, vabc disabled Change-Id: Ib1d5549ac615504a47c96a12b046975cfff01886
2021-03-16Pass in source slot to ctor of dynamic controlKelvin Zhang
When DynamicPartitionControlAndroid is constructed, it initializes both source and target slot to -1. These values get updated during PreparePartitionsForUpdate call. And we only PreparePartitionsForUpdate() when applying an OTA or applocating space for an OTA(not when verifying OTA metadata). Which means if VerifyPayloadApplicable() is called before any call two other APIs, we could be using an "Uninitialiazed" dynamic partition control. To mitigate this problem, we pass in source_slot at ctor of DynamicPartitionControl, also make IsDynamicPartition() api take in a slot number to avoid reading uninitialized member fields. Bug: 181643302 Test: apply an OTA, abort, restart update_engine, verify a payload Change-Id: I9a8a0fe8a9aca48e91241e15bdec33a1c1228553
2021-03-08Only skip CowCopy ops with same src/dst blockKelvin Zhang
When installing an VABC package, update_engine will omit COPY operations whose src and dst blocks are the same. As any untouched block during update is assumed to hold same content as the one in source slot. There's an additional complication: When applying a VABC update, update_engine might selectively convert some COPY operation to REPLACE operation to avoid merge conflicts. Current code skips these converted REPLACE operation if src/dst block are the same. But this optimization should be handled on the merge sequence generation side instead of apply side. Therefore, we are changing this to only skip COPY operations. This might slightly increase COW size, but once we implement proper optimization on merge sequence generation side COW size should be the same as before. Test: th Change-Id: I07f86c9fbcae48ba0dca6a4482fdbaea4a967fc3
2021-02-26Allow payload_metadata to take raw pointers with size am: 9b8a2bb050Kelvin Zhang
Original change: https://android-review.googlesource.com/c/platform/system/update_engine/+/1573381 MUST ONLY BE SUBMITTED BY AUTOMERGER Change-Id: Ie61cddccdef1323d1a4de0cd366ae41b2381a563
2021-02-26Allow payload_metadata to take raw pointers with sizeKelvin Zhang
This way, callers of payload_metadata don't have to store their data in std::vector. This makes parsing a `mmap()`ed payload file possible. While we don't need the ability to parse an mmap()ed payload on device, it is really handy for parsing payload from host-side tooling. In which case the entire payload will be available on disk anyway, and a streaming parse will complicate code a lot. Test: treehugger Change-Id: Icfbd8e849093c8f389ea802864680ab214bdd721
2021-02-26Fix an edge case where AllocateSpaceForPayload causes OTA to fail am: 51e08b90a7Kelvin Zhang
Original change: https://android-review.googlesource.com/c/platform/system/update_engine/+/1596613 MUST ONLY BE SUBMITTED BY AUTOMERGER Change-Id: I860ba7f321bdbb318af6f7032bb4d958ef5a7b89
2021-02-25Fix an edge case where AllocateSpaceForPayload causes OTA to failKelvin Zhang
When AllocateSpaceForPayload is called, it resets state in libsnapshot, but doesn't reset OTA checkpoints. This causes inconsistency. Bug: 180517664 Test: Update A allocate && Apply, update B allocate, Update A allocate && apply Change-Id: I6255d5e1fe77d9e41f88eb34b4b2879e37aee8ba
2021-02-21Merge ab/7061308 into stage.Xin Li
Bug: 180401296 Merged-In: Ib1d0f8080bc35a9af330dc768b78f3385b50a47c Change-Id: I9992904c3a85090ba0c352520c51d5f7650751cd
2021-02-17Use UpdateUsesSnapshotCompression to determine if VABC is usedKelvin Zhang
Test: th Change-Id: Ia04f92d46da34fcd28d7e97c24b6e02fd676e1c7
2021-02-16Don't list dynamic if a slot doesn't support DAPTianjie
If a slot doesn't support dynamic partitions, it's impossible to list dynamic partitions on that slot. And we should just fall back to the regular A/B in this case. Bug: 180025432 Test: apply a retrofit package Change-Id: I16c457b591e8c1d0cf1077a7be50dd9d8f61b8eb
2021-02-08Check for null of cow_writer_ before call Finalize()Kelvin Zhang
Test: fake-ota on small verbose, make sure update_engine doesn't crash Bug: 179231797 Change-Id: I7df45f93fc16444c51d8738b244cf65dc3bb8b7e
2021-01-28Calculate vbmetadigest of inactive slotTianjie
We want to improve the security of the keystore encryption key for resume on reboot. One AI is to create the key with the vbmeta digest of the next slot to boot into. After reboot, the decryption will fail if ro.boot.vbmeta.digest of the new slot doesn't match the calculated value before reboot. Since update_engine manages the slot switch, let it calculate the digest as well. Bug: 177625570 Test: do a update, check that the sysprop is set. Change-Id: I74b20fca72f6946d980b93d76990f1c8b8f246a9
2021-01-04Move signature persistence to CheckpointUpdateProgressKelvin Zhang
After update_engine extracts the signature, it immediately saves it to disk. However, if the signature is saved but subsequent update checkpoint isn't saved for some reason, resume will fail. Therefore we move the signature saving logic to CheckpointUpdateProgress() Test: treehugger Bug: 176087961 Change-Id: Iccb6854eaf07c7b81272376e685df43015ea46e2
2021-01-04Use string_view for pref interface to reduce copyKelvin Zhang
If you pass in a static string literal like "Hello World!", then with parameter type of const string& you need to construct a new string object, requiring a copy. It will also require a copy if your data is in another container, for example std::vector<char> . In update_engine, we store manifest bytes in std::vector, and sometimes we want to save that manifest to disk. This CL can help us reduce copy of the manifest(up to 2MB). Test: treehugger Change-Id: I70feb4c0673c174fd47f02c4bd41994f74cda743
2020-12-18Validate payload data for each operation am: 1205ea684eTianjie
Original change: https://googleplex-android-review.googlesource.com/c/platform/system/update_engine/+/12128977 MUST ONLY BE SUBMITTED BY AUTOMERGER Change-Id: Ib1d0f8080bc35a9af330dc768b78f3385b50a47c
2020-12-16Unconditionally save checkpoints when force = trueKelvin Zhang
When saving update progress, we skiip a checkpoint if operation_num isn't changed. But sometimes buffer offset changes without changing operation num, namely, when we extract signature. If a checkpoint is saved immediately before we extract signature, then we will skip the checkpoint after signature is extracted, causing the on disk pref to contain incorrect buffer_offset. Pref causing resume failure: kPrefsUpdateStateNextDataOffset Location on disk: /data/misc/update_engine/prefs/update-state-next-data-offset Test: treehugger, serve an OTA update w/o slot switch then perform slot switch Bug: 175785279 Change-Id: Iea2b0b27e1af40e1961a04019e7dd498c02feb13
2020-12-16Add unittest for filesystem verification actionKelvin Zhang
Test: treehuggre Change-Id: I03f69b7add96eaa481b1152a1425f4cb669d1113
2020-12-15Add unittest for load cache manifest success pathKelvin Zhang
Add a unittest that generates a valid manifest, signed with unittest keys. Then assert that download action can load the cached manifest correctly. Since the unittest keys are RSA keys, we add support for RSA keys to payload_verifier Test: treehugger Change-Id: Iebf4bd740ad2c18f4e71527eeee4c12e3f8e7dea
2020-12-10Add unittest for set next op indexKelvin Zhang
In order to add this unittest, we need to ensure that a checkpoint is saved after each InstallOp. However, current logic throttles checkpoint by time, it also skips checkpoint if buffer offset is the same. To address these issues, 1. Add a ShouldCheckpoint() to control whether to throttle checkpoint based on time, unittest overrides this and just return true 2. Modify logic to only skip a checkpoint if operation_index hasn't changed. For SOURCE_COPY operations, buffer offset won't be advanced but operation index will. Test: treehugger Change-Id: Ib81bfe0c4ecb7200096b4b22390fb1f2fcff1581
2020-12-10For ext4 filesystem log if created/modified timestamp differsHåkan Kvist
If a filesystem has been written to without being remounted (e.g fsck), log filesystem timestamps. This information could help understanding why there is a checksum failure when applying an update and device has not been remounted. Bug: 174577460 Test: Inspect logs when installing ota on a device where fsck has force checked an ext4 a/b partition. Change-Id: Ic2cc52179179e40b6563f170724a971a2a3c5da9
2020-12-10Skip SOURCE_COPY with same src/dst extentKelvin Zhang
Test: treehugger Change-Id: I9e8301704b21850c8f836d83655b4c85ae3d843e
2020-12-09Don't clear fs_mount_dir_Kelvin Zhang
During postinstall, fs_mount_dir_ is cleared after postinstall is completed for a partition. If there are more than 1 partition which runs postinstall, the next partition will see an empty fs_mount_dir_ and fail. Test: treehugger Bug: 175232022 Change-Id: I42110d0e2a886b631f0984edcdc7a916b1121e3e
2020-12-07Mock a /postinstall dir at test timeKelvin Zhang
Bug: 172696594 Test: treehugger Change-Id: Ib6264569d090dc61fc9ded5f833e3841ec16a8dd
2020-12-05Merge remote-tracking branch 'aosp/upstream-master' into mergeKelvin Zhang
Test: treehugger Change-Id: I4984f03fa95a753fb17779451eb458f177432d4f
2020-12-03update_engine: Make InstallPlan's dump nicerAmin Hassani
Its really hard to read anything out of the current InstallPlan's logs. This CL makes it a bit more structured so it can be read easier. Also added a few other properties of InstallPlan that were missing in the Dump(). Added unittest for it too. BUG=b:171829801 TEST=cros_workon_make --board reef --test update_engine Change-Id: Iaa327e875877e9645ef8f0af875c280e11ee485d Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/2558933 Tested-by: Amin Hassani <ahassani@chromium.org> Auto-Submit: Amin Hassani <ahassani@chromium.org> Reviewed-by: Jae Hoon Kim <kimjae@chromium.org> Commit-Queue: Jae Hoon Kim <kimjae@chromium.org>
2020-12-01Add unittest for resume updateKelvin Zhang
Test: treehugger Bug: 173392916 Change-Id: I4b320685af93b20fc1f4abcdd6a416f1c97731ab
2020-12-01Check if mount directory exists before attempting mount() syscallKelvin Zhang
Test: treehugger Bug: 172696594 Change-Id: If5d0971a5443db2f26cddb31ac2034112e346fe2
2020-11-30update_engine: use new base::Delete{File,PathRecursively}hscham
base::DeleteFile(const FilePath& path, bool recursive) would be deprecated in next libchrome uprev. BUG=chromium:1144735 TEST=cros_run_unit_tests --board=eve --packages update_engine Change-Id: Iaeac97f533a156c2c29f7ba53755664d6591b0a1 Reviewed-on: https://chromium-review.googlesource.com/c/aosp/platform/system/update_engine/+/2543515 Tested-by: Grace Cham <hscham@chromium.org> Reviewed-by: Amin Hassani <ahassani@chromium.org> Reviewed-by: Hidehiko Abe <hidehiko@chromium.org> Commit-Queue: Grace Cham <hscham@chromium.org>
2020-11-24Check for return value of ConsumeWithBufferKelvin Zhang
Test: treehugger Bug: 168554689 Change-Id: I1447c705143fe3d88267dcd08bbc9674d9cc6676
2020-11-23Do not map dynamic partitions on VABC devicesKelvin Zhang
With VABC, we no longer need to map all partitions before reading/writing, so don't try to map them. 1. modify GetPartitionDevice to return empty path for target partitions on VABC 2. Add a separate GetMountableTargetDevice for obtaining a mountable device path, specifically for postinstall Test: treehugger Change-Id: Ib1f608914fc49c677ce7389140ca79b028171191
2020-11-20Handle resume of VABC updates by emitting labelsKelvin Zhang
To support resuming an update with Virtual AB Compression, we emit labels in between operations. After writing all SOURCE_COPY, we emit label 0. Each time we finished writing an InstallOp, we emit a label incremented by 1. When resuming, we pass the label to CowWriter. Test: treehugger 1. update_device.py ota.zip --extra-headers="SWITCH_SLOT_ON_REBOOT=0" 2. update_device.py ota.zip 3. Verify that 2 did not re-start the entire update, only fs verification and postinstall may re-run. Change-Id: I785cd04a35457181621ed7b8c0be9a46b6004b7b
2020-11-17Don't use unix open() syscall on VABC partitionsKelvin Zhang
Move logic dealing with target partitions to a function, such that it can be overriden by subclasses. Bug: 168554689 Test: treehugger Change-Id: I59053a70915e51b0ab1b30922d14f211e1ba0605
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-03 17:50:20 +0000