summaryrefslogtreecommitdiff
path: root/extract_utils.sh
AgeCommit message (Collapse)Author
2019-09-10extract_utils: Support extracting from product partitionrazorloves
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f (cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-07-18extract_utils: Support multidexRashed Abdel-Tawab
Co-authored-by: Erfan Abdi <erfangplus@gmail.com> Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-06-26extract_utils: Add functions to extract vendor blobs from vendor.imgRashed Abdel-Tawab
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-03-03extract_utils: introduce support for executing blob fixupsVladimir Oltean
* Traditionally, the task of hex-editing blobs has been approached in 2 ways: (1) Do it out-of-band, commit the modified blob, and record its edited sha1sum in proprietary-files.txt (aka pin it). (2) Do it in-band, by adding code to the device-level extract-files.sh (usually this performs patchelf or sed). This code runs after the extract_utils functions were invoked. * Problems of approach (1): - It relies on verbal (basically commit message) documentation of the hex-editing that was done. Makes it more difficult to reproduce. - Each time blobs are updated, pinning needs to be temporarily removed, hex-editing done again manually and new hash put back. * Problems of approach (2): - It is incompatible with the concept of pinning, which is useful for kanging blobs from another device. A pinned blob would either: - Match the hash, get hex-edited, then it won't match the hash next time around. - Not match the hash (because of, say, hex-editing), then the extraction script would use an unwanted blob version instead of the pinned one (either that, or say "!! file not found in source"). * In summary, this patch adds system-wide support for approach (2) in order to address the aforementioned shortcomings. * At device level, users of extract_utils who wish to perform blob fixups can override a blob_fixup() Bash function in their extract-files.sh immediately after running "source ${HELPER}". The blob_fixup() function will be called by the common extract() function after extracting every individual blob, giving the user the opportunity to hook custom code after this operation takes place. * In proprietary-files.txt, the line corresponding to this blob which needs fixups can look in one of 2 ways: (a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so Do this if you are taking the blob from the stock ROM. The fixup script will always run after the blob is extracted. (b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258 Do this if you are kanging the blob from somebody else. The pinning logic now applies for both the pre- and the post-fixup hashes. The fixup script will only run if the blob doesn't match the hex-edited blob, although the fixup script should really be idempotent. Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03extract_utils: make --section match words anywhere in the lineVladimir Oltean
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03extract_utils: introduce -k (kang mode) optionVladimir Oltean
* The use case is easier updating of pinned blobs. When --kang is set, pinning is automatically ignored, and the script will print lines at its output that can be directly copied back into the proprietary-files.txt. * Best served together with --section ${SECTION}, and proper grouping of the proprietary-files.txt. Change-Id: I648fbcbd4580a4a002b00828bcfee18d1e265d7b Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03extract_utils: cleanup variable names and arguments passed to extract() functionVladimir Oltean
* This also makes the --section argument non-positional, since otherwise it is not possible to easily support more than one optional positional argument. This is in preparation of one more optional argument to come in a follow-up patch: --kang. Change-Id: Ieb142e0854319defb9a278ab68cd4aeefd0fbdd5 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03extract_utils: be compatible with system-as-root layoutsVladimir Oltean
* The use case is that if you have the following layout: $TOP --- system.img | +-- vendor.img you should be able (from $TOP) to: mkdir system; mount -o ro,loop system.img system mkdir vendor; mount -o ro,loop vendor.img vendor and then (from device tree) ./extract-files.sh $TOP But this doesn't work if system.img is SAR and contains another "system" dir inside. This patch makes sure it searches for a "system" dir in the provided path as well, if it couldn't find the blob anywhere else. Change-Id: Ib49cd5b587b3a57478a66ff69cf840270c2b1403 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03extract_utils: drop the "/system/" prefix from prints of blob namesVladimir Oltean
* This makes the printed output closer to the proprietary-files.txt syntax Change-Id: I81b844bb6bb1d1a2f91a39151a892fbfc0bed20b Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-01-17extract_utils: fix mismatch with non-lowercase pinned sha1sumVladimir Oltean
Change-Id: I86421ca69007ad1992aa67deedf7d2e32bbe4dba Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-12-24extract_utils: avoid doing stuff on files that don't existVladimir Oltean
* If an apk/jar doesn't exist, the script would still try to deodex it. * If an xml doesn't exist, the script would still try to "fix" it. * Take it easier, man, it's not your fault. Change-Id: I3061fb48b403da5121e3c17dd9ecdb6cd148bf97 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-11-13extract_utils: Redirect vdexextractor log to /dev/nullLuca Stefani
Change-Id: I55028997c0396d57c024e6c42a53907f6d1510c8
2018-10-21Add CDEX support for boot JAR filesLuca Stefani
Change-Id: I0039256c883a58c5a5125fd9bff6e6ec3b98688d
2018-10-07extract_utils: Fix output dex file name generated by vdexExtractorTheStrix
- Reflect the changes required by: https://github.com/anestisb/vdexExtractor/commit/ff58255706a18c11567df97da484157f832a1df7 - Also fix JARVDEX file path for deodexing jars Change-Id: Ief355b0bd9e69a67b552d0bd2060f43adb39fb5f
2018-10-05extract_utils: add CDEX support via compact_dex_convertercodeworkx
Source: https://github.com/anestisb/vdexExtractor#compact-dex-converter Change-Id: I63664cae02e51a05e5a62c0c5c0595eead6c6513
2018-07-12extract_utils: Newline love for adb extractionLuca Stefani
Change-Id: Ibbf7a03ed6920d556ed7658c277f37b02d8876b8
2018-07-08extract_utils: make get_file() able to search paths with and w/o /system prefixVladimir Oltean
Change-Id: I75eba627646caa8ceb8a30575124c57e5ee9445c Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: extract(): fix src not being searched in "src:dst;args" specVladimir Oltean
* Root cause of the issue is improper naming of variables. Turns out, there was no variable to even denote where the blob should have been searched for, at "src". * Previously there was one such variable, suggestively called "TARGET", that was desperately trying to serve as both, depending on who +2d hacks harder. * One such example is "e02a942 extract_utils: Fix makefile generation issues". That patch deliberately trimmed the "src:" from a spec (therefore obviously breaking the search at src) but enabling the searching at dst, via the good-for-all TARGET variable. * This patch introduces the following variables: - SRC_FILE: absolute path corresponding to SPEC_SRC_FILE in the Android filesystem. - DST_FILE: absolute path corresponding to SPEC_DST_FILE in the Android filesystem. Somewhat analogous to the old TARGET variable, but actually contains the leading / as well (/system/bin/adsprpcd instead of system/bin/adsprpcd). * Use existing common get_file() function (which previously was impossible to use, because it was impossible to determine calling arguments properly) to reduce complexity of handling adb and disk image as blob sources. * Via the new SRC_FILE and DST_FILE variables, search for a blob in all possible locations (including paths stripped of "/system" which transforms an absolute path in the Android filesystem into a proper relative path to that file in a disk image). Change-Id: Ic40fb4dc93541d8b3f33fde586b773199cf4ded2 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: extract(): rename DEST variable to VENDOR_REPO_FILEVladimir Oltean
* This denotes the path of the file that results from the extraction process, relative to the "proprietary" directory. * This is a cleanup patch. Change-Id: I38e759bb6ed697f0a31ca35a7aa9b9b92f8b6793 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: extract(): rename ARGS variable to SPEC_ARGSVladimir Oltean
* This is a cleanup patch. Change-Id: I2a4aac840c06c4f9b94b77b32ead33352ce395ff Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: extract(): rename FILE variable to SPEC_SRC_FILEVladimir Oltean
* This is a cleanup patch. Change-Id: I971b2b9eba5c1f303c0d5f902feaaad25b3b26ed Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: extract(): rename FROM variable to SPEC_DST_FILEVladimir Oltean
* This is a cleanup patch. Change-Id: I308b0478484ccf9a594078825731606fce5f5507 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: create src_file() function to mirror behavior of target_file()Vladimir Oltean
* This is a cleanup-only patch. Change-Id: I144252de765686ac3469652994a04820ecc96d4d Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: prefix_match(): do not strip target_args from its outputVladimir Oltean
* The write_product_copy_files() and write_product_packages() functions rely on its undocumented behavior of keeping target_args in the returned list, because they are users of target_args (such as ";PRESIGNED" etc). * Make the behavior documented. Change-Id: If71595dca32abd40039706d4fed2d7f12e005365 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08extract_utils: target_file() and target_args() cleanupVladimir Oltean
* Strip target_args from target_file at callee instead of at caller * WARNING! Changes (improperly documented) behavior of prefix_match() function, which is expected to not strip target_args(), and the root cause why stripping target_args was currently done at caller. Will be addressed in next patch. Change-Id: I820d2350aa64ff41374809fcb22f812257132652 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08Revert "extract_files: Add support for paths without system/"Vladimir Oltean
This reverts commit 1f56fc5064f2556e479fe75ff8be0e0ccfea978c. This will be addressed cleaner in the cleanup commit to follow. Change-Id: Iec013bfafdbb334ab2657b10c433158002809b62
2018-07-08Revert "extract_utils: Fix makefile generation issues"Vladimir Oltean
This reverts commit e02a9423adf04ff1316b53d5fb456dfe7ec88680. This breaks extracting from src in "src:dst", such as from a disk image. Will be addressed in the cleanup commit that follows. Change-Id: Iff84a926f0c3bf908320b43ba40235e0a89db644
2018-04-09write_makefiles: fix paths in BUILD_PREBUILT with destinationVladimir Oltean
* proprietary-files.txt entries such as "-app/TimeService/TimeService.apk:priv-app/TimeService/TimeService.apk" should generate a "LOCAL_SRC_FILES := proprietary/priv-app/TimeService/TimeService.apk" in the Makefile definition. * However, currently, the prefix_match function is being called on the whole PRODUCT_PACKAGES_LIST entry (whole line, including ":"), and therefore, TimeService.apk would be included in the APPS list instead of PRIV_APPS. * Furthermore, because of the incorrect prefix_match, the generated LOCAL_SRC_FILES is "proprietary/app/priv-app/TimeService/TimeService.apk", which breaks the build because there is no file at that wrong path. * The fix is to make the extract function match up with write_product_packages by applying the target_file function on all BUILD_PREBUILT source files. Change-Id: Ib4859b8854db0f2142bb3f28cce8dd25f7141732 Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-03-30extract_utils: Use vdexExtractor for extracting oreo classes.dexJoe Maples
vdexExtractor is a tool made by anestisb that is written in C++ with code taken directly from art. However, anestisb has also added a quicken decompiler to oatdump that was merged upstream, so we only need vdexExtractor for 8.1 and 8.0. Change-Id: Ic2cf2dc627a1ad2fa4d500d02d9eac8b8a9577b5 Signed-off-by: Joe Maples <joe@frap129.org>
2018-02-27extract_utils: Fix vendor framework makefile generationMichael Bestas
* Fix copy-pasta Change-Id: I9b2f61702985cb59af34ad146fd61bedba043f81
2018-02-26extract-utils: Use LOCAL_VENDOR_FILE instead of LOCAL_PROPRIETARY_FILEEthan Chen
* Oreo expects VNDK compatible files to be listed as LOCAL_VENDOR_FILE, not LOCAL_PROPRIETARY_FILE. Change-Id: Ia2384c4f3ab3a99b79df52c796c53dc25a0f4a88
2018-01-29extract_utils: Fix regressions from 0d9fa43a2a9b4f134942f95e1c4eb1a05608e131Paul Keith
Change-Id: I78fc47c63e6bfbd5ee98a56a1d80f763e96ac266
2018-01-27extract_utils: Split optional args with semicolon instead of colonMichael Bestas
* Fix optional flags like PRESIGNED after I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed Change-Id: I889f995d7f19b09908cfbaf7474442ce48bfabdb
2018-01-27extract_utils: Fix makefile generation issuesMichael Bestas
* Fix makefile generation for packages that have set a different target destination * Thanks to rashed and javelinanddart for their help in debugging and solving this issue. Change-Id: I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed
2017-12-03extract_utils: Use temporary folder for system dumpconbold
* This way it's automatically deleted at the end of the extraction Change-Id: I77b3357875b8070d370f04c5a245f9aa3ca1939c
2017-11-13extract_utils: Fix pinning when not cleaning vendor dirGabriele M
Skip the extraction of pinned files if the ones currently available have the expected sha1. If we don't, we will overwrite pinned files with potentially incorrect files when the current vendor files are not moved to a temporary directory (i.e. when not cleaning vendor). Change-Id: I640d6bf2ed98eb366a4df17f0ebeaec81cb5274b
2017-10-14extract_utils: Allow to interrupt the extractionGabriele M
We want to cleanup our temporary files independently on the signal, so just execute a trap on 0. This will ensure temporary files are always removed and doesn't require any extra care when trapping signals such as SIGINT that require an explicit exit call. Change-Id: Ieff4f15c44a9ac9d5a543d14c140ebd72c0e7344
2017-10-08extract_files: Mark vendor etc files as proprietaryRashed Abdel-Tawab
Change-Id: Iad8d87a18e5b1dc68c7f2265c0e35a9313804408
2017-10-08extract_utils: cover vendor/frameworksChristian Oder
Change-Id: If60b9b6c47cfd9c32477f92ad87011eddb41753b
2017-10-07extract_files: Add support for treble compatible makefilesRashed Abdel-Tawab
To be honest, this name is a little misleading, this is how it should have been done in the first place. This allows devices to copy vendor files to the proper location depending on TARGET_COPY_OUT_VENDOR rather than hardcoding system/vendor. This allows devices with dedicated vendor partitions to copy directly to vendor. The only reason it's optional is that some nexi set TARGET_COPY_OUT_VENDOR to system which would cause some weird breakage. Change-Id: Ic46bc1086737835340abef9f61693d386bc6a5dc
2017-10-07extract_files: Adjust deodex path and tmpdirRashed Abdel-Tawab
* Make a tempdir using the mktemp command rather than just making a dir in /tmp to accomodate for systems that don't set proper perms or dont have /tmp * Fix the deodex procedure to pull files from the right path Change-Id: I181863599b6670e3a149069dbb7b13ebf73bae8e
2017-09-20extract_utils: implement LOCAL_MODULE_RELATIVE_PATHM1cha
Change-Id: Ia97633652ff9647a6e82013b208097be5c5575a9
2017-09-20extract_utils: Add VDEX deodex capabilities for oreoRashed Abdel-Tawab
Oreo introduces new optimized dex files (or vdex). While smali/baksmali have supported vdex since 2.2.1, you actually have to have the .vdex in the same location as the .oat or else the smali will fail due to not finding the .vdex. Copy it (if it exists) and echo a warning since vdex deodexing is still experimental. Change-Id: Ic612751be45a8b6e54b8794b426a4d5adbbf13d2
2017-08-23lineage: Complete the rebrandLuca Stefani
* TODO: Rebrand cmsdk when it's in a working state Change-Id: I0d8d573f9e0032110ae3a0f54ebe11a877b83137
2017-08-23vendor: change vendor to lineageDan Pasanen
Change-Id: I2df13091f5d62b552ef434aef1a389f4fe1249e0
2017-08-05Fix extract_utils when not using blob sectionsHarry Youd
"$3" evaluates as "" when $3 is empty, and so $# is still 3 When $3 is empty, do not use it Change-Id: I87df7dd8ed23cac3d4cff16dce0996bbe0a3e6cc
2017-07-30extract_files: Add support for paths without system/Rashed Abdel-Tawab
Sometime blob directories are contained in dirs not named system (ex. extracting blobs from another vendor dir) so check if the file exists without system/ before appending "system/" to filenames. Change-Id: I4ba946e178260b4f4804b1ac4422645e3085c665
2017-07-30extract_files: Add support for specifying blob sectionsRashed Abdel-Tawab
Sometimes the need arises to update one specific subset of blobs while holding others steady. Introduce a way to specify the "section" of blobs to update (obviously requires the proprietary-files to be properly split up first) Requires additional changes in device tree extract_files.sh to support passing the additional arguments Change-Id: I98feab56f8bfade2818b1b5264e019440d4c57c5
2017-07-29extract_utils: Fix cleanup variables assigmentGabriele M
The current assignment is wrong, we cleanup the files when we tell not to do so. Change-Id: I73a9a17f5ebae6f2dbdc3d6d9aaa28cd75fb4f87
2017-05-18extract_utils: fix extracting XML files containing binary charactersDobroslaw Kijowski
When a line from the XML file contains a character which grep classifies as a binary one, it will omit this line at all. Moreover it will append 'Binary file matches' to the stdout. This behavior breaks some XML files. To fix this issue, let's tell grep to process a binary file as if it were text. Change-Id: I6ec5689f73a0926870e502f3f8ee77346b47d05f
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-03 14:54:43 +0000