diff options
| l---------[-rw-r--r--] | android-changes-for-ndk-developers.md | 420 | ||||
| -rw-r--r-- | docs/android-changes-for-ndk-developers.md | 419 |
2 files changed, 420 insertions, 419 deletions
diff --git a/android-changes-for-ndk-developers.md b/android-changes-for-ndk-developers.md index 1630db8e4..328d3eb96 100644..120000 --- a/android-changes-for-ndk-developers.md +++ b/android-changes-for-ndk-developers.md @@ -1,419 +1 @@ -# Android changes for NDK developers - -This document details important changes related to native code -loading in various Android releases. - -Required tools: the NDK has an _arch_-linux-android-readelf binary -(e.g. arm-linux-androideabi-readelf or i686-linux-android-readelf) -for each architecture (under toolchains/), but you can use readelf for -any architecture, as we will be doing basic inspection only. On Linux -you need to have the “binutils” package installed for readelf, -and “pax-utils” for scanelf. - - -## How we manage incompatible changes - -Our general practice with dynamic linker behavior changes is that they -will be tied to an app's target API level: - -* Below the affected API level we'll preserve the old behavior or issue -a warning, as appropriate. - -* At the affected API level and above, we’ll refuse to load the library. - -* Warnings about any behavior change that will affect a library if you -increase your target API level will appear in logcat when that library -is loaded, even if you're not yet targeting that API level. - -* On a developer preview build, dynamic linker warnings will also show up -as toasts. Experience has shown that many developers don’t habitually -check logcat for warnings until their app stops functioning, so the -toasts help bring some visibility to the issues before it's too late. - -## Changes to library dependency resolution - -Until it was [fixed](https://issuetracker.google.com/36950617) in -JB-MR2, Android didn't include the application library directory -on the dynamic linker's search path. This meant that apps -had to call `dlopen` or `System.loadLibrary` on all transitive -dependencies before loading their main library. Worse, until it was -[fixed](https://issuetracker.google.com/36935779) in JB-MR2, the -dynamic linker's caching code cached failures too, so it was necessary -to topologically sort your libraries and load them in reverse order. - -If you need to support Android devices running OS -versions older than JB-MR2, you might want to consider -[ReLinker](https://github.com/KeepSafe/ReLinker) which claims to solve -these problems automatically. - -## Changes to library search order - -We have made various fixes to library search order when resolving symbols. - -With API 22, load order switched from depth-first to breadth-first to -fix dlsym(3). - -Before API 23, the default search order was to try the main executable, -LD_PRELOAD libraries, the library itself, and its DT_NEEDED libraries -in that order. For API 23 and later, for any given library, the dynamic -linker divides other libraries into the global group and the local -group. The global group is shared by all libraries and contains the main -executable, LD_PRELOAD libraries, and any library with the DF_1_GLOBAL -flag set (by passing “-z global” to ld(1)). The local group is -the breadth-first transitive closure of the library and its DT_NEEDED -libraries. The M dynamic linker searches the global group followed by -the local group. This allows ASAN, for example, to ensure that it can -intercept any symbol. - - -## RTLD_LOCAL (Available in API level >= 23) - -The dlopen(3) RTLD_LOCAL flag used to be ignored but is implemented -correctly in API 23 and later. Note that RTLD_LOCAL is the default, -so even calls to dlopen(3) that didn’t explicitly use RTLD_LOCAL will -be affected (unless they explicitly used RTLD_GLOBAL). With RTLD_LOCAL, -symbols will not be made available to libraries loaded by later calls -to dlopen(3) (as opposed to being referenced by DT_NEEDED entries). - - -## GNU hashes (Availible in API level >= 23) - -The GNU hash style available with --hash-style=gnu allows faster -symbol lookup and is now supported by the dynamic linker in API 23 and -above. (Use --hash-style=both if you want to build code that uses this -feature >= Android M but still works on older releases.) - - -## Correct soname/path handling (Available in API level >= 23) - -The dynamic linker now understands the difference -between a library’s soname and its path (public bug -https://code.google.com/p/android/issues/detail?id=6670). API level 23 -is the first release where search by soname is implemented. Earlier -releases would assume that the basename of the library was the soname, -and used that to search for already-loaded libraries. For example, -`dlopen("/this/directory/does/not/exist/libc.so", RTLD_NOW)` would -find `/system/lib/libc.so` because it’s already loaded. This also meant -that it was impossible to have two libraries `"dir1/libx.so"` and -`"dir2/libx.so"` --- the dynamic linker couldn’t tell the difference -and would always use whichever was loaded first, even if you explicitly -tried to load both. This also applied to DT_NEEDED entries. - -Some apps have bad DT_NEEDED entries (usually absolute paths on the build -machine’s file system) that used to work because we ignored everything -but the basename. These apps will fail to load on API level 23 and above. - - -## Symbol versioning (Available in API level >= 23) - -Symbol versioning allows libraries to provide better backwards -compatibility. For example, if a library author knowingly changes -the behavior of a function, they can provide two versions in the same -library so that old code gets the old version and new code gets the new -version. This is supported in API level 23 and above. - - -## Opening shared libraries directly from an APK - -In API level 23 and above, it’s possible to open a .so file directly from -your APK. Just use `System.loadLibrary("foo")` exactly as normal but set -`android:extractNativeLibs="false"` in your `AndroidManifest.xml`. In -older releases, the .so files were extracted from the APK file -at install time. This meant that they took up space in your APK and -again in your installation directory (and this was counted against you -and reported to the user as space taken up by your app). Any .so file -that you want to load directly from your APK must be page aligned -(on a 4096-byte boundary) in the zip file and stored uncompressed. -Current versions of the zipalign tool take care of alignment. - -Note that in API level 23 and above dlopen(3) will open a library from -any zip file, not just your APK. Just give dlopen(3) a path of the form -"my_zip_file.zip!/libs/libstuff.so". As with APKs, the library must be -page-aligned and stored uncompressed for this to work. - - -## Private API (Enforced for API level >= 24) - -Native libraries must use only public API, and must not link against -non-NDK platform libraries. Starting with API 24 this rule is enforced and -applications are no longer able to load non-NDK platform libraries. The -rule is enforced by the dynamic linker, so non-public libraries -are not accessible regardless of the way code tries to load them: -System.loadLibrary, DT_NEEDED entries, and direct calls to dlopen(3) -will all work exactly the same. - -Users should have a consistent app experience across updates, -and developers shouldn't have to make emergency app updates to -handle platform changes. For that reason, we recommend against using -private C/C++ symbols. Private symbols aren't tested as part of the -Compatibility Test Suite (CTS) that all Android devices must pass. They -may not exist, or they may behave differently. This makes apps that use -them more likely to fail on specific devices, or on future releases --- -as many developers found when Android 6.0 Marshmallow switched from -OpenSSL to BoringSSL. - -In order to reduce the user impact of this transition, we've identified -a set of libraries that see significant use from Google Play's -most-installed apps, and that are feasible for us to support in the -short term (including libandroid_runtime.so, libcutils.so, libcrypto.so, -and libssl.so). In order to give you more time to transition, we will -temporarily support these libraries; so if you see a warning that means -your code will not work in a future release -- please fix it now! - -In O and later, the system property `debug.ld.greylist_disabled` can be -used to deny access to the greylist even to an app that would normally -be allowed it. This allows you to test compatibility without bumping the -app's `targetSdkVersion`. Use `setprop debug.ld.greylist_disabled true` -to turn this on (any other value leaves the greylist enabled). - -``` -$ readelf --dynamic libBroken.so | grep NEEDED - 0x00000001 (NEEDED) Shared library: [libnativehelper.so] - 0x00000001 (NEEDED) Shared library: [libutils.so] - 0x00000001 (NEEDED) Shared library: [libstagefright_foundation.so] - 0x00000001 (NEEDED) Shared library: [libmedia_jni.so] - 0x00000001 (NEEDED) Shared library: [liblog.so] - 0x00000001 (NEEDED) Shared library: [libdl.so] - 0x00000001 (NEEDED) Shared library: [libz.so] - 0x00000001 (NEEDED) Shared library: [libstdc++.so] - 0x00000001 (NEEDED) Shared library: [libm.so] - 0x00000001 (NEEDED) Shared library: [libc.so] -``` - -*Potential problems*: starting from API 24 the dynamic linker will not -load private libraries, preventing the application from loading. - -*Resolution*: rewrite your native code to rely only on public API. As a -short term workaround, platform libraries without complex dependencies -(libcutils.so) can be copied to the project. As a long term solution -the relevant code must be copied to the project tree. SSL/Media/JNI -internal/binder APIs should not be accessed from the native code. When -necessary, native code should call appropriate public Java API methods. - -A complete list of public libraries is available within the NDK, under -platforms/android-API/usr/lib. - -Note: SSL/crypto is a special case, applications must NOT use platform -libcrypto and libssl libraries directly, even on older platforms. All -applications should use GMS Security Provider to ensure they are protected -from known vulnerabilities. - - -## Missing Section Headers (Enforced for API level >= 24) - -Each ELF file has additional information contained in the section -headers. These headers must be present now, because the dynamic linker -uses them for sanity checking. Some developers strip them in an -attempt to obfuscate the binary and prevent reverse engineering. (This -doesn't really help because it is possible to reconstruct the stripped -information using widely-available tools.) - -``` -$ readelf --header libBroken.so | grep 'section headers' - Start of section headers: 0 (bytes into file) - Size of section headers: 0 (bytes) - Number of section headers: 0 -``` - -*Resolution*: remove the extra steps from your build that strip section -headers. - -## Text Relocations (Enforced for API level >= 23) - -Starting with API 23, shared objects must not contain text -relocations. That is, the code must be loaded as is and must not be -modified. Such an approach reduces load time and improves security. - -The usual reason for text relocations is non-position independent -hand-written assembler. This is not common. Use the scanelf tool as -described in our documentation for further diagnostics: - -``` -$ scanelf -qT libTextRel.so - libTextRel.so: (memory/data?) [0x15E0E2] in (optimized out: previous simd_broken_op1) [0x15E0E0] - libTextRel.so: (memory/data?) [0x15E3B2] in (optimized out: previous simd_broken_op2) [0x15E3B0] - ... -``` - -If you have no scanelf tool available, it is possible to do a basic -check with readelf instead, look for either a TEXTREL entry or the -TEXTREL flag. Either alone is sufficient. (The value corresponding to the -TEXTREL entry is irrelevant and typically 0 --- simply the presence of -the TEXTREL entry declares that the .so contains text relocations). This -example has both indicators present: - -``` -$ readelf --dynamic libTextRel.so | grep TEXTREL - 0x00000016 (TEXTREL) 0x0 - 0x0000001e (FLAGS) SYMBOLIC TEXTREL BIND_NOW -``` - -Note: it is technically possible to have a shared object with the TEXTREL -entry/flag but without any actual text relocations. This doesn't happen -with the NDK, but if you're generating ELF files yourself make sure -you're not generating ELF files that claim to have text relocations, -because the Android dynamic linker trusts the entry/flag. - -*Potential problems*: Relocations enforce code pages being writable, and -wastefully increase the number of dirty pages in memory. The dynamic -linker has issued warnings about text relocations since Android K -(API 19), but on API 23 and above it refuses to load code with text -relocations. - -*Resolution*: rewrite assembler to be position independent to ensure -no text relocations are necessary. The -[Gentoo Textrels guide](https://wiki.gentoo.org/wiki/Hardened/Textrels_Guide) -has instructions for fixing text relocations, and more detailed -[scanelf documentation](https://wiki.gentoo.org/wiki/Hardened/PaX_Utilities). - - -## Invalid DT_NEEDED Entries (Enforced for API level >= 23) - -While library dependencies (DT_NEEDED entries in the ELF headers) can be -absolute paths, that doesn't make sense on Android because you have -no control over where your library will be installed by the system. A -DT_NEEDED entry should be the same as the needed library's SONAME, -leaving the business of finding the library at runtime to the dynamic -linker. - -Before API 23, Android's dynamic linker ignored the full path, and -used only the basename (the part after the last ‘/') when looking -up the required libraries. Since API 23 the runtime linker will honor -the DT_NEEDED exactly and so it won't be able to load the library if -it is not present in that exact location on the device. - -Even worse, some build systems have bugs that cause them to insert -DT_NEEDED entries that point to a file on the build host, something that -cannot be found on the device. - -``` -$ readelf --dynamic libSample.so | grep NEEDED - 0x00000001 (NEEDED) Shared library: [libm.so] - 0x00000001 (NEEDED) Shared library: [libc.so] - 0x00000001 (NEEDED) Shared library: [libdl.so] - 0x00000001 (NEEDED) Shared library: -[C:\Users\build\Android\ci\jni\libBroken.so] -``` - -*Potential problems*: before API 23 the DT_NEEDED entry's basename was -used, but starting from API 23 the Android runtime will try to load the -library using the path specified, and that path won't exist on the -device. There are broken third-party toolchains/build systems that use -a path on a build host instead of the SONAME. - -*Resolution*: make sure all required libraries are referenced by SONAME -only. It is better to let the runtime linker to find and load those -libraries as the location may change from device to device. - - -## Missing SONAME (Enforced for API level >= 23) - -Each ELF shared object (“native library”) must have a SONAME (Shared -Object Name) attribute. The NDK toolchain adds this attribute by default, -so its absence indicates either a misconfigured alternative toolchain -or a misconfiguration in your build system. A missing SONAME may lead -to runtime issues such as the wrong library being loaded: the filename -is used instead when this attribute is missing. - -``` -$ readelf --dynamic libWithSoName.so | grep SONAME - 0x0000000e (SONAME) Library soname: [libWithSoName.so] -``` - -*Potential problems*: namespace conflicts may lead to the wrong library -being loaded at runtime, which leads to crashes when required symbols -are not found, or you try to use an ABI-incompatible library that isn't -the library you were expecting. - -*Resolution*: the current NDK generates the correct SONAME by -default. Ensure you're using the current NDK and that you haven't -configured your build system to generate incorrect SONAME entries (using -the -soname linker option). - - -## DT_RUNPATH support (Available in API level >= 24) - -If an ELF file contains a DT_RUNPATH entry, the directories listed there -will be searched to resolve DT_NEEDED entries. The string `${ORIGIN}` will -be rewritten at runtime to the directory containing the ELF file. This -allows the use of relative paths. The `${LIB}` and `${PLATFORM}` -substitutions supported on some systems are not currently implemented on -Android. - - -## Writable and Executable Segments (Enforced for API level >= 26) - -Each segment in an ELF file has associated flags that tell the -dynamic linker what permissions to give the corresponding page in -memory. For security, data shouldn't be executable and code shouldn't be -writable. This means that the W (for Writable) and E (for Executable) -flags should be mutually exclusive. This wasn't historically enforced, -but is now. - -``` -$ readelf --program-headers -W libBadFlags.so | grep WE - LOAD 0x000000 0x00000000 0x00000000 0x4c01d 0x4c01d RWE 0x1000 -``` - -*Resolution*: we're aware of one middleware product that introduces these -into your app. The middleware vendor is aware of the problem and has a fix -available. - -## Invalid ELF header/section headers (Enforced for API level >= 26) - -In API level 26 and above the dynamic linker checks more values in -the ELF header and section headers and fails if they are invalid. - -*Example error* -``` -dlopen failed: "/data/data/com.example.bad/lib.so" has unsupported e_shentsize: 0x0 (expected 0x28) -``` - -*Resolution*: don't use tools that produce invalid/malformed -ELF files. Note that using them puts application under high risk of -being incompatible with future versions of Android. - -## Enable logging of dlopen/dlsym and library loading errors for apps (Available in Android O) - -Starting with Android O it is possible to enable logging of dynamic -linker activity for debuggable apps by setting a property corresponding -to the fully-qualified name of the specific app: -``` -adb shell setprop debug.ld.app.com.example.myapp dlerror,dlopen,dlsym -adb logcat -``` - -Any combination of `dlerror`, `dlopen`, and `dlsym` can be used. There's -no separate `dlclose` option: `dlopen` covers both loading and unloading -of libraries. Note also that `dlerror` doesn't correspond to actual -calls of dlerror(3) but to any time the dynamic linker writes to its -internal error buffer, so you'll see any errors the dynamic linker would -have reported, even if the code you're debugging doesn't actually call -dlerror(3) itself. - -On userdebug and eng builds it is possible to enable tracing for the -whole system by using the `debug.ld.all` system property instead of -app-specific one. For example, to enable logging of all dlopen(3) -(and thus dclose(3)) calls, and all failures, but not dlsym(3) calls: -``` -adb shell setprop debug.ld.all dlerror,dlopen -``` - -## dlclose interacts badly with thread local variables with non-trivial destructors - -Android allows `dlclose` to unload a library even if there are still -thread-local variables with non-trivial destructors. This leads to -crashes when a thread exits and attempts to call the destructor, the -code for which has been unloaded (as in [issue 360], fixed in P). - -[issue 360]: https://github.com/android-ndk/ndk/issues/360 - -Not calling `dlclose` or ensuring that your library has `RTLD_NODELETE` -set (so that calls to `dlclose` don't actually unload the library) -are possible workarounds. - -| | Pre-M | M+ | P+ | -| ----------------- | -------------------------- | ------- | ----- | -| No workaround | Works for static STL | Broken | Works | -| `-Wl,-z,nodelete` | Works for static STL | Works | Works | -| No `dlclose` | Works | Works | Works | +docs/android-changes-for-ndk-developers.md
\ No newline at end of file diff --git a/docs/android-changes-for-ndk-developers.md b/docs/android-changes-for-ndk-developers.md new file mode 100644 index 000000000..1630db8e4 --- /dev/null +++ b/docs/android-changes-for-ndk-developers.md @@ -0,0 +1,419 @@ +# Android changes for NDK developers + +This document details important changes related to native code +loading in various Android releases. + +Required tools: the NDK has an _arch_-linux-android-readelf binary +(e.g. arm-linux-androideabi-readelf or i686-linux-android-readelf) +for each architecture (under toolchains/), but you can use readelf for +any architecture, as we will be doing basic inspection only. On Linux +you need to have the “binutils” package installed for readelf, +and “pax-utils” for scanelf. + + +## How we manage incompatible changes + +Our general practice with dynamic linker behavior changes is that they +will be tied to an app's target API level: + +* Below the affected API level we'll preserve the old behavior or issue +a warning, as appropriate. + +* At the affected API level and above, we’ll refuse to load the library. + +* Warnings about any behavior change that will affect a library if you +increase your target API level will appear in logcat when that library +is loaded, even if you're not yet targeting that API level. + +* On a developer preview build, dynamic linker warnings will also show up +as toasts. Experience has shown that many developers don’t habitually +check logcat for warnings until their app stops functioning, so the +toasts help bring some visibility to the issues before it's too late. + +## Changes to library dependency resolution + +Until it was [fixed](https://issuetracker.google.com/36950617) in +JB-MR2, Android didn't include the application library directory +on the dynamic linker's search path. This meant that apps +had to call `dlopen` or `System.loadLibrary` on all transitive +dependencies before loading their main library. Worse, until it was +[fixed](https://issuetracker.google.com/36935779) in JB-MR2, the +dynamic linker's caching code cached failures too, so it was necessary +to topologically sort your libraries and load them in reverse order. + +If you need to support Android devices running OS +versions older than JB-MR2, you might want to consider +[ReLinker](https://github.com/KeepSafe/ReLinker) which claims to solve +these problems automatically. + +## Changes to library search order + +We have made various fixes to library search order when resolving symbols. + +With API 22, load order switched from depth-first to breadth-first to +fix dlsym(3). + +Before API 23, the default search order was to try the main executable, +LD_PRELOAD libraries, the library itself, and its DT_NEEDED libraries +in that order. For API 23 and later, for any given library, the dynamic +linker divides other libraries into the global group and the local +group. The global group is shared by all libraries and contains the main +executable, LD_PRELOAD libraries, and any library with the DF_1_GLOBAL +flag set (by passing “-z global” to ld(1)). The local group is +the breadth-first transitive closure of the library and its DT_NEEDED +libraries. The M dynamic linker searches the global group followed by +the local group. This allows ASAN, for example, to ensure that it can +intercept any symbol. + + +## RTLD_LOCAL (Available in API level >= 23) + +The dlopen(3) RTLD_LOCAL flag used to be ignored but is implemented +correctly in API 23 and later. Note that RTLD_LOCAL is the default, +so even calls to dlopen(3) that didn’t explicitly use RTLD_LOCAL will +be affected (unless they explicitly used RTLD_GLOBAL). With RTLD_LOCAL, +symbols will not be made available to libraries loaded by later calls +to dlopen(3) (as opposed to being referenced by DT_NEEDED entries). + + +## GNU hashes (Availible in API level >= 23) + +The GNU hash style available with --hash-style=gnu allows faster +symbol lookup and is now supported by the dynamic linker in API 23 and +above. (Use --hash-style=both if you want to build code that uses this +feature >= Android M but still works on older releases.) + + +## Correct soname/path handling (Available in API level >= 23) + +The dynamic linker now understands the difference +between a library’s soname and its path (public bug +https://code.google.com/p/android/issues/detail?id=6670). API level 23 +is the first release where search by soname is implemented. Earlier +releases would assume that the basename of the library was the soname, +and used that to search for already-loaded libraries. For example, +`dlopen("/this/directory/does/not/exist/libc.so", RTLD_NOW)` would +find `/system/lib/libc.so` because it’s already loaded. This also meant +that it was impossible to have two libraries `"dir1/libx.so"` and +`"dir2/libx.so"` --- the dynamic linker couldn’t tell the difference +and would always use whichever was loaded first, even if you explicitly +tried to load both. This also applied to DT_NEEDED entries. + +Some apps have bad DT_NEEDED entries (usually absolute paths on the build +machine’s file system) that used to work because we ignored everything +but the basename. These apps will fail to load on API level 23 and above. + + +## Symbol versioning (Available in API level >= 23) + +Symbol versioning allows libraries to provide better backwards +compatibility. For example, if a library author knowingly changes +the behavior of a function, they can provide two versions in the same +library so that old code gets the old version and new code gets the new +version. This is supported in API level 23 and above. + + +## Opening shared libraries directly from an APK + +In API level 23 and above, it’s possible to open a .so file directly from +your APK. Just use `System.loadLibrary("foo")` exactly as normal but set +`android:extractNativeLibs="false"` in your `AndroidManifest.xml`. In +older releases, the .so files were extracted from the APK file +at install time. This meant that they took up space in your APK and +again in your installation directory (and this was counted against you +and reported to the user as space taken up by your app). Any .so file +that you want to load directly from your APK must be page aligned +(on a 4096-byte boundary) in the zip file and stored uncompressed. +Current versions of the zipalign tool take care of alignment. + +Note that in API level 23 and above dlopen(3) will open a library from +any zip file, not just your APK. Just give dlopen(3) a path of the form +"my_zip_file.zip!/libs/libstuff.so". As with APKs, the library must be +page-aligned and stored uncompressed for this to work. + + +## Private API (Enforced for API level >= 24) + +Native libraries must use only public API, and must not link against +non-NDK platform libraries. Starting with API 24 this rule is enforced and +applications are no longer able to load non-NDK platform libraries. The +rule is enforced by the dynamic linker, so non-public libraries +are not accessible regardless of the way code tries to load them: +System.loadLibrary, DT_NEEDED entries, and direct calls to dlopen(3) +will all work exactly the same. + +Users should have a consistent app experience across updates, +and developers shouldn't have to make emergency app updates to +handle platform changes. For that reason, we recommend against using +private C/C++ symbols. Private symbols aren't tested as part of the +Compatibility Test Suite (CTS) that all Android devices must pass. They +may not exist, or they may behave differently. This makes apps that use +them more likely to fail on specific devices, or on future releases --- +as many developers found when Android 6.0 Marshmallow switched from +OpenSSL to BoringSSL. + +In order to reduce the user impact of this transition, we've identified +a set of libraries that see significant use from Google Play's +most-installed apps, and that are feasible for us to support in the +short term (including libandroid_runtime.so, libcutils.so, libcrypto.so, +and libssl.so). In order to give you more time to transition, we will +temporarily support these libraries; so if you see a warning that means +your code will not work in a future release -- please fix it now! + +In O and later, the system property `debug.ld.greylist_disabled` can be +used to deny access to the greylist even to an app that would normally +be allowed it. This allows you to test compatibility without bumping the +app's `targetSdkVersion`. Use `setprop debug.ld.greylist_disabled true` +to turn this on (any other value leaves the greylist enabled). + +``` +$ readelf --dynamic libBroken.so | grep NEEDED + 0x00000001 (NEEDED) Shared library: [libnativehelper.so] + 0x00000001 (NEEDED) Shared library: [libutils.so] + 0x00000001 (NEEDED) Shared library: [libstagefright_foundation.so] + 0x00000001 (NEEDED) Shared library: [libmedia_jni.so] + 0x00000001 (NEEDED) Shared library: [liblog.so] + 0x00000001 (NEEDED) Shared library: [libdl.so] + 0x00000001 (NEEDED) Shared library: [libz.so] + 0x00000001 (NEEDED) Shared library: [libstdc++.so] + 0x00000001 (NEEDED) Shared library: [libm.so] + 0x00000001 (NEEDED) Shared library: [libc.so] +``` + +*Potential problems*: starting from API 24 the dynamic linker will not +load private libraries, preventing the application from loading. + +*Resolution*: rewrite your native code to rely only on public API. As a +short term workaround, platform libraries without complex dependencies +(libcutils.so) can be copied to the project. As a long term solution +the relevant code must be copied to the project tree. SSL/Media/JNI +internal/binder APIs should not be accessed from the native code. When +necessary, native code should call appropriate public Java API methods. + +A complete list of public libraries is available within the NDK, under +platforms/android-API/usr/lib. + +Note: SSL/crypto is a special case, applications must NOT use platform +libcrypto and libssl libraries directly, even on older platforms. All +applications should use GMS Security Provider to ensure they are protected +from known vulnerabilities. + + +## Missing Section Headers (Enforced for API level >= 24) + +Each ELF file has additional information contained in the section +headers. These headers must be present now, because the dynamic linker +uses them for sanity checking. Some developers strip them in an +attempt to obfuscate the binary and prevent reverse engineering. (This +doesn't really help because it is possible to reconstruct the stripped +information using widely-available tools.) + +``` +$ readelf --header libBroken.so | grep 'section headers' + Start of section headers: 0 (bytes into file) + Size of section headers: 0 (bytes) + Number of section headers: 0 +``` + +*Resolution*: remove the extra steps from your build that strip section +headers. + +## Text Relocations (Enforced for API level >= 23) + +Starting with API 23, shared objects must not contain text +relocations. That is, the code must be loaded as is and must not be +modified. Such an approach reduces load time and improves security. + +The usual reason for text relocations is non-position independent +hand-written assembler. This is not common. Use the scanelf tool as +described in our documentation for further diagnostics: + +``` +$ scanelf -qT libTextRel.so + libTextRel.so: (memory/data?) [0x15E0E2] in (optimized out: previous simd_broken_op1) [0x15E0E0] + libTextRel.so: (memory/data?) [0x15E3B2] in (optimized out: previous simd_broken_op2) [0x15E3B0] + ... +``` + +If you have no scanelf tool available, it is possible to do a basic +check with readelf instead, look for either a TEXTREL entry or the +TEXTREL flag. Either alone is sufficient. (The value corresponding to the +TEXTREL entry is irrelevant and typically 0 --- simply the presence of +the TEXTREL entry declares that the .so contains text relocations). This +example has both indicators present: + +``` +$ readelf --dynamic libTextRel.so | grep TEXTREL + 0x00000016 (TEXTREL) 0x0 + 0x0000001e (FLAGS) SYMBOLIC TEXTREL BIND_NOW +``` + +Note: it is technically possible to have a shared object with the TEXTREL +entry/flag but without any actual text relocations. This doesn't happen +with the NDK, but if you're generating ELF files yourself make sure +you're not generating ELF files that claim to have text relocations, +because the Android dynamic linker trusts the entry/flag. + +*Potential problems*: Relocations enforce code pages being writable, and +wastefully increase the number of dirty pages in memory. The dynamic +linker has issued warnings about text relocations since Android K +(API 19), but on API 23 and above it refuses to load code with text +relocations. + +*Resolution*: rewrite assembler to be position independent to ensure +no text relocations are necessary. The +[Gentoo Textrels guide](https://wiki.gentoo.org/wiki/Hardened/Textrels_Guide) +has instructions for fixing text relocations, and more detailed +[scanelf documentation](https://wiki.gentoo.org/wiki/Hardened/PaX_Utilities). + + +## Invalid DT_NEEDED Entries (Enforced for API level >= 23) + +While library dependencies (DT_NEEDED entries in the ELF headers) can be +absolute paths, that doesn't make sense on Android because you have +no control over where your library will be installed by the system. A +DT_NEEDED entry should be the same as the needed library's SONAME, +leaving the business of finding the library at runtime to the dynamic +linker. + +Before API 23, Android's dynamic linker ignored the full path, and +used only the basename (the part after the last ‘/') when looking +up the required libraries. Since API 23 the runtime linker will honor +the DT_NEEDED exactly and so it won't be able to load the library if +it is not present in that exact location on the device. + +Even worse, some build systems have bugs that cause them to insert +DT_NEEDED entries that point to a file on the build host, something that +cannot be found on the device. + +``` +$ readelf --dynamic libSample.so | grep NEEDED + 0x00000001 (NEEDED) Shared library: [libm.so] + 0x00000001 (NEEDED) Shared library: [libc.so] + 0x00000001 (NEEDED) Shared library: [libdl.so] + 0x00000001 (NEEDED) Shared library: +[C:\Users\build\Android\ci\jni\libBroken.so] +``` + +*Potential problems*: before API 23 the DT_NEEDED entry's basename was +used, but starting from API 23 the Android runtime will try to load the +library using the path specified, and that path won't exist on the +device. There are broken third-party toolchains/build systems that use +a path on a build host instead of the SONAME. + +*Resolution*: make sure all required libraries are referenced by SONAME +only. It is better to let the runtime linker to find and load those +libraries as the location may change from device to device. + + +## Missing SONAME (Enforced for API level >= 23) + +Each ELF shared object (“native library”) must have a SONAME (Shared +Object Name) attribute. The NDK toolchain adds this attribute by default, +so its absence indicates either a misconfigured alternative toolchain +or a misconfiguration in your build system. A missing SONAME may lead +to runtime issues such as the wrong library being loaded: the filename +is used instead when this attribute is missing. + +``` +$ readelf --dynamic libWithSoName.so | grep SONAME + 0x0000000e (SONAME) Library soname: [libWithSoName.so] +``` + +*Potential problems*: namespace conflicts may lead to the wrong library +being loaded at runtime, which leads to crashes when required symbols +are not found, or you try to use an ABI-incompatible library that isn't +the library you were expecting. + +*Resolution*: the current NDK generates the correct SONAME by +default. Ensure you're using the current NDK and that you haven't +configured your build system to generate incorrect SONAME entries (using +the -soname linker option). + + +## DT_RUNPATH support (Available in API level >= 24) + +If an ELF file contains a DT_RUNPATH entry, the directories listed there +will be searched to resolve DT_NEEDED entries. The string `${ORIGIN}` will +be rewritten at runtime to the directory containing the ELF file. This +allows the use of relative paths. The `${LIB}` and `${PLATFORM}` +substitutions supported on some systems are not currently implemented on +Android. + + +## Writable and Executable Segments (Enforced for API level >= 26) + +Each segment in an ELF file has associated flags that tell the +dynamic linker what permissions to give the corresponding page in +memory. For security, data shouldn't be executable and code shouldn't be +writable. This means that the W (for Writable) and E (for Executable) +flags should be mutually exclusive. This wasn't historically enforced, +but is now. + +``` +$ readelf --program-headers -W libBadFlags.so | grep WE + LOAD 0x000000 0x00000000 0x00000000 0x4c01d 0x4c01d RWE 0x1000 +``` + +*Resolution*: we're aware of one middleware product that introduces these +into your app. The middleware vendor is aware of the problem and has a fix +available. + +## Invalid ELF header/section headers (Enforced for API level >= 26) + +In API level 26 and above the dynamic linker checks more values in +the ELF header and section headers and fails if they are invalid. + +*Example error* +``` +dlopen failed: "/data/data/com.example.bad/lib.so" has unsupported e_shentsize: 0x0 (expected 0x28) +``` + +*Resolution*: don't use tools that produce invalid/malformed +ELF files. Note that using them puts application under high risk of +being incompatible with future versions of Android. + +## Enable logging of dlopen/dlsym and library loading errors for apps (Available in Android O) + +Starting with Android O it is possible to enable logging of dynamic +linker activity for debuggable apps by setting a property corresponding +to the fully-qualified name of the specific app: +``` +adb shell setprop debug.ld.app.com.example.myapp dlerror,dlopen,dlsym +adb logcat +``` + +Any combination of `dlerror`, `dlopen`, and `dlsym` can be used. There's +no separate `dlclose` option: `dlopen` covers both loading and unloading +of libraries. Note also that `dlerror` doesn't correspond to actual +calls of dlerror(3) but to any time the dynamic linker writes to its +internal error buffer, so you'll see any errors the dynamic linker would +have reported, even if the code you're debugging doesn't actually call +dlerror(3) itself. + +On userdebug and eng builds it is possible to enable tracing for the +whole system by using the `debug.ld.all` system property instead of +app-specific one. For example, to enable logging of all dlopen(3) +(and thus dclose(3)) calls, and all failures, but not dlsym(3) calls: +``` +adb shell setprop debug.ld.all dlerror,dlopen +``` + +## dlclose interacts badly with thread local variables with non-trivial destructors + +Android allows `dlclose` to unload a library even if there are still +thread-local variables with non-trivial destructors. This leads to +crashes when a thread exits and attempts to call the destructor, the +code for which has been unloaded (as in [issue 360], fixed in P). + +[issue 360]: https://github.com/android-ndk/ndk/issues/360 + +Not calling `dlclose` or ensuring that your library has `RTLD_NODELETE` +set (so that calls to `dlclose` don't actually unload the library) +are possible workarounds. + +| | Pre-M | M+ | P+ | +| ----------------- | -------------------------- | ------- | ----- | +| No workaround | Works for static STL | Broken | Works | +| `-Wl,-z,nodelete` | Works for static STL | Works | Works | +| No `dlclose` | Works | Works | Works | |