Improve compaction by skipping bad VMAs instead of fully bailing out

Previously when a VMA failed compaction for any reason the system would
stop trying to compact all the rest of the VMAs and bail out. However,
there are multiple reasons that a VMA can fail due to not being
reclaimable with -EINVAL, in such instances we will just skip the VMA
and keep going with the rest of the VMAs and any other error will still
cause compaction to bail out as it would likely be irrecoverable.

Test: Manual. Verified that once a VMA errors with -EINVAL, it continues
with the rest of the VMAs and other errors bail out.
Bug: 205658049
CRs-Fixed:3121545

Change-Id: Ifc190e371c4dce0eaa6dbab104aa0b666e06027d
(cherry picked from commit 1ec21f5ed784b9b6e49b012b2e99dafb71df6518)

Change-Id: I5642823d6303ad848771a2bf0d7cd466a2ccb589

1 files changed, 12 insertions, 2 deletions

diff --git a/services/core/jni/com_android_server_am_CachedAppOptimizer.cpp b/services/core/jni/com_android_server_am_CachedAppOptimizer.cpp
index f0271f806e13..0271277acb7a 100644
--- a/services/core/jni/com_android_server_am_CachedAppOptimizer.cpp
+++ b/services/core/jni/com_android_server_am_CachedAppOptimizer.cpp
@@ -84,6 +84,9 @@ static inline void compactProcessProcfs(int pid, const std::string& compactionTy
 
 // Compacts a set of VMAs for pid using an madviseType accepted by process_madvise syscall
 // Returns the total bytes that where madvised.
+//
+// If any VMA fails compaction due to -EINVAL it will be skipped and continue.
+// However, if it fails for any other reason, it will bail out and forward the error
 static int64_t compactMemory(const std::vector<Vma>& vmas, int pid, int madviseType) {
     static struct iovec vmasToKernel[MAX_VMAS_PER_COMPACTION];
 
@@ -148,8 +151,15 @@ static int64_t compactMemory(const std::vector<Vma>& vmas, int pid, int madviseT
         auto bytesProcessed = process_madvise(pidfd, vmasToKernel, iVec, madviseType, 0);
 
         if (CC_UNLIKELY(bytesProcessed == -1)) {
-            compactionInProgress = false;
-            return -errno;
+            if (errno == EINVAL) {
+                // This error is somewhat common due to an unevictable VMA if this is
+                // the case silently skip the bad VMA and continue compacting the rest.
+                continue;
+            } else {
+                // Forward irrecoverable errors and bail out compaction
+                compactionInProgress = false;
+                return -errno;
+            }
         }
 
         totalBytesProcessed += bytesProcessed;