fix corruption in Vector<> when malloc falied

1. When alloc or realloc failed in the function SharedBuffer::editResize,
it would return a NULL pointer, then mStorage would update to be 1 by
SharedBuffer::data() if no pointer check here, which is an obviously
wrong address, and would cause corruption when used it e.g. in capacity().

So add the pointer check here for the return value of SharedBuffer::editResize,
if it's NULL do not use it to update mStorage, to avoid the value of mStorage
polluted.

2. when alloc or realloc falied in _grow & _shrink function, mStorage keep
the original value, so mCount should not be updated here.

Otherwise, mStorage might be 0 but mCount>0, so a corruption would happend
when it try to delete items from the Vector since mCount>0.

Change-Id: I7c3814e843c459834ca5eed392e8d63d1cb7d2d8
Signed-off-by: Shuo Gao <shuo.gao@intel.com>
Signed-off-by: Jian Luo <jian.luo@intel.com>
Signed-off-by: Bruce Beare <bruce.j.beare@intel.com>
Signed-off-by: Jack Ren <jack.ren@intel.com>
Author-tracking-BZ: 139626

1 files changed, 14 insertions, 2 deletions

diff --git a/libutils/VectorImpl.cpp b/libutils/VectorImpl.cpp
index 5a79647cb..30ca6635e 100644
--- a/libutils/VectorImpl.cpp
+++ b/libutils/VectorImpl.cpp
@@ -384,7 +384,11 @@ void* VectorImpl::_grow(size_t where, size_t amount)
         {
             const SharedBuffer* cur_sb = SharedBuffer::bufferFromData(mStorage);
             SharedBuffer* sb = cur_sb->editResize(new_capacity * mItemSize);
-            mStorage = sb->data();
+            if (sb) {
+                mStorage = sb->data();
+            } else {
+                return NULL;
+            }
         } else {
             SharedBuffer* sb = SharedBuffer::alloc(new_capacity * mItemSize);
             if (sb) {
@@ -399,6 +403,8 @@ void* VectorImpl::_grow(size_t where, size_t amount)
                 }
                 release_storage();
                 mStorage = const_cast<void*>(array);
+            } else {
+                return NULL;
             }
         }
     } else {
@@ -436,7 +442,11 @@ void VectorImpl::_shrink(size_t where, size_t amount)
         {
             const SharedBuffer* cur_sb = SharedBuffer::bufferFromData(mStorage);
             SharedBuffer* sb = cur_sb->editResize(new_capacity * mItemSize);
-            mStorage = sb->data();
+            if (sb) {
+                mStorage = sb->data();
+            } else {
+                return;
+            }
         } else {
             SharedBuffer* sb = SharedBuffer::alloc(new_capacity * mItemSize);
             if (sb) {
@@ -451,6 +461,8 @@ void VectorImpl::_shrink(size_t where, size_t amount)
                 }
                 release_storage();
                 mStorage = const_cast<void*>(array);
+            } else{
+                return;
             }
         }
     } else {