RI Geng series: potential memory leak caused by abuse of mutable in Google Pb structure

The algorithm communication protocol sets a big pb, Then go to pb It's stuffed with features . These characteristics will be different redis Table query get .redis Query different tables , Parse the return data , Because the callback function is made concurrent and asynchronous , Therefore, not all feature requests are serially serialized , So there will be a concurrent contention problem .

For custom complex objects, it is not pld type ,protobuf (c++) There are two ways to set the infill class The value of the object , Namely set_allocated_ and mutable_. This mutable_ If abused in a non thread safe environment , There will be a potential memory leak , This problem is more hidden .

Suppose we have an example ：

message browser_user_feature
{
     feature_user_download   user_download   = 1;
     feature_query_word  query_word = 2;
}

message UserFeature {
     browser_user_feature  user_feature = 1;
}

Then the user downloads the data and query The data is passed through different redis Table access . Then we register a callback function to parse , This is pseudo code

register_parse_func("key1",  Analytic user_download Callback function for )
register_parse_func("key2",  Analytic query_word Callback function for )

 Analytic user_download Callback function for (data, size) {
    result.mutable_user_feature()->mutable_user_download()->ParseFromString(data, size);

}

 Analytic query_word Callback function for (data, size) {
    result.mutable_user_feature()->mutable_query_word()->ParseFromString(data, size);
}

that mutable This aspect internally determines whether a null pointer exists , If it's a null pointer , Do object memory allocation .

 inline ::feature_process::browser_user_feature* UserFeature::mutable_browser_search_user_feature() {
 
   if (browser_user_feature_ == nullptr) {
     auto* p = CreateMaybeMessage<::feature_process::browser_user_feature>(GetArenaNoVirtual());
     browser_user_feature_ = p;
   }
   // @@protoc_insertion_point(field_mutable:feature_process.UserFeature.browser_user_feature)
   return browser_user_feature_;
 }

Use memory analysis tools ：gperftools  https://github.com/gperftools/gperftools, You will see a lot of memory allocated in this function CreateMaybeMessage. Note here mutable_xxx Not a thread safe function , So when it comes to distribution , There may be two or more assignments , Then even if the memory is released later, it will only be released once .

Then modify this bug What is the method of ？ We need to make sure that we call it first in a place where there is no thread safety problem mutable Method , Pre allocate memory , No new memory will be created for subsequent use .