PR Category

Inference

PR Types

Bug fixes

Description

The provided C++ code snippet contains a bug related to tensor allocation:

for (const auto& it : buffers) {
    phi::DenseTensor temp_tensor;
    temp_tensor.Resize(data_shape);
    data_tensors_.push_back(temp_tensor);
    data_buffers_[input_name] = std::pair<void*, size_t>(
        static_cast<void*>(temp_tensor.mutable_data<int16_t>(place)),
        data_size);
}

• When temp_tensor is pushed into data_tensors_ using data_tensors_.push_back(temp_tensor);, the tensor’s memory is not yet allocated. The Resize function only changes the tensor's shape without allocating memory.
• Memory allocation occurs only when mutable_data is called (see PaddlePaddle's source code).
• The buffer data_buffers_ is set to point to temp_tensor.mutable_data<int16_t>(place), but since the tensor in data_tensors_ has no allocated memory, it leads to an invalid memory reference.
• At the end of the scope, temp_tensor deallocates, leaving data_tensors_ with an invalid pointer.

With a stream-safe allocator, this bug is not immediately apparent because the allocator does not trigger cudaFree immediately. However, with an async allocator, the bug is detected due to stricter memory management.

To fix this bug, reorder the operations to allocate memory before pushing the tensor into data_tensors_