[cpp] If the messages are not writen by their logtime, MCapReader will use lots of memory to read this file

[Li-Zhaoxi]

Description
I use multi-threads to process the lidar files, each file will be directly writen into the mcap. Therefore, messages are not in log time order. The MCap file is 24GB (compression is zstd),.

When I use the MCapReader to read this file, the memory usage continuously increased, eventually reaching 30GB.

In cpp/mcap/include/mcap/reader.hpp, std::vector<ChunkSlot> chunkSlots_ is used to record decompressedChunk (ByteArray). After reading this file, the slot size is 4520. When unreadMessages == 0, decompressedChunk will not be released automatically.

Is there a better way to read the mcap file that the messages are not stored in logtime order.

• Version: v1.4.1
• Platform: Linux/Arm64

[linear]

FIRE-255

[james-rms]

Do you care about messages being read out in log time order? You can select File order and you will not see the same level of buffering occur.