[WIP] feat: multi-thread support - #2853
Draft
irozzo-1A wants to merge 35 commits into
Draft
Conversation
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: irozzo-1A The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
irozzo-1A
force-pushed
the
experiment/folly-concurrent-hashmap
branch
from
February 25, 2026 01:03
f51d695 to
e06f862
Compare
poiana
added
dco-signoff: yes
dco-signoff: no
and removed
dco-signoff: no
dco-signoff: yes
labels
irozzo-1A
force-pushed
the
experiment/folly-concurrent-hashmap
branch
from
February 25, 2026 08:31
58be2dd to
8ef1ed0
Compare
irozzo-1A
force-pushed
the
experiment/folly-concurrent-hashmap
branch
from
February 25, 2026 15:02
8c2726d to
e1dab17
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2853 +/- ##
==========================================
+ Coverage 75.76% 76.39% +0.62%
==========================================
Files 299 311 +12
Lines 33099 34437 +1338
Branches 5126 5432 +306
==========================================
+ Hits 25079 26309 +1230
- Misses 8020 8128 +108
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
irozzo-1A
force-pushed
the
experiment/folly-concurrent-hashmap
branch
from
February 26, 2026 10:42
617960b to
32689a5
Compare
irozzo-1A
force-pushed
the
experiment/folly-concurrent-hashmap
branch
from
February 27, 2026 15:10
53b8b73 to
0c94fc4
Compare
|
Please double check driver/API_VERSION file. See versioning. /hold |
Add missing terminating quotes to buffers-num option description string literals Signed-off-by: irozzo-1A <iacopo@sysdig.com>
/home/runner/work/libs/libs/userspace/libsinsp/parsers.h:39:1: error: 'sinsp_parser_shared_params' defined as a struct here but previously declared as a class; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Werror,-Wmismatched-tags]
39 | struct sinsp_parser_shared_params {
| ^
/home/runner/work/libs/libs/userspace/libsinsp/sinsp.h:96:1: note: did you mean struct here?
96 | class sinsp_parser_shared_params;
| ^~~~~
| struct
/home/runner/work/libs/libs/userspace/libsinsp/sinsp_buffer.h:27:1: note: did you mean struct here?
27 | class sinsp_parser_shared_params;
| ^~~~~
| struct
1 error generated
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
open_common() called init() before allocating m_buffers. init() runs consume_initialstate_events() when is_capture(), which uses m_buffers.at(SINSP_INVALID_BUFFER_HANDLE). With an empty m_buffers this threw std::out_of_range when opening a savefile (e.g. in plugin_dump test). Move the m_buffers allocation block to before init() so the default buffer exists when consume_initialstate_events() runs. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Force reset of last_access_time in remove_inactive_threads method of sinsp_with_test_input. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
error: "C atomics require C11 or later" Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Extend BPF iterator logic to be useful in multi-thread mode. Specifically, BPF iterator maps have different shapes depending on the number of "maximum allowed iterator threads" (that is specified at probe initialization through a `iters_num` parameter): - if there is at most a single iterator, these will be array maps with a single entry; - otherwise, they will be hash maps with an entry for each iterator, indexed by the thread id of the iterator thread. Before starting event fetching, the implementation checks if it is the first time that the calling thread uses one of the `pman_fetch_*` APIs and, if it is, initializes maps to take the new ecountered thread into account (specifically, this means that both `iter_auxiliary_map` and `iter_counters_map` are initialized). Iterator maps initialization is done from userspace, and leverages thread local flags and atomic counters to ensure maps are not initialized twice for the same thread, and that no more than `iters_num` different threads will ever call `pman_fetch_*` APIs. In multi-thread mode, iterator maps initialization is done by adding a new entry for each encountered new thread to each map. Entries' values are initialized leveraging zeroed values allocated in static memory. In single-thread mode, the array map has simply a pre-initialized single entry, and no additional work is required when the single thread is encountered for the first time. The whole design makes the switch between single- and multi-thread mode transparent to users, apart from the `iters_num` probe parameter: this must be evaluated in advance, and include any thread that will ever use any `fetch_*` API. Each iterator metric is the result of the sum of the corresponding counters on each `iter_counters_map` entry. Signed-off-by: Leonardo Di Giovanna <leonardodigiovanna1@gmail.com>
Add design proposals documenting the approach for making libsinsp's thread manager and usergroup manager thread-safe using Folly ConcurrentHashMap and fine-grained locking. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
… option Add cmake modules for building a minimal Folly subset (ConcurrentHashMap only) with its dependencies (fmt, double-conversion, glog). Introduce the ENABLE_MULTI_THREAD cmake option that gates concurrent data structures and real mutexes in libsinsp. Update CI workflows to install Folly build dependencies and add the multi-thread build matrix entry. Signed-off-by: irozzo-1A <iacopo@sysdig.com> Co-authored-by: Leonardo Grasso <leonardo.grasso@sysdig.com>
Introduce SyncPolicy template infrastructure (sync_policy.h) and refactor sinsp_threadinfo and sinsp_thread_manager into template classes parameterized on SyncPolicy. Add threadinfo_map.h backed by Folly ConcurrentHashMap when ENABLE_MULTI_THREAD is set. Add getter/setter API for threadinfo fields (m_pid, m_comm, etc.) with atomic/mutex-backed implementations. Fields remain public during the migration period to maintain backward compatibility with existing callers. Update forward declarations across the codebase to use the new template form. Co-locate test updates required by breaking API changes (get_main_thread() now returns shared_ptr, get_first_thread() returns shared_ptr). Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Template sinsp_fdinfo and sinsp_fdtable on SyncPolicy, mirroring the thread manager approach. Add getter/setter API for fdinfo fields with mutex-backed implementations and exclusive_lock() for compound operations. Fields remain public during migration. Introduce atomic_helpers.h for copyable atomic flag used in fdinfo name change tracking. Update fdtable to use Folly ConcurrentHashMap when ENABLE_MULTI_THREAD is set, with shared_ptr-based fd lookups. Co-locate test updates for breaking fdinfo API changes. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Migrate parsers.cpp to use getter/setter API for threadinfo and fdinfo access. Use exclusive_lock() for compound fdinfo mutations, shared_ptr for fd lookups, and per-buffer parser state for multi-thread safety. Co-locate all parser test updates that adapt to the new API (fdinfo getters, threadinfo getters, shared_ptr fd lookups). Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Migrate all filtercheck implementations to use getter/setter API for fdinfo and threadinfo access. Add sockinfo cache in sinsp_filtercheck_fd for thread-safe socket info access. Update filter_check_list with clone support for per-worker filtercheck instances. Co-locate filtercheck test updates. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Seal the getter/setter encapsulation by moving threadinfo fields (m_pid, m_comm, m_pgid, etc.) and fdinfo fields (m_type, m_openflags, m_name, m_sockinfo, etc.) to private sections. All callers have been migrated in previous commits; any remaining direct access would now produce a compile error, confirming migration completeness. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Add mutex protection to sinsp_usergroup_manager for concurrent access to user and group lookup tables. Include concurrent unit tests for parallel user/group resolution. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
…d plumbing Implement multi-buffer event processing in sinsp with per-buffer parsers and filter check lists. Use shared_ptr for threadinfo/fdinfo in sinsp_evt to ensure safe cross-buffer access. Add reserve_buffer_handle() API for worker thread allocation. Update sinsp_suppress to use ConcurrentHashMap for thread-safe tid suppression. Add thread-local stats counters in metrics_collector. Refactor event.h to remove raw pointer members in favor of shared_ptr. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Make scap error handling thread-safe with thread-local strerror buffers and per-buffer event counters. Add encoded buffer handle support for multi-buffer event retrieval. Update engine vtable to support buffer-specific next() calls. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Route sched_process_fork events to the child process's ring buffer instead of the parent's, ensuring the correct worker thread processes fork events in multi-threaded mode. Fix libpman BPF iterator maps to use root-namespace TID for correct process enumeration across PID namespaces. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Add TSAN suppression file for known benign races in third-party code (Folly hazard pointers, etc.). Update test CMakeLists.txt with ENABLE_MULTI_THREAD gating for concurrent test targets. Adapt e2e tests to use getter/setter API for threadinfo and fdinfo access, matching the refactored libsinsp interfaces. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
The ${ZIG_TOOLCHAIN_FILE:+-DCMAKE_TOOLCHAIN_FILE=$ZIG_TOOLCHAIN_FILE}
parameter expansion was dropped when -DENABLE_MULTI_THREAD=ON was added
to the build step. Without it, the zig build variant uses the system
compiler and links against the host glibc (~2.36 on bookworm) instead
of targeting glibc 2.17 via the zig toolchain.
Signed-off-by: irozzo-1A <iacopo@sysdig.com>
…rings Replace the per-fdinfo std::shared_mutex with a lightweight seqlock for multi-field consistency (type + sockinfo), relaxed atomic helpers for scalar fields, and COW std::shared_ptr<const std::string> for string fields. This eliminates reader-side locking overhead on the hot path while preserving thread safety under concurrent multi-worker processing. - Scalar getters (get_type, get_flags, etc.) use lock-free load_relaxed - String getters (get_name, get_name_raw, get_oldname) use atomic_load on COW shared_ptr — zero-copy for readers - Seqlock read() protects multi-field snapshots (type + sockinfo, clone) - Writers hold sinsp_seqlock_write_guard (CAS-based exclusive access) - State framework gains COW string field support (define_cow_string_field) - TSAN suppression added for seqlock speculative reader false positives Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Publish comm/exe/scalar exec fields in an immutable snapshot so hot-path getters avoid m_state_mutex; wire state framework fields through lambdas that read the snapshot instead of duplicated shadow members. Keep the mutex only for args/env/cgroups vectors and cwd read-modify-write. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Treat a null m_name like the pre-refactor default empty string when comparing names, so storing "" no longer sets name_changed once. Signed-off-by: irozzo-1A <iacopo@sysdig.com>
Replace get_thread_list() copy-based iteration with callback-based for_each_thread() and predicate-based find_thread() methods that iterate under a shared lock without copying the weak_ptr list. On ARM64 multi-threaded workloads, get_thread_list() was a major CPU hotspot due to: - Full std::list<weak_ptr> copy on every call (memory allocation) - Atomic reference counting (LDAXR/STLXR) for each weak_ptr copy - High contention on jemalloc from frequent alloc/dealloc cycles The new methods hold the shared_mutex reader lock and iterate in-place, promoting each weak_ptr to shared_ptr only once and invoking the caller's lambda directly. This eliminates the intermediate list copy entirely. Migrated callers: - create_thread_dependencies: update_main_fdtable iteration - find_new_reaper (both local and ancestor group searches) - parse_execve_exit: collect TIDs for removal Signed-off-by: irozzo-1A <iacopo@sysdig.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Any specific area of the project related to this PR?
Does this PR require a change in the driver versions?
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
The falco branch with multi-thread support can be found here.
To build assuming the two repositories are in the same directory, use the following:
Does this PR introduce a user-facing change?: