PyTorch + LightGBM OpenMP segfault on macOS ARM64

[bruAristimunha]

Summary

Tutorials that use both PyTorch and LightGBM segfault on macOS ARM64 due to competing OpenMP runtimes. Both libraries link against different OpenMP implementations (libomp vs libgomp/libiomp5), and when LightGBM initializes multi-threaded OpenMP after PyTorch has already initialized its own, the process crashes with SIGSEGV.

Reproduction

import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

# Train any torch model
X = torch.randn(100, 50); y = torch.randint(0, 2, (100,))
model = nn.Sequential(nn.Linear(50, 100), nn.Linear(100, 2))
for x, t in DataLoader(TensorDataset(X, y), batch_size=10):
    loss = torch.nn.functional.cross_entropy(model(x), t)
    loss.backward(); 

# Now use LightGBM with multi-threading -> SEGFAULT
from lightgbm import LGBMClassifier
import numpy as np
clf = LGBMClassifier(n_jobs=-1)  # <-- crashes
clf.fit(np.random.randn(100, 10), np.random.randint(0, 2, 100))

Workaround

Set n_jobs=1 on LGBMClassifier/LGBMRegressor in tutorials that also use PyTorch. This forces LightGBM to use single-threaded mode, avoiding the OpenMP conflict.

Applied in commit 0200af7 to:

• examples/core/tutorial_feature_extractor_open_close_eye.py
• examples/tutorials/noplot_tutorial_feature_extraction.py
• examples/tutorials/noplot_tutorial_pfactor_features.py

Affected environment

• macOS ARM64 (Apple Silicon)
• pip-installed PyTorch and LightGBM (each bundles its own OpenMP)
• Does not affect Linux CI (Ubuntu x86_64) where OpenMP runtimes are compatible

Potential permanent fix

Install both packages via conda/mamba with a shared OpenMP runtime (libomp), or pin compatible builds. Alternatively, set OMP_NUM_THREADS=1 globally during docs builds.