pcap compression causes invalid caplen in packet headers

[dduranc]

I've noticed errors while trying to run npcapextract when n2disk is running with --pcap-compression:

# npcapextract -u root -t /storage/timeline/ -b "2026-01-07 12:03:00" -e "2026-01-07 12:03:10" -o out.pcap
Invalid caplen 1667457877 [offset: 931246]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783779.736712.npcap, 23999942) failed (-99)
Invalid caplen 1543504000 [offset: 947439]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783781.120325.npcap, 10384647) failed (-99)
Invalid caplen 956301440 [offset: 483383]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783782.872971.npcap, 28794959) failed (-99)
Invalid caplen 3976200320 [offset: 972979]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783785.620848.npcap, 15653067) failed (-99)
Invalid caplen 3858759808 [offset: 240426]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783788.172000.npcap, 31697730) failed (-99)
Invalid caplen 1258291308 [offset: 638458]
Error: npcap_read_at(/storage/timeline/2026/01/07/12/00/1767783788.695377.npcap, 25804306) failed (-99)
[npcapextract.c:1250] 16603931 packets (3585801414 bytes) matched the filter in 29.477 sec.
[npcapextract.c:1262] Dumped into 1 different output files.
[npcapextract.c:1272] Total processing time: 29.477 sec.

When trying to decompress one of the npcap files and open it in tshark, it shows the same error:

# npcapdecompress -i /storage/timeline/2026/01/07/12/00/1767783781.120325.npcap -o decompressed.pcap
# tshark -r decompressed.pcap
tshark: The file "decompressed.pcap" appears to be damaged or corrupt.
(pcap: File has 1543504000-byte packet, bigger than maximum of 262144)

n2disk is using pf_ring and ixgbe zc driver on intel X520

-i=zc:enp4s0f1
--buffer-len=8192
--index=/storage/index
--timeline-dir=/storage/timeline
-o=/storage/disk0
-o=/storage/disk1
-o=/storage/disk2
-o=/storage/disk3
-o=/storage/disk4
-o=/storage/disk5
-o=/storage/disk6
-o=/storage/disk7
-o=/storage/disk8
-o=/storage/disk9
-o=/storage/disk10
-o=/storage/disk11
--disk-limit=90%
--reader-cpu-affinity=0
--writer-cpu-affinity=2,4,6
--time-pulse=8
--compressor-cpu-affinity=10,12,14,16,18,20,22,24
--index-on-compressor-threads
--max-file-len=32
--trace-log=/var/log/n2disk.log
--event-log=/var/log/n2disk-event.log
--pcap-compression

The issue exists on both versions:
n2disk-3.8.251230-5548.x86_64 + pfring-9.2.0-10424.x86_64
n2disk-3.8.250501-5542.x86_64 + pfring-9.0.0-9880.x86_64

[cardigliano]

@dduranc I pushed a safety check (printing warnings) in nightly builds, are you able to switch to the dev repo and try to reproduce the same issue?

[dduranc]

I've switched to nightly build n2disk-3.9.260109-5571.x86_64 but I couldn't find any new warnings in either npcapextract output or n2disk logs

[cardigliano]

Just to make sure: if you run npcapdecompress on a corrupted npcap you do not get any warning. I will add some more check in this case.

[dduranc]

Yes, I do not get any additional warnings in nightly version but there are still errors about invalid caplen

[cardigliano]

@dduranc since I am unable to reproduce, are you able to provide a (corrupted) .npcap compressed file to take a look at the format perhaps? Thank you.