extracts via ntop and npcapextract only contain N packets from the beginning of each npcap file

[dethmetaljeff]

When I run npcapextract (and from ntopng gui) the pcaps returned/extracted appear to only include a number of packets from the beginning of each npcap file and not the entire thing.

517 packets from npcapextract

[19:48:11] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ npcapextract -t /opt/data/capture/n2disk/cu/timeline/ -o /tmp/test.pcap -b "2025-12-02 18:18:01" -e "2025-12-02 18:18:59"
[npcapextract.c:1250] 517 packets (183139 bytes) matched the filter in 0.118 sec.
[npcapextract.c:1262] Dumped into 1 different output files.
[npcapextract.c:1272] Total processing time: 0.118 sec.
[19:50:19] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ tcpdump -tttt -nnn -r /tmp/test.pcap | grep 18:18: | head -1
reading from file /tmp/test.pcap, link-type EN10MB (Ethernet), snapshot length 9100
2025-12-02 18:18:00.000662 IP 10.49.11.159.16702 > 10.20.84.15.5120: Flags [P.], seq 1432540:1432846, ack 2981780, win 51, options [nop,nop,TS val 184668566 ecr 705851126], length 306
tcpdump: Unable to write output: Broken pipe
[19:50:22] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ tcpdump -tttt -nnn -r /tmp/test.pcap | grep 18:18: | tail -1
reading from file /tmp/test.pcap, link-type EN10MB (Ethernet), snapshot length 9100
2025-12-02 18:18:00.056263 IP 10.20.84.15.6311 > 10.49.11.159.43887: Flags [.], ack 88, win 77, options [nop,nop,TS val 705851192 ecr 1742711442], length 0
[19:50:30] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$


# relevant debug bits from npcapextract
Info: Evaluating /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap
Debug: Found compressed PCAP file /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap
Info: Reading PCAP /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap
Info: Reading index /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap.idx
Debug: Index /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap.idx decompressed in 5 msec
Debug: Stats for /opt/data/capture/n2disk/cu/timeline/2025/12/02/18/10/1764699480.000662.npcap [52926883 bytes] [processed in 2 msec] [519 digests inspected] [518 packets extracted] [183507 bytes extracted]

this is from ntopng. if multiple npcap files are spanned, the first N packets from each file are included causing what looks like massive gaps in the resulting pcap file:

176750 packets by just decompressing and reading the relevant npcap file

[19:12:04] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ tcpdump -tttt -nnn -r dummy.pcap  | wc -l
reading from file dummy.pcap, link-type EN10MB (Ethernet), snapshot length 9100
176750
[19:12:09] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$
[19:12:09] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ tcpdump -tttt -nnn -r dummy.pcap  | head -1
reading from file dummy.pcap, link-type EN10MB (Ethernet), snapshot length 9100
2025-12-02 18:18:00.000662 IP 10.49.11.159.16702 > 10.20.84.15.5120: Flags [P.], seq 865110286:865110592, ack 707177642, win 51, options [nop,nop,TS val 184668566 ecr 705851126], length 306
tcpdump: Unable to write output: Broken pipe
[19:12:58] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$ tcpdump -tttt -nnn -r dummy.pcap  | tail -1
reading from file dummy.pcap, link-type EN10MB (Ethernet), snapshot length 9100
2025-12-02 18:18:59.999557 IP 10.49.10.202.19060 > 10.20.84.15.5120: Flags [P.], seq 1700136:1700442, ack 3301034, win 130, options [nop,nop,TS val 221234343 ecr 488087812], length 306
[19:13:02] [root@capture01.sec1:/opt/data/capture/n2disk/12/pcap/1764696115.595397]$

Not sure what other info you may need to help troubleshoot.

Same behavior on both of these:

[16:39:59] [root@capture01.chi1:~]$ rpm -qa | grep n2disk
n2disk-3.8.250917-5545.x86_64

[16:41:01] [root@capture01.sec1:~]$ rpm -qa | grep n2disk
n2disk-3.9.251118-5565.x86_64

[cardigliano]

@dethmetaljeff I ran a few tests and it seems to be working for me. Please provide your n2disk configuration file and any additional information (e.g. what MTU are you using?)

[dethmetaljeff]

@cardigliano

This is happening on both my servers. Here's one of the n2disk configs:

--interface=dummy6
--dump-directory=/opt/data/capture/n2disk/12/pcap
--index
--timeline-dir=/opt/data/capture/n2disk/12/timeline
--buffer-len=2048
--chunk-len=4096
#--active-wait
--pcap-compression
--max-file-len=1024
--max-file-duration=60
--disk-limit=212240
--snaplen=9100
--writer-cpu-affinity=27
--reader-cpu-affinity=13
--compressor-cpu-affinity=15,17
--index-on-compressor-threads
--unprivileged-user=n2disk
--capture-direction=0
--daemon
--pid=/var/run/n2disk-12.pid
--nanoseconds
--trace-log=/var/log/n2disk-12-trace.log
--event-log=/var/log/n2disk-12-event.log

The packets are coming by way of a napatech card stream -> zbalance_ipc -> dummy interface. The MTU on that dummy interface is 1500.

[cardigliano]

@dethmetaljeff this looks weird. I tried to reproduce it in many ways, even with the very same configuration, no luck (always working for me). If you are able to provide access to this system I can try debugging it there directly..