Skip to content

Dpdk often stuck in write log to /var/log/message when start VM?  #4

@jesonliang

Description

@jesonliang

1, The backtrace for vhost that it hit this issue
(gdb) info threads
Id Target Id Frame
7 Thread 0x7faa860ed700 (LWP 2179359) "eal-intr-thread" 0x00007faa86612923 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
6 Thread 0x7faa858ec700 (LWP 2179363) "reactor_4" 0x0000000000485e27 in reactor_post_process_lw_thread ()
5 Thread 0x7faa850eb700 (LWP 2179364) "reactor_5" 0x00000000004862ec in reactor_run ()
4 Thread 0x7faa848ea700 (LWP 2179365) "reactor_6" 0x00000000004b4612 in spdk_get_ticks ()
3 Thread 0x7faa840e9700 (LWP 2179366) "vhost" 0x00007faa868eb98d in accept () at ../sysdeps/unix/syscall-template.S:81
2 Thread 0x7faa7afd7700 (LWP 2179400) "vhost-events" 0x00007faa8660385d in write () at ../sysdeps/unix/syscall-template.S:81

  • 1 Thread 0x7faa87b88bc0 (LWP 2179357) "reactor_3" __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
    (gdb) bt
    #0 __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
    fix dpdk runtime bug based on spdk/dpdk #1 0x00007faa8656659f in _L_lock_1030 () from /lib64/libc.so.6
    Release 19.05? #2 0x00007faa86561344 in _IO_vfprintf_internal (s=0x1a8df80, format=0x5a0d18 "VHOST_CONFIG: vhost-user %s: socket created, fd: %d\n", ap=0x7ffc3e263808) at vfprintf.c:1313
    Dpdk build getting failed due to issue in rpath #3 0x00000000004bf355 in rte_vlog ()
    Dpdk often stuck in write log to /var/log/message when start VM?  #4 0x0000000000405f99 in rte_log ()
    Mention move to own git hosting  #5 0x00000000005091b2 in create_unix_socket ()
    Acomment #6 0x000000000050a1ef in rte_vhost_driver_register ()
    eal_common_options.c:verify_perms(): remove per-thread object. #7 0x00000000004833ac in vhost_register_unix_socket ()
    #8 0x000000000047d708 in vhost_dev_register ()
    #9 0x0000000000482d36 in spdk_vhost_blk_construct ()
    #10 0x000000000047eec5 in rpc_vhost_create_blk_controller ()
    #11 0x0000000000489c74 in jsonrpc_parse_request ()
    #12 0x000000000048a708 in spdk_jsonrpc_server_poll ()
    #13 0x0000000000486ca9 in rpc_subsystem_poll ()
    #14 0x00000000004a4e61 in spdk_thread_poll ()
    #15 0x0000000000486310 in reactor_run ()
    #16 0x0000000000486818 in spdk_reactors_start ()
    #17 0x000000000048440d in spdk_app_start ()
    #18 0x000000000040938f in main ()

2, Host is rehat 7.4 and kernal version is 3.10.0-693.el7.x86_64
3, spdk and dpdk version show as below.

Starting SPDK v21.01-pre git sha1 602b134 / DPDK 20.08.0 initialization...
[2021-06-30 12:45:22.064330] [ DPDK EAL parameters: [2021-06-30 12:45:22.064341] vhost [2021-06-30 12:45:22.064348] --no-shconf [2021-06-30 12:45:22.064355] -c 0x78 [2021-06-30 12:45:22.064361] --log-level=lib.eal:6 [2021-06-30 12:45:22.064367] --log-level=lib.cryptodev:5 [2021-06-30 12:45:22.064373] --log-level=user1:6 [2021-06-30 12:45:22.064380] --base-virtaddr=0x200000000000 [2021-06-30 12:45:22.064386] --match-allocations [2021-06-30 12:45:22.064393] --file-prefix=spdk_pid15246 [2021-06-30 12:45:22.064400] ]

  1. qemu thread run with parameter as below.
    -object memory-backend-file,id=mem0,size=4G,mem-path=/dev/hugepages,share=on -numa node,memdev=mem0 -chardev socket,id=spdk_vhost_blkc44a0c1e-df13-11eb-a280-525400a98761,path=/var/tmp/vhost.c44a0c1e-df13-11eb-a280-525400a98761,reconnect=1
    -device vhost-user-blk-pci,chardev=spdk_vhost_blkc44a0c1e-df13-11eb-a280-525400a98761,bootindex=1,num-queues=2 -chardev socket,id=spdk_vhost_blk12ac81ce-62ed-4b18-ae41-79ab44b34fbb,path=/var/tmp/vhost.12ac81ce-62ed-4b18-ae41-79ab44b34fbb,reconnect=1
    -device vhost-user-blk-pci,chardev=spdk_vhost_blk12ac81ce-62ed-4b18-ae41-79ab44b34fbb,num-queues=2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions