Skip to content

bug: OSError syslog message to long when deleting 500 static EVCs #613

Description

@viniarck

Ended up hitting this issue when developing https://github.com/kytos-ng/scalability-scripts, after provisionig 500 static EVPLs and then shutting down a link of the primary path, the impact is major since the log crash will impact on other operations resulting in retries that won't succeed:

2026-04-17 15:02:02,671 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Flows received summary: {switch: 00:00:00:00:00:00:00:01, flows_length: 1000}, {switch: 00:00:00:00:00:00:00:03, flows_length: 1000}, {switch: 00:00:00:00:00:00:00:02, flows_length: 1000},  total_flows_length: 3000
...

Arguments: None
--- Logging error ---
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.11/3.11.15/Frameworks/Python.framework/Versions/3.11/lib/python3.11/logging/handlers.py", line 1007, in emit
    self.socket.sendto(msg, self.address)
OSError: [Errno 40] Message too long
Call stack:
  File "/opt/homebrew/Cellar/python@3.11/3.11.15/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002, in _bootstrap
    self._bootstrap_inner()
  File "/opt/homebrew/Cellar/python@3.11/3.11.15/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Cellar/python@3.11/3.11.15/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
Message: "Send FlowMod from request command: delete, force: True, dpids: ['00:00:00:00:00:00:00:02'], flows[400, 600]: [{'cookie': 12301985525548328263, 'match': {'in_port': 2, 'dl_vlan': 201}, 'owner': 'mef

...
2026-04-17 15:02:43,392 - WARNING [kytos.core.retry] (thread_pool_app_342) Retry #1 for _send_flow_mods, seconds since start: 30.02
2026-04-17 15:02:43,393 - WARNING [kytos.core.retry] (thread_pool_app_271) Retry #1 for _send_flow_mods, seconds since start: 30.03
2026-04-17 15:02:43,444 - WARNING [kytos.core.retry] (thread_pool_app_214) Retry #1 for _send_flow_mods, seconds since start: 30.01
2026-04-17 15:02:43,445 - WARNING [kytos.core.retry] (thread_pool_app_141) Retry #1 for _send_flow_mods, seconds since start: 30.01
2026-04-17 15:02:43,459 - WARNING [kytos.core.retry] (thread_pool_app_276) Retry #1 for _send_flow_mods, seconds since start: 30.02

This happened on a macOS where syslog UDP maximum syslog socket is usually smaller by default than on Linux OSes, and officially we recommend Debian 12, where this typically won't be an issue. But, clearly, this is a sign that our default logging.ini with syslog UDP transport might not be appropriate, and folks who want to use it should add it themselves while making sure their OS max datagram size can handle it (also TCP transport on syslog will require other tweaks). I mean it's not appropriate since deliberately as a developer we decided to have richer and longer log messages, so if we'll keep long messages, we should not let potential serialization transport issues to happen by default.

But, notice that on Linux even if with a larger but eventually the problem can happen too, it's just a matter of having a larger payload, although in the parts where we have extensive verbose logs we tried to slice them, so that's also another point that's generating extra work on application level to be maintained just due to syslog being supported

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority_mediumMedium priority

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions