A device's free-text `devComments` containing an unescaped `<` crashes the entire notification build (yattag.indentation.XMLTokenError), silently dropping every pending alert in the same cycle

[mormegil6]

What installation are you running?

Production (netalertx) 📦

Is there an existing issue for this?

• I have searched the existing open and closed issues and I checked the docs https://docs.netalertx.com/

The issue occurs in the following browsers. Select at least 2.

• Firefox
• Chrome
• Edge
• Safari (unsupported) - PRs welcome
• N/A - This is an issue with the backend

Current Behavior

If any device's devComments (or likely devName/devOwner - not individually tested, but they flow through the same unescaped path) contains a bare < that isn't valid XML markup, the entire notification build crashes with yattag.indentation.XMLTokenError - not just the one device's row, the whole process. This happens in notification.create(), well after all sections have already been assembled ("events section done." etc. all log successfully) - the crash is specifically in the final indent() call that pretty-prints the whole assembled HTML document.

Severity: this kills the entire main() process for that cycle via an unhandled exception, and since the crash happens before any event's evePendingAlertEmail flag gets cleared, the same poisoned pending event is picked up again on the next cycle and crashes identically - a permanent, recurring failure for every pending notification (not just the one with the bad comment) until someone manually clears or edits the offending row in the database. In my case this device was Meta Quest Pro, but any device is equally capable of triggering this via a normal free-text comment.

One thing worth flagging about the error message itself, since it's actively misleading: XMLTokenError's message is built from repr(string[:100]) - the first 100 characters of the entire document being tokenized, not the actual failing position. tokenize()'s regex match happens at an offset (start) into the original string, but the exception only ever shows string[:100], so the reported snippet is always just the document's opening HTML regardless of where the real problem is. In my case this pointed at an unrelated <span> near the top of the template, not the actual malformed content, which cost real time chasing the wrong lead before rebuilding the exact HTML locally with proper position tracking to find the real offset.

Expected Behavior

A single device's free-text field containing ordinary punctuation (parentheses, a <= comparison, anything a person might reasonably type into a comment box) should never be able to crash notification generation for the entire installation. At minimum, free-text fields should be HTML-escaped before being embedded into the notification's HTML; ideally the indent() call should also be resilient to a single malformed fragment (e.g. catch XMLTokenError and fall back to sending the unindented HTML rather than losing the whole notification).

Steps To Reproduce

Reproduced in an isolated Docker container (ghcr.io/jokob-sk/netalertx:latest, same image as production - confirmed on v26.8.5), separate from any production data, via the real scheduler (not manually invoked).

1. Fresh NetAlertX instance. LOG_LEVEL='trace'.
2. Insert one device with a devComments value containing a bare < followed by something that isn't a valid tag name - e.g. Test comment with a bare less-than: values <=2 break things - and one pending event for it (evePendingAlertEmail=1):

   INSERT INTO Devices (devMac, devName, devOwner, devType, devVendor, devFirstConnection, devLastConnection, devLastIP, devStaticIP, devPresentLastScan, devIsArchived, devParentMAC, devParentRelType, devComments)
   VALUES ('aa:bb:cc:dd:ee:ff', 'ReproDevice', '', '', 'TestVendor', datetime('now'), datetime('now'), '192.168.1.99', 0, 1, 0, '', 'default', 'Test comment with a bare less-than: values <=2 break things');
   INSERT INTO Events (eveMac, eveIp, eveDateTime, eveEventType, evePendingAlertEmail)
   VALUES ('aa:bb:cc:dd:ee:ff', '192.168.1.99', datetime('now', '-1 minutes'), 'Disconnected', 1);

3. Do not call any notification logic manually - just wait for the app's own normal scheduler to run the next notification check.

Result: the process crashes with the traceback below on the very next cycle. evePendingAlertEmail for the test event is still 1 afterward (confirmed via the same query) - the crash happens before it would ever be cleared, so this recurs on every subsequent cycle unmodified.

Relevant app.conf settings

TIMEZONE='Europe/Berlin'
ARPSCAN_RUN='schedule'
LOG_LEVEL='trace'

docker-compose.yml

services:
  netalertx:
    image: ghcr.io/jokob-sk/netalertx:latest
    container_name: netalertx-repro-htmlescape
    ports:
      - "11035:1031"
      - "10215:20212"
    environment:
      TZ: Europe/Warsaw
      PORT: 1031
      GRAPHQL_PORT: 20212
    volumes:
      - ./data:/data

(Isolated repro - bridge networking instead of production's `network_mode: host`, distinct ports so it doesn't collide with a real instance on the same host. Unlike the previous two isolated repros, no `cap_add` flags were needed here - the container started and ran normally without them for this specific test.)

Debug or Trace enabled

• I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.

Relevant app.log section

11:50:05 [Notification] Check if something to report
11:50:05 [Notification] Included sections:  ['new_devices', 'down_devices', 'events']
11:50:05 [Notification] Open text Template
11:50:05 [Notification] Open html Template
11:50:05 [Notification] new_devices section done.
11:50:05 [Notification] down_devices section done.
11:50:05 [Notification] down_reconnected section done.
11:50:05 [Notification] events section done.
11:50:05 [Notification] plugins section done.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/server/__main__.py", line 291, in <module>
    sys.exit(main())
             ^^^^^^
  File "/app/server/__main__.py", line 197, in main
    notificationObj = notification.create(final_json, "")
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/server/models/notification_instance.py", line 160, in create
    final_html = indent(
                 ^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/yattag/indentation.py", line 326, in indent
    tokens = tokenize(string)
             ^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/yattag/indentation.py", line 180, in tokenize
    raise XMLTokenError("Unrecognized XML token near %s" % repr(string[:100]))
yattag.indentation.XMLTokenError: Unrecognized XML token near '<html>\n<head></head>\n<body>\n    <span style="display:none !important;visibility:hidden;mso-hide:all;'

Note on that last line: as explained above, string[:100] here is misleading - it's just the start of the whole document, not where the actual bad token is. In this run the real failure was 3375 characters in, landing exactly inside the test device's devComments text (confirmed by rebuilding the exact same HTML locally and re-running tokenize() with real position tracking instead of yattag's own truncated message).

Docker Logs

netalertx-bug3-docker-logs.txt

[mormegil6]

Suggested fix

The unescaped value flows in through server/messaging/reporting.py's construct_notifications() (server/models/notification_instance.py, same file as create()), which passes the raw JSON rows straight into json2table.convert(). json2table has no escaping option - checked its convert() signature directly, it's a plain dict-to-HTML-table renderer with no concept of untrusted input. The generated table HTML then gets folded into the full document via plain string .replace(), and the whole thing is finally passed to yattag's indent(), which - per leforestier/yattag#32 - has always required valid markup as input; it's not designed to tolerate arbitrary raw text containing </>.

Two independent fixes, either would help, ideally both:

1. Escape free-text fields before they reach the table. At minimum devComments; likely worth doing for any user-editable string field that ends up in a notification (devName, devOwner, etc.). Something like:

   import html
   for row in jsn:
       for key, value in row.items():
           if isinstance(value, str):
               row[key] = html.escape(value)

   applied before the convert() call in construct_notifications().

2. Make the final indent() call fail soft, not hard. Right now one bad character anywhere in the whole document takes down the entire notification for every device in that cycle, not just the offending row. Wrapping it so a tokenizer failure logs the error and falls back to sending the un-indented mail_html (which doesn't go through tokenize() at all) would mean a single malformed field degrades formatting instead of silently dropping every pending alert:

   try:
       final_html = indent(mail_html, indentation="    ", newline="\r\n", indent_text=True)
   except Exception as e:
       mylog("none", [f"[Notification] indent() failed, sending unindented HTML: {e}"])
       final_html = mail_html

(1) fixes the root cause; (2) is a safety net for whatever (1) doesn't anticipate - free text from other sources (plugin output, DNS names, SSIDs) could plausibly hit the same wall.

[jokob-sk]

Thanks for the report. This should be fixed in the next release.
It would be great if you could test the ghcr.io/netalertx/netalertx-dev / ghcr.io/netalertx/netalertx-dev-unsafe Docker image in ~15 minutes (or once the latest GitHub Action finishes).

⚠️ Please back up everything first, or ideally spin up a separate container (make sure you use a unique PORT and GRAPHQL_PORT if running instances concurrently) with its own DB and config (a copy of your current setup), as database changes may occur.

After updating:

• Click the 🔄 refresh button in the blue header at the top
• Hard refresh your browser cache (shift + reload) to force js and css files reload

Disclaimer: Running the dev container long-term may lead to data loss, as upgrade paths are not guaranteed.

Thanks in advance,
j

[mormegil6]

Tested ghcr.io/netalertx/netalertx-dev-unsafe:unsafe. Repro: a device named Meta Quest Pro with devComments = "battery <=20%, check charger" (bare <, same shape as the original report) plus a pending New Device event so the notification path actually runs. Confirmed fixed: no crash, no XMLTokenError, the comment came out correctly HTML-escaped in the rendered notification, and evePendingAlertEmail flipped from 1 to 0 - the notification was actually sent and marked complete, not silently swallowed, which matters here since the old bug crashed before that flag cleared and would re-crash the same pending event every cycle forever. Thanks for the quick turnaround!

[jokob-sk]

Thanks for testing and confirmation @mormegil6 - will keep this issue open until released to production.