Skip to content

perf(imap): fetch only required header fields during sync#12893

Draft
ChristophWurst wants to merge 1 commit intomainfrom
perf/imap-selective-header-fetch
Draft

perf(imap): fetch only required header fields during sync#12893
ChristophWurst wants to merge 1 commit intomainfrom
perf/imap-selective-header-fetch

Conversation

@ChristophWurst
Copy link
Copy Markdown
Member

@ChristophWurst ChristophWurst commented May 6, 2026

Replace full BODY.PEEK[HEADER] with BODY.PEEK[HEADER.FIELDS (...)] in findByIds(), downloading only the 5 headers actually needed by parseHeaders(). Reduces header data per message 10-30x, cutting initial sync time from minutes to seconds on slow IMAP connections and preventing web request timeouts.

How to test

  1. Add an account
  2. Close the web UI
  3. Wait a bit for any processes to finish
  4. Delete all mailboxes of the account in the DB
  5. Reset the account's last_mailbox_sync
  6. Run occ mail:account:sync -vvv

main: slower
here: faster

When I patch IMAPClientFactory to have $params['debug_literal'] = true; and enable debug for the account, I see the log sizes:

main: 20M
here: 8.2M

  ┌──────────────────────────┬────────┬───────┬─────────┐                                                                                                                                                                                      
  │         Mailbox          │ Before │ After │ Speedup │
  ├──────────────────────────┼────────┼───────┼─────────┤                                                                                                                                                                                      
  │ INBOX pass 1 (2292 msgs) │ 107s   │ 10s   │ 10.7×   │               
  ├──────────────────────────┼────────┼───────┼─────────┤
  │ INBOX pass 2 (3680 msgs) │ 242s   │ 33s   │ 7.3×    │                                                                                                                                                                                      
  ├──────────────────────────┼────────┼───────┼─────────┤
  │ Sent (2945 msgs)         │ 131s   │ 12s   │ 10.9×   │                                                                                                                                                                                      
  ├──────────────────────────┼────────┼───────┼─────────┤                                                                                                                                                                                      
  │ Total                    │ 480s   │ 55s   │ 8.7×    │
  └──────────────────────────┴────────┴───────┴─────────┘

AI-assisted: Claude Code (claude-sonnet-4-6)

Replace full BODY.PEEK[HEADER] with BODY.PEEK[HEADER.FIELDS (...)] in
findByIds(), downloading only the 5 headers actually needed by
parseHeaders(). Reduces header data per message 10-30x, cutting initial
sync time from minutes to seconds on slow IMAP connections and preventing
web request timeouts.

AI-assisted: Claude Code (claude-sonnet-4-6)
Signed-off-by: Christoph Wurst <1374172+ChristophWurst@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant