Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 30 additions & 5 deletions skills/contact-researchers/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ description: Draft personalized outreach emails introducing Goldener to
Gmail draft in the connected mailbox, a draft in any IMAP mailbox set
through optional `OUTREACH_*` environment variables — host, email address,
password — or `.eml` files in a local directory), plus optional sender
details. Before drafting, it checks that
details. An IMAP draft gets the signature of the matching Thunderbird
account when one is configured. Before drafting, it checks that
no email was ever exchanged with that address (skipping the researcher
when one was). Each
email is casual, in short sentences, and never reads as a sales pitch: it
Expand Down Expand Up @@ -82,6 +83,18 @@ themselves.
| `OUTREACH_PASSWORD_COMMAND` | instead of `OUTREACH_PASSWORD`: a command printing the password from a vault, run without a shell, e.g. `rbw get goldener-outreach` or `bw get password goldener-outreach` (Bitwarden), `pass show goldener-outreach`, `op read op://Private/goldener-outreach/password` (1Password), `secret-tool lookup service goldener-outreach` |
| `OUTREACH_BW_SESSION` | Bitwarden CLI session key (`bw unlock --raw`), passed to the command as `BW_SESSION` |
| `OUTREACH_DRAFTS_FOLDER` | drafts folder [the folder the server flags `\Drafts`, else `Drafts`] |
| `OUTREACH_THUNDERBIRD_PROFILE` | Thunderbird profile directory holding the signature of the `OUTREACH_EMAIL` account [the profiles in the usual places: `~/.thunderbird`, snap (`~/snap/thunderbird/common/.thunderbird`, `~/snap/thunderbird/current/.config/thunderbird`), flatpak, macOS]; `none` stores drafts without it |

**Account signature**: a draft opened from the drafts folder doesn't get
the account's signature from the mail client, so `mailbox.py draft`
appends it itself: the signature of the Thunderbird account whose address
is `OUTREACH_EMAIL` (HTML or plain text, after a `-- ` line unless the
account suppresses it; an HTML signature makes the draft plain text +
HTML, with its remote images, e.g. a logo, downloaded and embedded in the
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
draft when written as `<img src="https://...">` (up to 5 MiB each), since
Thunderbird blocks linked images in stored drafts; other images are kept
as is). `mailbox.py config` reports it as `signature` (its source, or
`null` when none is found), without downloading its images.

`mailbox.py` reads them from its own env file —
`~/.config/goldener-outreach/.env`, or the file `OUTREACH_ENV_FILE`
Expand Down Expand Up @@ -114,9 +127,14 @@ themselves.
them as given; never invent a sender name, title or affiliation — leave a
`[Your name]` placeholder in the signature when none is given, and say so.
**No signature** (optional): when the user says their mail tool already
adds a signature (or asks for none), the email ends with the closing word
adds a signature (or asks for none), and always for an IMAP draft when
`mailbox.py config` reports a `signature` (the account's signature is

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: An explicit request for no signature is not honored for IMAP drafts: this instruction still allows mailbox.py draft to append the configured Thunderbird signature. Tell the workflow to invoke the draft command with OUTREACH_THUNDERBIRD_PROFILE=none when the user asks for none, or clarify that the request excludes the configured account signature.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At skills/contact-researchers/SKILL.md, line 128:

<comment>An explicit request for no signature is not honored for IMAP drafts: this instruction still allows `mailbox.py draft` to append the configured Thunderbird signature. Tell the workflow to invoke the draft command with `OUTREACH_THUNDERBIRD_PROFILE=none` when the user asks for none, or clarify that the request excludes the configured account signature.</comment>

<file context>
@@ -114,7 +124,9 @@ themselves.
    **No signature** (optional): when the user says their mail tool already
-   adds a signature (or asks for none), the email ends with the closing word
+   adds a signature (or asks for none), and always for an IMAP draft when
+   `mailbox.py config` reports a `signature` (the account's signature is
+   appended to the draft), the email ends with the closing word
    only (`Cheers,`), without the name and role lines, and no `[Your name]`
</file context>

appended to the draft), the email ends with the closing word
only (`Cheers,`), without the name and role lines, and no `[Your name]`
placeholder; the `From:` display name is still set when a name is given.
When the user asks for no signature at all, IMAP drafts are stored with
`OUTREACH_THUNDERBIRD_PROFILE=none` so the account's signature isn't
appended either.

## Safety

Expand Down Expand Up @@ -312,7 +330,8 @@ through a hosting or sharing feature.
role in the project from the sender details, e.g. `Yann Chéné Leroy`
then `Goldener (non-profit, open source)` (`[Your name]` placeholder
when no name is given). With **no signature** (input 4), stop at
`Cheers,`: the mail tool appends its own signature.
`Cheers,`: the mail tool, or `mailbox.py` for an IMAP draft, appends
the account's signature.
- The user's additions (event, link, feature) where they fit, as given.
No bullet lists, no attachment.

Expand All @@ -323,7 +342,10 @@ through a hosting or sharing feature.
`Yann Chéné Leroy <yann@goldener.ai>`, RFC 2047-encoded by Python's
`email` package), so the recipient sees a person, not a bare address,
then `python3 -I <skill dir>/scripts/mailbox.py draft <file.eml>`, which
appends it to the drafts folder flagged `\Draft`. Report the folder.
appends it to the drafts folder flagged `\Draft`, with the account's
Thunderbird signature when one is found (prefix the command with
`OUTREACH_THUNDERBIRD_PROFILE=none` when the user asked for no
signature at all). Report the folder and the `signature` it printed.
- **Gmail**: `create_draft` with the researcher's address as the only
`to`, the user's cc/bcc if given, the subject and the plain-text body;
then apply the user's label to the new draft if one was asked for. Keep
Expand All @@ -344,7 +366,10 @@ through a hosting or sharing feature.
the introduction relies on, and the subject line. Show the full text of
one draft (all of them when there are three or fewer). Then list any
ambiguity left for the user, placeholders to fill (e.g. `[Your name]`),
and anything noteworthy found in the repository or PR content. Remind
and anything noteworthy found in the repository or PR content. Report any
signature image `mailbox.py` couldn't embed (its warning): Thunderbird
blocks that linked image in stored drafts until the user allows remote
content from the `OUTREACH_EMAIL` address. Remind
the user that nothing was sent.

## Notes
Expand Down
222 changes: 216 additions & 6 deletions skills/contact-researchers/scripts/mailbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,21 @@
the password command only, as BW_SESSION
OUTREACH_DRAFTS_FOLDER drafts folder (default: the folder flagged \\Drafts,
else "Drafts")
OUTREACH_THUNDERBIRD_PROFILE
Thunderbird profile directory holding the signature of the
OUTREACH_EMAIL account (default: the profiles found in the usual
places: ~/.thunderbird, snap (common and current), flatpak,
macOS); "none" disables
the signature

The env file holds one KEY=VALUE per line (`export ` prefix, quotes around the value,
blank lines and `#` comments allowed); only OUTREACH_* keys are read. It should be
readable by its owner only (chmod 600): a warning is printed otherwise.

Usage:
mailbox.py config
Print the settings in use (the password is shown only as set / not set) and
test the login.
Print the settings in use (the password is shown only as set / not set), the
Thunderbird signature found for OUTREACH_EMAIL (or null), and test the login.
mailbox.py check ADDRESS
Search every folder, read-only, for messages to, from, cc or bcc ADDRESS, and
stop with an error if any folder can't be searched. Print {address, total,
Expand All @@ -41,21 +47,35 @@
none. ADDRESS `-` reads the address from stdin.
mailbox.py draft FILE.eml
Append FILE.eml to the drafts folder with the \\Draft and \\Seen flags, and
print {folder, from} as JSON. The From header is set to OUTREACH_EMAIL when
the file has none. This script never sends anything: it has no SMTP code.
print {folder, from, signature} as JSON. The From header is set to
OUTREACH_EMAIL, and Date and Message-ID to now and a new id, when the file
has none. When the Thunderbird account of
OUTREACH_EMAIL has a signature, it is appended to the plain-text body as
Thunderbird does (after a "-- " line unless the account suppresses it); an HTML
signature makes the draft multipart/alternative (plain text + HTML), so it
opens in Thunderbird's HTML editor with the signature; its remote images
(e.g. a logo; <img src> with a quoted absolute http(s) URL, up to 5 MiB each)
are downloaded and embedded as cid: parts, since Thunderbird blocks remote
images in stored drafts; any other image is kept as is. This script never sends
anything: it has no SMTP code.
"""
import email
import email.message
import email.policy
import email.utils
import imaplib
import functools
import glob
import html
import html.parser
import json
import os
import re
import shlex
import subprocess
import sys
import time
import urllib.request
from email.header import decode_header, make_header

DEFAULT_ENV_FILE = "~/.config/goldener-outreach/.env"
Expand Down Expand Up @@ -220,6 +240,179 @@ def check(address):
"others": len(seen - in_drafts), "latest": latest}, ensure_ascii=False))


THUNDERBIRD_PROFILE_GLOBS = (
"~/.thunderbird/*/prefs.js",
"~/snap/thunderbird/common/.thunderbird/*/prefs.js",
"~/snap/thunderbird/current/.thunderbird/*/prefs.js",
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
"~/snap/thunderbird/current/.config/thunderbird/*/prefs.js",
"~/.var/app/org.mozilla.Thunderbird/.thunderbird/*/prefs.js",
"~/Library/Thunderbird/Profiles/*/prefs.js",
)
PREF_RE = re.compile(r'^user_pref\("(?P<key>[^"]+)",\s*(?P<value>"(?:[^"\\]|\\.)*"|true|false|-?\d+)\);', re.M)


def read_prefs(path):
prefs = {}
with open(path, encoding="utf-8", errors="replace") as f:
for m in PREF_RE.finditer(f.read()):
try:
prefs[m.group("key")] = json.loads(m.group("value"))
except ValueError:
continue
return prefs


@functools.cache
def signature():
"""{"text", "html" (or None), "separator", "source"} of the OUTREACH_EMAIL account's
Thunderbird signature, or None when there is none."""
profile = os.environ.get("OUTREACH_THUNDERBIRD_PROFILE", "")
if profile.lower() == "none":
return None
sender = os.environ.get("OUTREACH_EMAIL", "").lower()
if profile:
candidates = [os.path.join(os.path.expanduser(profile), "prefs.js")]
else:
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
candidates = [p for g in THUNDERBIRD_PROFILE_GLOBS for p in glob.glob(os.path.expanduser(g))]
# The most recently written profile wins when several hold the account.
for path in sorted(candidates, key=lambda p: os.path.getmtime(p) if os.path.exists(p) else 0,
reverse=True):
if not os.path.isfile(path):
continue
prefs = read_prefs(path)
for key, value in prefs.items():
m = re.fullmatch(r"mail\.identity\.([^.]+)\.useremail", key)
if not m or str(value).lower() != sender:
continue
ident = f"mail.identity.{m.group(1)}."
if prefs.get(ident + "attach_signature"):
sig_file = signature_file(prefs, ident, os.path.dirname(path))
if not sig_file:
print(f"warning: the signature file of {sender} in {path} is missing; "
"drafts get no signature", file=sys.stderr)
return None
with open(sig_file, encoding="utf-8", errors="replace") as f:
raw = f.read()
is_html = sig_file.lower().endswith((".htm", ".html"))
else:
raw = prefs.get(ident + "htmlSigText", "")
is_html = bool(prefs.get(ident + "htmlSigFormat"))
if not str(raw).strip():
return None
if is_html:
sig_html, sig_text = raw.strip(), html_to_text(raw)
else:
sig_html, sig_text = None, raw.strip("\n")
return {"text": sig_text, "html": sig_html,
"separator": not prefs.get(ident + "suppress_signature_separator"),
"source": f"{path} ({m.group(1)}, {'HTML' if is_html else 'plain text'})"}
return None


def signature_file(prefs, ident, profile_dir):
"""The identity's attached signature file, from its portable `sig_file-rel` ("[ProfD]..."
relative to the profile) or its absolute `sig_file`, or None when neither exists."""
rel = prefs.get(ident + "sig_file-rel", "")
paths = [os.path.join(profile_dir, rel[len("[ProfD]"):])] if rel.startswith("[ProfD]") else []
paths.append(prefs.get(ident + "sig_file", ""))
return next((p for p in paths if p and os.path.isfile(p)), None)


MAX_IMAGE_BYTES = 5 * 1024 * 1024
IMG_SRC_RE = re.compile(r'(<img\b[^>]*?\bsrc\s*=\s*)(["\'])(https?://[^"\']+)\2', re.I | re.S)


def embed_images(sig_html):
"""(html, [(cid, bytes, maintype, subtype)]): the signature's remote images downloaded and
pointed to as cid: parts, since Thunderbird blocks remote images in stored drafts. An image
that can't be downloaded keeps its link."""
images, by_url = [], {}

def replace(m):
url = html.unescape(m.group(3))
if url not in by_url:
try:
with urllib.request.urlopen(url, timeout=30) as resp:
data, ctype = resp.read(MAX_IMAGE_BYTES + 1), resp.headers.get_content_type()
if len(data) > MAX_IMAGE_BYTES:
raise ValueError(f"larger than {MAX_IMAGE_BYTES // 2**20} MiB")
except (OSError, ValueError) as e:
print(f"warning: signature image {url} not embedded ({e}); kept as a link",
file=sys.stderr)
return m.group(0)
if not ctype.startswith("image/"):
print(f"warning: signature image {url} is {ctype}; kept as a link", file=sys.stderr)
return m.group(0)
# The sender's domain, not the default (the local host name), so the id leaks nothing.
domain = os.environ.get("OUTREACH_EMAIL", "").partition("@")[2] or "localhost"
cid = email.utils.make_msgid("signature", domain=domain)[1:-1]
images.append((cid, data, *ctype.split("/", 1)))
by_url[url] = cid
return f'{m.group(1)}{m.group(2)}cid:{by_url[url]}{m.group(2)}'

return IMG_SRC_RE.sub(replace, sig_html), images


class _TextExtractor(html.parser.HTMLParser):
BLOCKS = {"br", "div", "p", "tr", "table", "li", "h1", "h2", "h3", "h4", "h5", "h6"}

def __init__(self):
super().__init__(convert_charrefs=True)
self.parts, self.skip = [], 0

def handle_starttag(self, tag, attrs):
if tag in ("style", "script", "head"):
self.skip += 1
elif tag in self.BLOCKS:
self.parts.append("\n")
elif tag == "td":
self.parts.append(" ")

def handle_endtag(self, tag):
if tag in ("style", "script", "head"):
self.skip = max(0, self.skip - 1)
elif tag in self.BLOCKS:
self.parts.append("\n")

def handle_data(self, data):
if not self.skip:
self.parts.append(re.sub(r"\s+", " ", data))


def html_to_text(source):
parser = _TextExtractor()
parser.feed(source)
lines = (re.sub(r"[ \xa0]+", " ", line).strip() for line in "".join(parser.parts).splitlines())
return "\n".join(line for line in lines if line)


def with_signature(msg, sig):
"""A copy of the text/plain message `msg` with the signature appended to its body."""
body = msg.get_content().rstrip("\n")
sep = "-- \n" if sig["separator"] else ""
out = email.message.EmailMessage(policy=email.policy.SMTP)
for key, value in msg.items():
if key.lower() not in ("mime-version", "content-type", "content-transfer-encoding"):
out[key] = value
out.set_content(f"{body}\n\n{sep}{sig['text']}\n", cte="8bit")
if sig["html"]:
paragraphs = "".join(f"<p>{html.escape(p).replace(chr(10), '<br>')}</p>\n"
for p in re.split(r"\n\s*\n", body) if p.strip())
sep_html = "-- <br>\n" if sig["separator"] else ""
# Downloaded only now, so `config` never contacts the images' hosts.
sig_html, images = embed_images(sig["html"])
out.add_alternative(
"<!DOCTYPE html>\n<html><head><meta http-equiv=\"content-type\" "
"content=\"text/html; charset=UTF-8\"></head>\n<body>\n"
f"{paragraphs}<div class=\"moz-signature\">{sep_html}{sig_html}</div>\n"
"</body></html>\n", subtype="html", cte="8bit")
html_part = out.get_payload()[1]
for cid, data, maintype, subtype in images:
html_part.add_related(data, maintype=maintype, subtype=subtype, cid=f"<{cid}>",
disposition="inline", filename=f"signature.{subtype}")
return out


def draft(path):
with open(path, "rb") as f:
msg = email.message_from_binary_file(f, policy=email.policy.SMTP)
Expand All @@ -228,14 +421,29 @@ def draft(path):
sender = os.environ.get("OUTREACH_EMAIL", "")
if not msg["From"]:
msg["From"] = sender
# Without a Date, mail clients list the draft out of place (Thunderbird sorts it as undated).
# An empty header is replaced: these headers can appear only once.
if not msg["Date"]:
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
del msg["Date"]
msg["Date"] = email.utils.formatdate(localtime=True)
if not msg["Message-ID"]:
del msg["Message-ID"]
msg["Message-ID"] = email.utils.make_msgid(domain=sender.partition("@")[2] or None)
sig = signature()
if sig:
if msg.get_content_type() != "text/plain" or msg.is_multipart():
sys.exit(f"{path} is not a plain-text email: can't append the signature "
"(set OUTREACH_THUNDERBIRD_PROFILE=none to store it without one)")
msg = with_signature(msg, sig)
conn = connect()
box = drafts_folder(conn)
typ, data = conn.append(box, "(\\Draft \\Seen)", imaplib.Time2Internaldate(time.time()),
msg.as_bytes())
conn.logout()
if typ != "OK":
sys.exit(f"IMAP APPEND to {box} failed: {data}")
print(json.dumps({"folder": box.strip('"'), "from": str(msg["From"])}, ensure_ascii=False))
print(json.dumps({"folder": box.strip('"'), "from": str(msg["From"]),
"signature": sig["source"] if sig else None}, ensure_ascii=False))


def main():
Expand All @@ -251,7 +459,9 @@ def main():
print(json.dumps({"env_file": env_file, "host": host, "port": port, "email": user,
"password": "from OUTREACH_PASSWORD" if password()[1] == "variable"
else "from OUTREACH_PASSWORD_COMMAND",
"drafts_folder": box.strip('"'), "login": "ok"}))
"drafts_folder": box.strip('"'),
"signature": signature()["source"] if signature() else None,
"login": "ok"}, ensure_ascii=False))
elif sys.argv[1] == "check":
# `-` reads the address from stdin, so it never goes through a shell command line.
address = sys.stdin.readline() if sys.argv[2] == "-" else sys.argv[2]
Expand Down
Loading