Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ aegisgate/tests/*
!aegisgate/tests/test_hot_reload_unit.py
!aegisgate/tests/test_injection_detector.py
!aegisgate/tests/test_sqlite_mapping_prune.py
!aegisgate/tests/test_doc_alignment.py

# Virtual env
.venv/
Expand Down
2 changes: 1 addition & 1 deletion OTHER_TERMINAL_CLIENTS_USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Notes:

## Claude API Support

Supported via generic proxy:
Supported via the v1 OpenAI-compatible adapter (`/v1/messages` is handled natively with the full safety pipeline; other subpaths such as `count_tokens` go through the v1 generic pass-through):
- `POST /v1/messages`
- `POST /v1/messages/count_tokens`
- `stream=true` streaming passthrough
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,9 +289,11 @@ When a token is configured with `"compat": "openai_chat"` in `config/gw_tokens.j

### Security Pipeline

**Request side:** PII redaction → exact-value redaction → untrusted content guard → request sanitizer → RAG poison guard
**Request side** (default policy): exact-value redaction → PII redaction → request sanitizer → RAG poison guard

**Response side:** anomaly detector → injection detector → RAG poison guard → privilege guard → tool call guard → restoration → post-restore guard → output sanitizer
**Response side** (default policy): exact-value redaction → anomaly detector → injection detector → RAG poison guard → privilege guard → tool call guard → restoration → post-restore guard → output sanitizer

> `untrusted content guard` and `system prompt guard` are constructed into the pipeline but not part of the default policy's `enabled_filters`; enable them via policy YAML plus the matching feature flag.

### Error Response Format

Expand Down
23 changes: 14 additions & 9 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ curl -X POST http://127.0.0.1:18080/__gw__/register \
- `stream=true` 流式透传
- 支持 query 透传(例如 `?anthropic-version=2023-06-01`)
- 默认仍沿用 v1 请求/响应安全管道;若使用 `__passthrough` 或命中上游白名单绕过,才会跳过过滤
- 请求侧:`redaction`、`exact_value_redaction`、`untrusted_content_guard`、`request_sanitizer`、`rag_poison_guard`
- 响应侧`anomaly_detector`、`injection_detector`、`rag_poison_guard`、`privilege_guard`、`tool_call_guard`、`restoration`、`post_restore_guard`、`output_sanitizer`
- 请求侧(默认策略):`exact_value_redaction`、`redaction`、`request_sanitizer`、`rag_poison_guard`
- 响应侧(默认策略):`exact_value_redaction`、`anomaly_detector`、`injection_detector`、`rag_poison_guard`、`privilege_guard`、`tool_call_guard`、`restoration`、`post_restore_guard`、`output_sanitizer`
- 扩展脱敏:覆盖 `P0/P1` 常见敏感字段 + `Crypto` 专项字段(地址/私钥/助记词/交易所密钥)
- `responses` 结构化 `input` 预转发脱敏:覆盖 `user/developer/system/assistant` 与 `function_call_output/tool_output` 等节点
- 高风险自动处理:命中高风险时自动遮挡/分割危险片段后返回,无需人工确认
Expand Down Expand Up @@ -236,18 +236,18 @@ curl -X POST http://127.0.0.1:18080/__gw__/register \

`v1` 链路(OpenAI 兼容):

1. 请求侧过滤:`redaction -> exact_value_redaction -> untrusted_content_guard -> request_sanitizer -> rag_poison_guard`
1. 请求侧过滤:`exact_value_redaction -> redaction -> system_prompt_guard -> untrusted_content_guard -> request_sanitizer -> rag_poison_guard`
2. 转发到上游 LLM(chat/responses/generic 子路径)
3. 响应侧过滤:`anomaly_detector -> injection_detector -> rag_poison_guard -> privilege_guard -> tool_call_guard -> restoration -> post_restore_guard -> output_sanitizer`
3. 响应侧过滤:`exact_value_redaction -> anomaly_detector -> injection_detector -> rag_poison_guard -> privilege_guard -> tool_call_guard -> restoration -> post_restore_guard -> output_sanitizer`
4. 按风险处置:`allow / sanitize / block`(危险片段自动遮挡/分割,不走确认流程)
5. 记录审计事件(含风险标签、处置原因、确认状态)

说明:

- 上述顺序表示默认流水线构造顺序;实际是否执行仍取决于策略 `enabled_filters` 与全局开关。
- 当前默认策略包含 `tool_call_guard`,但 **不包含** `untrusted_content_guard`:
- 当前默认策略包含 `tool_call_guard`,但 **不包含** `system_prompt_guard` 与 `untrusted_content_guard`:
- 若需要对 `retrieval/web/tool/document` 等不可信来源做包裹与风险抬升,需在策略 YAML 中显式加入 `untrusted_content_guard`,并保持对应 feature flag 开启。
- `tool_call_guard` 默认对未命中白名单的工具名仅做审查(`review`),但对危险参数默认直接按 `block` 处理;编码工具(`apply_patch`/`write`/`edit`/`bash` 等 25+)自动跳过 `dangerous_param` 扫描,避免代码 diff 内容误触发。工具名白名单默认留空,避免误伤不同上游的自定义工具。若显式配置白名单,未命中的工具名默认按 `review` 处理。
- `tool_call_guard` 默认对未命中白名单的工具名仅做审查(`review`),但对危险参数默认直接按 `block` 处理;文件写入类工具(`apply_patch`/`write`/`edit` 等约 12 个)仅跳过“路径引用类”规则(`sensitive_file_access`/`path_traversal`/`ssh_key_access`),仍执行执行类危险参数扫描,避免代码 diff 内容误触发路径类规则;`bash`/`shell` 等执行类工具名列入危险工具名单(命中即标记,默认动作 `review`,可配为 `block`),且仍走完整危险参数扫描。工具名白名单默认留空,避免误伤不同上游的自定义工具。若显式配置白名单,未命中的工具名默认按 `review` 处理。

`v2` 链路(通用 HTTP 代理):

Expand Down Expand Up @@ -340,7 +340,7 @@ curl -X POST http://127.0.0.1:18080/__gw__/register \

**分级变形策略**:

- **极度危险指令**(`rm -rf`、SQL 注入、反弹 shell、fork bomb、`curl|bash`、`dd if=of=`、`mkfs`、`powershell -enc` 等约 45 条模式):片段被完全替换为 `【AegisGate已处理危险疑似片段】`,**原文不会出现在返回中**。
- **极度危险指令**(`rm -rf`、SQL 注入、反弹 shell、fork bomb、`curl|bash`、`dd if=of=`、`mkfs`、`powershell -enc` 等约 10 条命令模式):片段被完全替换为 `【AegisGate已处理危险疑似片段】`,**原文不会出现在返回中**。
- **一般危险片段**(系统提示词泄露、可疑权限操作等):使用 chunked-hyphen 分词变形(如 `dev-elo-per mes-sag-e`)。

建议:
Expand Down Expand Up @@ -478,8 +478,12 @@ curl -X POST http://127.0.0.1:18080/__gw__/remove \
}
```

3. 客户端配置(以 Claude Code 为例):
3. 放开 compat 端口路由并配置客户端(`AEGIS_COMPAT_ALLOWED_PORTS` 默认空=拒绝所有端口路由,必须显式放开,否则返回 `403 port_not_allowed`):
```bash
# 放开 compat 端口路由(默认空=全部拒绝)
export AEGIS_COMPAT_ALLOWED_PORTS=8317

# 客户端(以 Claude Code 为例)
export ANTHROPIC_BASE_URL=http://<网关IP>:18080/v1/__gw__/t/claude-to-gpt/8317
```

Expand Down Expand Up @@ -823,9 +827,10 @@ AEGIS_DOCKER_UPSTREAMS=8317:cli-proxy-api,8080:sub2api,3000:aiclient2api
- `system_exfil`(系统提示泄露)
- `obfuscated`(编码混淆攻击,含消息级多脚本噪声注入)
- `unicode_bidi`(bidi 方向控制攻击)
- `tool_call_injection`(伪造工具调用,覆盖 OpenAI/Anthropic/Gemini/Bedrock/ReAct/MCP 等 45+ 模式)
- `spam_noise`(赌博/色情/平台垃圾内容噪声,>=2 类别组合时触发)

> 注:`tool_call_injection`(伪造工具调用,覆盖 OpenAI/Anthropic/Gemini/Bedrock/ReAct/MCP 等约 26 种模式)默认动作为 `review`(抬高风险分并标记复核,按阈值处置),**并非**无条件强制拦截;如需强拦,可在 `security_filters.yaml` 的 `action_map.injection_detector.tool_call_injection` 改为 `block`。

### 5.3 语义复核模块

当前主链路的语义复核受 `AEGIS_ENABLE_SEMANTIC_MODULE` 控制,并通过 `AEGIS_SEMANTIC_SERVICE_URL` 指向的服务执行:
Expand Down
5 changes: 3 additions & 2 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
- **两种路由模式**(可同时启用):
- **Token 路由**(推荐多租户场景):每个 token 绑定独立的上游地址;注册/管理 token 时需提供网关密钥。
- v1(LLM):`http://<host>:18080/v1/__gw__/t/<TOKEN>/...`
- v2(通用 HTTP 代理):`http://<host>:18080/v2/__gw__/t/<TOKEN>`,须携带 `x-target-url: <完整目标URL>` 请求头
- v2(通用 HTTP 代理):`http://<host>:18080/v2/__gw__/t/<TOKEN>`,须携带 `x-target-url: <完整目标URL>` 请求头;且目标主机须在 `AEGIS_V2_TARGET_ALLOWLIST` 白名单内(默认空=拒绝全部目标,fail-closed)
- **直连上游**(单用户/Agent 快速接入):设置 `AEGIS_UPSTREAM_BASE_URL=<上游地址>` 后,可直接请求 `/v1/...`,无需注册 token。`v2` 仍必须使用 token 路径。
- **脱敏豁免字段(whitelist_key)**:注册 token 时可指定逗号分隔的字段名列表(如 `api_key,secret,token`)。请求体中这些字段的值**不做 PII 脱敏**,直接透传到上游。适用于需要原始凭证通过的场景。支持以下匹配模式:
- JSON:`"field":"value"` 或 `"field": "value"`
Expand Down Expand Up @@ -123,7 +123,8 @@ curl -X POST http://127.0.0.1:18080/__gw__/register \
```json
{
"token": "Ab3k9Qx7Yp",
"baseUrl": "http://127.0.0.1:18080/v1/__gw__/t/Ab3k9Qx7Yp"
"baseUrl": "http://127.0.0.1:18080/v1/__gw__/t/Ab3k9Qx7Yp",
"whitelist_key": []
}
```

Expand Down
64 changes: 64 additions & 0 deletions aegisgate/tests/test_doc_alignment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
"""Doc-code alignment regression guards.

These tests pin a handful of facts that the user-facing docs rely on, so that
documentation drift (filter count, compat model whitelist, default action_map
disposition) is caught by CI instead of by a manual audit.

If one of these fails, either the code changed on purpose — in which case update
the cited docs and the expected value here — or a doc/code mismatch slipped in.
"""

from __future__ import annotations

import importlib
import pkgutil
from pathlib import Path

import yaml

import aegisgate.filters
from aegisgate.adapters.openai_compat.mapper import COMPAT_ALLOWED_MODELS
from aegisgate.filters.base import BaseFilter

_REPO_ROOT = Path(__file__).resolve().parents[2]
_RULES = _REPO_ROOT / "aegisgate" / "policies" / "rules" / "security_filters.yaml"

# Number of concrete BaseFilter subclasses. README pipeline section and
# docs/prompts/audit_general.md both cite this count — update all three together.
_EXPECTED_FILTER_CLASS_COUNT = 13


def _import_all_filter_modules() -> None:
for mod in pkgutil.iter_modules(aegisgate.filters.__path__):
importlib.import_module(f"aegisgate.filters.{mod.name}")


def test_filter_class_count_matches_docs() -> None:
_import_all_filter_modules()
concrete = {cls.__name__ for cls in BaseFilter.__subclasses__()}
assert len(concrete) == _EXPECTED_FILTER_CLASS_COUNT, (
f"BaseFilter concrete subclasses are now {len(concrete)} "
f"({sorted(concrete)}); update _EXPECTED_FILTER_CLASS_COUNT and the "
f"filter-count claims in README.md / README_zh.md / "
f"docs/prompts/audit_general.md."
)


def test_compat_allowed_models_documented() -> None:
readme = (_REPO_ROOT / "README.md").read_text(encoding="utf-8")
readme_zh = (_REPO_ROOT / "README_zh.md").read_text(encoding="utf-8")
for model in COMPAT_ALLOWED_MODELS:
assert model in readme, f"{model} missing from README.md allowed-models list"
assert model in readme_zh, (
f"{model} missing from README_zh.md allowed-models list"
)


def test_tool_call_injection_default_action_is_review() -> None:
rules = yaml.safe_load(_RULES.read_text(encoding="utf-8"))
injection = rules.get("action_map", {}).get("injection_detector", {})
assert injection.get("tool_call_injection") == "review", (
"action_map.injection_detector.tool_call_injection changed; docs that "
"describe its disposition (README_zh §5.2) must be updated to match — "
"it is NOT an unconditional force-block by default."
)
4 changes: 2 additions & 2 deletions config/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ Docker 运行时挂载本目录。当前版本已支持对部分文件做轮询
1. **策略 YAML (`enabled_filters`)**:声明哪些过滤器是激活候选。策略文件位于 `aegisgate/policies/rules/`(如 `default.yaml`、`strict.yaml`)。

2. **Feature Flag (`enable_*` in settings.py / .env)**:与策略 YAML 构成 AND 条件——过滤器必须同时列在 YAML 中且对应 feature flag 开启才会激活。
- 例:`system_prompt_guard` 曾列在 default.yaml 中,但 `AEGIS_ENABLE_SYSTEM_PROMPT_GUARD` 默认为 `false`,因此该过滤器不会激活
- 要启用 `system_prompt_guard`,必须 **同时** 在策略 YAML 中保留该条目 **并且** 在 `.env` 中设置 `AEGIS_ENABLE_SYSTEM_PROMPT_GUARD=true`。
- 例:`system_prompt_guard` 当前既未列在 `default.yaml`,对应的 `AEGIS_ENABLE_SYSTEM_PROMPT_GUARD` 也默认为 `false`——两个条件都不满足,因此该过滤器默认不会激活
- 要启用 `system_prompt_guard`,必须 **同时** 在策略 YAML 中加入该条目 **并且** 在 `.env` 中设置 `AEGIS_ENABLE_SYSTEM_PROMPT_GUARD=true`。

3. **`AEGIS_SECURITY_LEVEL`**:不改变运行哪些过滤器,而是通过策略引擎调整 `risk_threshold`:
- `low`:更高阈值(更宽松,更少拦截)
Expand Down
Loading