Claude Code 接入 MIMO 模型：通过 PreToolUse Hook 解决多媒体识别问题

# 检查 Claude Code
claude --version

# 检查 Python
python --version

# 检查 Git
git --version

# 1. 克隆项目
git clone https://github.com/congxxx/mimo-media-recognition-mcp.git
cd mimo-media-recognition-mcp

# 2. 安装依赖
pip install uv
uv venv .venv
uv pip install -e .

claude mcp add mimo-media \
  --scope user \
  --env MIMO_API_KEY="你的API密钥" \
  --env MIMO_API_BASE="https://token-plan-cn.xiaomimimo.com/v1" \
  --env MIMO_MODEL="mimo-v2.5" \
  -- "你的python解释器路径" \
  -m mimo_media_recognition_mcp.server

# 可选：如需对短音频使用专用 ASR 模型（实时转写），添加以下环境变量：
# --env MIMO_ASR_MODEL="mimo-v2.5-asr" \
# --env MIMO_ASR_LANGUAGE="zh"

claude mcp get mimo-media
# 应显示：Status: ✓ Connected

"""
PreToolUse hook: Block Read tool on multimedia files.

When the agent tries to Read an image/audio/video file, this hook
blocks the call (exit 2) and instructs the model to use the
mimo-media MCP tools instead.
"""
import sys, json

MEDIA_MAP = {
    ".png": ("image", "understand_image"),
    ".jpg": ("image", "understand_image"),
    ".jpeg": ("image", "understand_image"),
    ".gif": ("image", "understand_image"),
    ".bmp": ("image", "understand_image"),
    ".webp": ("image", "understand_image"),
    ".svg": ("image", "understand_image"),
    ".mp3": ("audio", "understand_audio"),
    ".wav": ("audio", "understand_audio"),
    ".m4a": ("audio", "understand_audio"),
    ".ogg": ("audio", "understand_audio"),
    ".flac": ("audio", "understand_audio"),
    ".aac": ("audio", "understand_audio"),
    ".mp4": ("video", "understand_video"),
    ".mov": ("video", "understand_video"),
    ".avi": ("video", "understand_video"),
    ".mkv": ("video", "understand_video"),
    ".webm": ("video", "understand_video"),
    ".flv": ("video", "understand_video"),
}

try:
    data = json.load(sys.stdin)
except Exception:
    sys.exit(0)

tool_input = data.get("tool_input", {})
file_path = tool_input.get("file_path", "") or tool_input.get("path", "")
if not file_path:
    sys.exit(0)

ext = ""
if "." in file_path:
    ext = "." + file_path.rsplit(".", 1)[-1].lower()

if ext in MEDIA_MAP:
    media_type, mcp_tool = MEDIA_MAP[ext]
    print(
        f"BLOCKED: The Read tool cannot process {media_type} files. "
        f"Use the MCP tool `mcp__mimo-media__{mcp_tool}` instead. "
        f"Example: call mcp__mimo-media__{mcp_tool} with "
        f'{{"prompt": "describe this {media_type}", "{media_type}_path": "{file_path}"}}',
        file=sys.stderr
    )
    sys.exit(2)

sys.exit(0)

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Read",
        "hooks": [
          {
            "type": "command",
            "command": "python ~/.claude/hooks/block_media_read.py",
            "timeout": 5
          }
        ]
      }
    ]
  }
}

## 多媒体文件处理（强制）

当前推理模型是纯文本模型，**无法直接理解图片、音频、视频**。

### 铁律：遇到多媒体文件必须调用 MCP

- **图片** (.png, .jpg, .jpeg, .gif, .bmp, .webp, .svg) → 使用 `mcp__mimo-media__understand_image`
- **音频** (.mp3, .wav, .m4a, .ogg, .flac, .aac) → 使用 `mcp__mimo-media__understand_audio`
- **视频** (.mp4, .mov, .avi, .mkv, .webm, .flv) → 使用 `mcp__mimo-media__understand_video`

### 禁止行为

- **禁止**用 `Read` 工具打开上述格式的文件（会被 hook 拦截）
- **禁止**尝试自行"解读"二进制多媒体内容
- **禁止**假设图片/音频/视频的内容

### 正确流程

1. 识别到多媒体文件路径
2. 根据扩展名选择对应的 MCP 工具
3. 调用 MCP 工具获取理解结果
4. 基于 MCP 返回的文本结果继续推理

### 示例

```bash
# 错误 ❌
Read("screenshot.png")

# 正确 ✅
mcp__mimo-media__understand_image(prompt="描述这张图片", image_path="screenshot.png")
```

# 验证 MCP 连接
claude mcp get mimo-media

# 测试 Hook 脚本
echo '{"tool_name": "Read", "tool_input": {"file_path": "test.png"}}' | \
  python ~/.claude/hooks/block_media_read.py 2>&1
# 预期输出：BLOCKED: The Read tool cannot process image files...
# 预期退出码：2

# 测试普通文件不受影响
echo '{"tool_name": "Read", "tool_input": {"file_path": "readme.md"}}' | \
  python ~/.claude/hooks/block_media_read.py 2>&1
# 预期输出：（空）
# 预期退出码：0

# server.py 第 165 行（修复前）
content_blocks.append({
    "type": "text",
    "text": f"[Audio file: {media_url[:80]}...]",  # ← 只取了前 80 字符！
})

// ❌ MCP Server 旧格式（text 包装，走 Anthropic endpoint）
{"type": "text", "text": "[Audio file: data:audio/mpeg;base64,<截断>]"}
// API 返回：模型无法理解音频

// ✅ MIMO 官方格式（input_audio，走 OpenAI endpoint）
{"type": "input_audio", "input_audio": {"data": "data:audio/mpeg;base64,<完整base64>"}}
// API 返回：正确的转写文字