{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mage-VL Proactive Streaming Commentary — Google Colab\n",
    "\n",
    "Run the [Mage-VL](https://huggingface.co/microsoft/Mage-VL) proactive streaming demo on Colab.\n",
    "\n",
    "**Paper:** [arXiv:2607.24904](https://arxiv.org/pdf/2607.24904)\n",
    "\n",
    "### Architecture\n",
    "```\n",
    "Video stream -> Mage-ViT (Codec-Native patch encoding)\n",
    "                  | \n",
    "                  +-> StreamMind Gate (System 1, Mamba-SSM) -> p(speak)\n",
    "                  |                                           |\n",
    "                  +-> Qwen3-4B Decoder (System 2) <- if p >= threshold\n",
    "```\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. 确认 GPU\n",
    "!nvidia-smi --query-gpu=name,memory.total --format=csv,noheader\n",
    "import torch\n",
    "print(f\"CUDA available: {torch.cuda.is_available()}, device: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'none'}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. 安装依赖（Colab 已有 torch，只需装额外包）\n",
    "!pip install -q transformers accelerate decord opencv-python-headless moviepy Pillow scipy safetensors huggingface_hub\n",
    "# StreamMind gate 需要 mamba-ssm（GPU 编译，约 2-3 分钟）\n",
    "!pip install -q mamba-ssm\n",
    "print(\"Installation done\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3. 上传视频或下载示例视频\n",
    "from google.colab import files\n",
    "uploaded = files.upload()  # 上传一个 .mp4 文件\n",
    "VIDEO_PATH = list(uploaded.keys())[0]\n",
    "print(f\"Uploaded: {VIDEO_PATH}\")\n",
    "# 或者使用示例视频 URL:\n",
    "# !wget -q \"https://example.com/sample.mp4\" -O sample.mp4\n",
    "# VIDEO_PATH = \"sample.mp4\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 4. 运行 Mage-VL 流式解说\n",
    "!python /content/streaming_commentary.py \\\n",
    "    --video {VIDEO_PATH} \\\n",
    "    --checkpoint microsoft/Mage-VL \\\n",
    "    --backend codec \\\n",
    "    --segment-sec 8 \\\n",
    "    --threshold 0.5 \\\n",
    "    --max-tokens 120 \\\n",
    "    --gpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 参数说明\n",
    "| 参数 | 默认 | 说明 |\n",
    "|---|---|---|\n",
    "| `--video` | (必填) | 视频路径 |\n",
    "| `--checkpoint` | `microsoft/Mage-VL` | 模型 ID |\n",
    "| `--backend` | `codec` | `codec`=Codec-Native；`frames`=均匀帧采样 |\n",
    "| `--segment-sec` | `8.0` | 每段长度（秒）|\n",
    "| `--threshold` | `0.5` | Gate 触发阈值 |\n",
    "| `--max-tokens` | `120` | 每次生成最大 token 数 |\n",
    "| `--gpu` | 关 | 启用 CUDA（Colab 必需）|"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}