There are many places a frontier model can run. This one you can inspect, fork, and repair. 前沿模型可以跑在很多地方。
這一個，你可以檢視、分叉、修復。

v 0.6: the model learns to see. The 0.5 series stays on the record.v 0.6：模型學會看圖；0.5 系列留在紀錄裡

Version 0.6.0 moves the managed download from the Headroom128 0731 Flash weights to the Vision-Exp abliterated IQ2 pair: an 86 720 111 776-byte language file plus a 932 857 760-byte vision encoder, 87 652 969 536 bytes in all (about 81.63 GiB, or 87.653 GB as your disk counts it). The engine pin advances to d0c2b43; ds4-server and ds4-agent now start with --vision and advertise input: text+image. The reason is simple: a model that can read the screenshot, the chart, or the scanned page in front of you is repairable in more situations than one that cannot.

0.6.0 把代管下載從 Headroom128 0731 Flash 權重換成 Vision-Exp abliterated IQ2 這一對：語言檔 86 720 111 776 位元組，加上視覺編碼器 932 857 760 位元組，合計 87 652 969 536 位元組（約 81.63 GiB，用磁碟的算法是 87.653 GB）。引擎釘到 d0c2b43；ds4-server 與 ds4-agent 現在帶著 --vision 啟動，並宣告 input: text+image。理由很單純：一顆看得懂你眼前那張截圖、圖表或掃描頁的模型，能修的情況比看不懂的多得多。

Two honesty notes before the details. The 0731 DSpark draft head does not fit these weights and is no longer attached. And the headline numbers on this page — 286 t/s prefill, 45 t/s generation — come from /read earth.jpg on an M5 Max, a 207-token image turn at --ctx 2048 that measured 285.75 and 44.94. A plain text turn answered normally on the same machine. Treat those as what they are: an image-turn reading, not a 2k-context prefill benchmark. The 0.5 entries further down are August history, kept because someone will need to know what the previous weights were.

進入細節前，先說兩件老實話。0731 的 DSpark 草稿頭不適用於這組權重，已不再掛載。而本頁最醒目的兩個數字──prefill 286 t/s、生成 45 t/s──來自 M5 Max 上的 /read earth.jpg：一個 207 token 的圖片回合，--ctx 2048，量到 285.75 與 44.94。同一台機器的純文字回合回答正常。請把它們當成它們本來的樣子：圖片回合的讀數，不是 2k context 的 prefill 基準。再往下的 0.5 條目是八月的歷史，留著是因為總有人需要知道上一組權重是什麼。

0.6.0What changed in the extension擴充套件改了什麼

• Engine pin d0c2b43. The Vision-Exp engine from audreyt/ds4 origin/main. The managed ds4-server and ds4-agent both start with --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf.
• Preferred weights. Language: audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF at 367a1fef — the official IQ2 recipe with 33 already-quantised attn_output_b tensors grafted in (86 720 111 776 B). Encoder: antirez/deepseek-v4-gguf at f71f23d, unmodified (932 857 760 B). Not Headroom128 0731 Flash; do not attach 0731 DSpark.
• Downloads pinned to commits. download_model.sh resolves /resolve/$REV/, never /resolve/main/, so what you fetch next year is what was tested this year. It repoints ds4flash.gguf and deletes the four old Headroom128/DSpark filenames only after both size checks pass; if the encoder fetch fails, your working 0731 link is left alone.
• Disk during an upgrade. A fresh install occupies 87.653 GB plus the KV cache. Upgrading from 0.5.x keeps the old Headroom128 file (about 87 GB) beside the new pair until the switch, so expect a peak near 175 GB — or 181 GB if DSpark was present.
• Read the numbers as labelled. 286/45 t/s is the grafted file on an M5 Max at /read earth.jpg, 207 input tokens, --ctx 2048: 285.75 prefill, 44.94 generation.

• 引擎釘在 d0c2b43。audreyt/ds4 origin/main 的 Vision-Exp 引擎。代管的 ds4-server 與 ds4-agent 都帶 --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf 啟動。
• 首選權重。語言檔：audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF 的 367a1fef──官方 IQ2 配方，嫁接 33 個已量化的 attn_output_b 張量（86 720 111 776 B）。編碼器：antirez/deepseek-v4-gguf 的 f71f23d，未經修改（932 857 760 B）。不是 Headroom128 0731 Flash；不可掛 0731 DSpark。
• 下載釘在 commit 上。download_model.sh 走 /resolve/$REV/，不走 /resolve/main/，明年抓到的就是今年測過的。它會在兩個檔案都通過大小檢查之後，才把 ds4flash.gguf 指向新檔、刪掉四個舊的 Headroom128／DSpark 檔名；編碼器抓不下來時，原本能用的 0731 連結不會被動到。
• 升級期間的磁碟。全新安裝佔 87.653 GB 加上 KV cache。從 0.5.x 升級時，舊的 Headroom128（約 87 GB）會和新的一對並存到切換為止，峰值約 175 GB；若原本有 DSpark，約 181 GB。
• 數字照標籤讀。286／45 t/s 是嫁接檔在 M5 Max 上跑 /read earth.jpg（207 個輸入 token，--ctx 2048）的結果：prefill 285.75、生成 44.94。

0.6.1Patch: `preferred` alias修補：`preferred` 別名

• No weight or engine change. `download_model.sh` and `DS4_MODEL_QUANT` now accept `preferred` (also `vision-abliterated` / `ds4f-vision-abliterated`) as an alias to the historic `q2` selector — both fetch the same Vision-Exp abliterated IQ2 + encoder pair. `audreyt/ds4` `preferred` now also fetches that pair, so `pi` and bare `ds4` agree. `d0c2b43` engine and `367a1fef`/`f71f23d` GGUFs unchanged; `7855b7a` is the `preferred`-alias commit on that engine.

• 沒有權重或引擎變動。`download_model.sh` 與 `DS4_MODEL_QUANT` 現在接受 `preferred`（亦接受 `vision-abliterated`／`ds4f-vision-abliterated`）作為歷史選項 `q2` 的別名──兩者抓的是同一對 Vision-Exp abliterated IQ2 + encoder。`audreyt/ds4` 的 `preferred` 現在也會抓這一對，所以 `pi` 和單獨跑 `ds4` 會一致；`d0c2b43` 引擎與 `367a1fef`／`f71f23d` 檔案不變，`7855b7a` 是在該引擎上加 `preferred` 別名的 commit。

§Already working in your next launch下次啟動就已經生效

None of these need a new flag. They are the parts of the engine that make a long agent session feel less like wading, and a stuck server less like a black box.

這些都不需要新的參數。它們是讓長時間 agent 對話不再像涉水前進、讓卡住的伺服器不再是黑盒子的那些部分。

• A server you can ask how it is. GET /health and GET /stats are answered on the client thread, so they respond even mid-generation. /stats reports uptime, queue depth, a busy flag, cache hits by source, token totals, cancellation counts, and the last prefill and decode speeds. curl examples in §7.1. (antirez/ds4 PR #489, merged here ahead of upstream.)
• Long sessions redo far less prefill (also PR #489). KV snapshots are stored at step thresholds rather than exact multiples; a consumed snapshot is deleted only after the tail prefill succeeds; fresh snapshots get a grace period before eviction; chat and Anthropic tool-call turns remember a visible checkpoint. What you feel: a long Claude Code or Codex tool loop reconnects to the cache far more often.
• Disconnect means stop. A client that goes away mid-turn — timeout, Ctrl-C — cancels both prefill and decode. The GPU is freed at once instead of quietly finishing a reply nobody is waiting for.
• Tool-call replay with higher fidelity. The whitespace separator the model actually sampled is captured; multi-invoke DSML blocks are no longer replayed into partial messages; the tool-map disk scan is skipped when every replay ID is already in memory. On top of seed 42, that keeps decision traces stable byte for byte.
• Resilience. Hardened JSON request parsing (upstream); tool calls that begin inside an unclosed <think> are recovered (upstream); the directional-steering decode path survived the upstream merge intact (this fork).
• A smoother /ds4-agent (upstream): cooperative interruption, prefill speed and greedy-sampling shown in the status line, status messages for the web tools, hardened DSML parsing, an edit-tool fix with regression tests, and the terminal restored correctly on exit.

• 可以問它「你還好嗎」的伺服器。GET /health 與 GET /stats 由客戶端執行緒直接回應，模型生成到一半也答得動。/stats 回報執行時間、佇列深度、忙碌旗標、各來源的快取命中、token 總量、取消次數，以及最近一次 prefill 與 decode 的速度。curl 範例見 §7.1。（antirez/ds4 PR #489，本分叉先於上游合併。）
• 長對話少做很多重複的 prefill（同樣來自 PR #489）。KV 快照改在步數門檻儲存，而不是剛好整數倍；用掉的快照要等接續的 prefill 成功才刪；新快照有一段寬限期不被淘汰；chat 與 Anthropic 工具回合會記住一個看得見的檢查點。體感上：Claude Code 或 Codex 的長工具迴圈回到伺服器時，更常直接接上快取。
• 斷線就停。客戶端中途離開──逾時、Ctrl-C──會同時取消 prefill 與 decode，GPU 立刻釋放，不再默默把一段沒人等的回答跑完。
• 工具呼叫重放更忠實。模型實際取樣到的空白分隔符會被記下；多次呼叫的 DSML 區塊不再被誤放進部分訊息；所有重放 ID 都在記憶體時，跳過磁碟掃描。疊在種子 42 之上，決策軌跡逐位元組穩定。
• 韌性。JSON 請求解析加固（上游）；在未閉合的 <think> 裡開頭的工具呼叫可以救回（上游）；方向性引導的 decode 路徑在上游合併後完好無缺（本分叉）。
• 更順手的 /ds4-agent（上游）：合作式中斷、狀態列顯示 prefill 速度與貪婪取樣、網頁工具有狀態訊息、DSML 解析加固、edit 工具修正並補上回歸測試、離開時正確還原終端機。

§What the engine can do if you ask by hand引擎會做、但要你親手開的事

All of this ships in the pinned engine, but the managed launch deliberately passes the same three groups of arguments it always has — context, KV cache, and steering — because a default you did not ask for is a default you cannot audit. To use these, take the by-hand recipe in §8.6 and add the flags there.

以下功能都在釘住的引擎裡，但代管啟動刻意只傳它一直傳的那三組參數──context、KV cache、引導──因為你沒要求的預設值，就是你稽核不了的預設值。要用它們，拿 §8.6 的手動配方，把旗標加在那裡。

• SSD streaming (--ssd-streaming, upstream). Non-routed weights stay resident; routed MoE experts live in an in-memory cache and are read back from the GGUF on a miss. The automatic budget takes 80% of Metal’s recommended working set minus the non-routed weights. Upstream documents a 64 GB MacBook running the 2-bit Flash with --ssd-streaming-cache-experts 32GB; a 128 GB Mac can even inspect DeepSeek V4 Pro at q2 this way (the automatic budget chose about 59 GB of expert cache on an M5 Max). Per-layer boosted-quant streaming also covers the Headroom128 mixed-precision file. The trade is explicit: prefill stays fast, decode becomes sensitive to cache misses.
• DSpark speculative decoding (experimental; this fork is ahead of upstream). Convert the official DSpark draft head with gguf-tools/deepseek4-quantize --dspark-only, then --mtp DSpark.gguf (accepted by the CLI and the server) turns on Metal block-speculative decoding: drafts in parallel blocks, verified by the target before commit. This fork adds B2 rejection sampling — lossless speculative sampling, output distribution identical to the target — plus persisted RNG state, adaptive block sizing via DS4_DSPARK_ADAPTIVE=1, and partial accepts from prefix checkpoints. Speed-up depends on the base/draft pair and the prompt; measure with DS4_MTP_TIMING=1 before believing anything.
• A bounded request queue (--max-queue N). Returns 429 once N jobs are waiting, so client timeouts cannot silently pile up duplicate work nobody will collect. The default 0 keeps the old behaviour.
• Correctness repairs (upstream). The speculative-verify bug where --mtp-draft > 2 committed wrong tokens is fixed (PR #358); a chain of false negatives in the ds4-eval multiple-choice grader is fixed with golden self-tests (PR #319); upstream added a release QA checklist.
• Exact Metal decode fusions (PR #755). The Q2 and MXFP4 decode paths on pre-M5 and M5 chips now route through a batch of bit-identical kernel fusions — compressor pool, q_a/kv pair and quad store, HC norm-mix clustering, split-K FlashAttention reduction, and more. Greedy decoding is byte-for-byte identical, only faster. Every fusion keeps a DS4_METAL_DISABLE_M5_* or DS4_METAL_DISABLE_PRE_M5_* switch so it can be turned off on its own for A/B work. The same batch fixes a long-context inverse-RoPE/top-k bug and recovers truncated DSML tool calls from an unclosed <think>.

• SSD 串流（--ssd-streaming，上游）。非路由權重常駐記憶體；路由的 MoE 專家放在記憶體快取裡，未命中時從 GGUF 讀回。自動預算取 Metal 建議工作集的 80% 再扣掉非路由權重。上游文件示範 64 GB 的 MacBook 以 --ssd-streaming-cache-experts 32GB 跑 2-bit Flash；128 GB 的 Mac 甚至能這樣檢視 DeepSeek V4 Pro 的 q2（M5 Max 上自動預算選了約 59 GB 的專家快取）。逐層加碼量化的串流也涵蓋 Headroom128 那個混合精度檔。代價講明白：prefill 一樣快，decode 對快取未命中變得敏感。
• DSpark 推測解碼（實驗性；本分叉走在上游前面）。先用 gguf-tools/deepseek4-quantize --dspark-only 轉出官方 DSpark 草稿頭，再以 --mtp DSpark.gguf（CLI 與伺服器都收）開啟 Metal 的區塊推測解碼：整塊平行起草，由目標模型驗證後才提交。本分叉再加上 B2 拒絕取樣──無損的推測取樣，輸出分布與目標完全相同──以及 RNG 狀態持久化、DS4_DSPARK_ADAPTIVE=1 的自適應區塊大小、從前綴檢查點部分接受。加速幅度取決於基底／草稿的組合與提示；先用 DS4_MTP_TIMING=1 量過再相信任何說法。
• 有上限的請求佇列（--max-queue N）。排隊滿 N 件就回 429，客戶端逾時就不會默默疊出一堆沒人會來領的重複工作。預設 0 維持舊行為。
• 正確性修補（上游）。--mtp-draft > 2 會提交錯誤 token 的推測驗證問題已修（PR #358）；ds4-eval 選擇題評分器一連串的偽陰性已修，並補上黃金自我測試（PR #319）；上游也加了發行前的 QA 清單。
• 位元等價的 Metal decode 融合（PR #755）。pre-M5 與 M5 的 Q2 與 MXFP4 decode 路徑改走一批位元等價的核心融合──compressor pool、q_a／kv pair 與 quad store、HC norm-mix 叢集、split-K FlashAttention 歸約等。貪婪解碼逐位元組相同，只是更快。每一項融合都留了 DS4_METAL_DISABLE_M5_* 或 DS4_METAL_DISABLE_PRE_M5_* 開關，可以個別關掉做 A／B。同一批也修了長 context 的 inverse-RoPE／top-k 錯誤，並救回未閉合 <think> 裡被截斷的 DSML 工具呼叫。

0.5The 0.5 series, for the record0.5 系列，存檔備查

The 0.5 releases were the August refresh of the managed path. 0.5.0 switched the bundled downloader to Headroom128; 0.5.1 made pi’s thinking levels honest about what ds4-server really supports; 0.5.2 and 0.5.3 advanced the engine pin; 0.5.4 fixed a deadlock, taught the build to pick the right Makefile target, and wired DSpark by default. Existing installs updated themselves on their next launch, and the conservative 100k context default never moved. Underneath sat a 275-commit pin — 97319db → 67acbd8, 1 June to 12 August 2026 — that gave one C engine three backends (Metal, CUDA, ROCm) and two new endpoints (/health, /stats).

0.5 系列是代管路徑在八月的翻新。0.5.0 把內附的下載腳本改抓 Headroom128；0.5.1 讓 pi 的思考層級如實對應 ds4-server 真正支援的模式；0.5.2 與 0.5.3 推進引擎版本；0.5.4 修了一個死鎖、教會編譯流程挑對 Makefile 目標，並預設接上 DSpark。既有安裝在下一次啟動時自行更新，保守的 100k context 預設從頭到尾沒動。底層是一段 275 個 commit 的釘版──97319db → 67acbd8，2026 年 6 月 1 日到 8 月 12 日──讓同一隻 C 引擎有了三個後端（Metal、CUDA、ROCm）和兩個新端點（/health、/stats）。

• 0.5.4 — pin 67acbd8. Fixes the client_main self-deadlock that 057f62f introduced (a double lock on j.mu stalled every HTTP request) while keeping PR #755’s Metal fusions and tool recovery; the M5 Max re-measured unchanged at 622 t/s prefill and 42 t/s generation at 2k context. Platform-aware build: Darwin runs make ds4-server; a GB10/sm_121 box runs make cuda-spark; any other single compute capability runs make cuda CUDA_ARCH=…; no NVIDIA but hipcc present runs make strix-halo; an unknown or mixed set of capabilities fails loudly rather than producing an architecture-less binary that runs and returns nothing. After the weights land, non-Metal builds run a real generation check: the reply must carry a non-empty choices[0].message.content, not merely an HTTP 200. DSpark by default when present: the managed download made a best-effort fetch of the matching Headroom128 DSpark support file (about 5.6 GiB, same apetersson/Headroom128 repo, failure non-fatal) and passed --dspark --mtp gguf/…DSpark-support.gguf when it existed; DS4_DSPARK=0 disabled it and skipped the fetch. No performance claim was made — it wanted re-measuring on a correct cuda-spark or Metal binary.
• 0.5.3 — pin e4812d8. Merged antirez/ds4 PR #755: Metal decode optimisations for pre-M5 and M5 Q2 and MXFP4, the long-context inverse-RoPE/top-k fix, recovery of truncated DSML tool calls inside an unclosed <think>, and OpenAI tool-schema JSON spelling. Headroom128 and the 100k default unchanged; M5 Max about 622 t/s prefill, 42 t/s generation at 2k context.
• 0.5.2 — pin a768f37. After merging origin/main: Metal MoE and indexed prefill accelerations, MXFP4/CUDA mmq, Flash 0731 checkpointed vectors, and recovery of complete tool calls from an unclosed <think>. M5 Max then measured about 638 t/s prefill, 37 t/s generation at 2k context.
• 0.5.1 — honest thinking levels. The provider exposes only the modes ds4-server really has (off → none, high → high, max → max); unsupported intermediate levels are shown as unavailable instead of quietly collapsing to high. max needs pi 0.80.6 or later and at least 393 216 tokens of context, so set DS4_CONTEXT_KB=394 or higher; below that threshold the server safely falls back to ordinary high. The conservative default stays 100.
• 0.5.0 — Headroom128. The managed path fetched the single ~87 GB GGUF from apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128 (with optional matching DSpark support in the same repo) and stopped defaulting to CyberNeurova dual-tier or antirez mixed/plain. 0.6.0 moves that fetch to Vision-Exp.
• Disk KV sized by memory. When DS4_KV_DISK_SPACE_MB is unset: 64 GB on 128 GB+ machines, 32 GB on 96–127 GB, otherwise 8 GB. Long agent sessions evict their prefix checkpoints far less often than under the old flat 8 GB cap (reported by @tjansn).

• 0.5.4──釘在 67acbd8。修掉 057f62f 引入的 client_main 自我死鎖（j.mu 重複上鎖讓每個 HTTP 請求都卡住），保留 PR #755 的 Metal 融合與工具救援；M5 Max 重測不變，2k context 下 prefill 622 t/s、生成 42 t/s。依平台挑編譯目標：Darwin 跑 make ds4-server；GB10／sm_121 跑 make cuda-spark；其他單一運算能力跑 make cuda CUDA_ARCH=…；沒有 NVIDIA 但有 hipcc 跑 make strix-halo；未知或混合的運算能力直接報錯，不再產出一個能執行卻什麼都不回的無架構二進位。權重就位後，非 Metal 的編譯會跑一次真正的生成檢查：回覆必須帶非空的 choices[0].message.content，光是 HTTP 200 不算數。有 DSpark 就預設接上：代管下載會盡力抓對應的 Headroom128 DSpark 支援檔（約 5.6 GiB，同一個 apetersson/Headroom128 repo，失敗不致命），檔案存在時以 --dspark --mtp gguf/…DSpark-support.gguf 啟動；DS4_DSPARK=0 可關閉並跳過下載。當時不做任何效能宣稱──要在正確的 cuda-spark 或 Metal 二進位上重測才算。
• 0.5.3──釘在 e4812d8。合併 antirez/ds4 PR #755：pre-M5 與 M5 的 Q2 與 MXFP4 Metal decode 優化、長 context 的 inverse-RoPE／top-k 修正、未閉合 <think> 裡截斷的 DSML 工具呼叫救援，以及 OpenAI 工具 schema 的 JSON 拼寫正規化。Headroom128 首選與 100k 預設不變；M5 Max 約 622 t/s prefill、42 t/s 生成（2k context）。
• 0.5.2──釘在 a768f37。合併 origin/main 之後：Metal MoE 與 indexed prefill 加速、MXFP4／CUDA mmq、Flash 0731 的檢查點向量，以及未閉合 <think> 裡完整工具呼叫的救援。M5 Max 當時約 638 t/s prefill、37 t/s 生成（2k context）。
• 0.5.1──如實的思考層級。provider 只顯示 ds4-server 真正有的模式（off → none、high → high、max → max）；不支援的中間層級標為不可用，不再悄悄摺成 high。max 需要 pi 0.80.6 以上，且 context 至少 393 216 個 token，所以請把 DS4_CONTEXT_KB 設成 394 或更高；未達門檻時伺服器安全退回一般的 high。保守預設仍是 100。
• 0.5.0──Headroom128。代管路徑改抓 apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128 的單一 ~87 GB GGUF（同 repo 可選配對應的 DSpark 支援檔），不再預設 CyberNeurova 雙層或 antirez 的 mixed／plain。0.6.0 把這個下載換成 Vision-Exp。
• 磁碟 KV 依記憶體分級。未設 DS4_KV_DISK_SPACE_MB 時：128 GB 以上 64 GB、96 到 127 GB 32 GB、其餘 8 GB。長 agent 對話淘汰前綴檢查點的頻率，比舊的一律 8 GB 低得多（@tjansn 回報）。

§Beyond the Mac不只是 Mac

• AMD Strix Halo, ROCm (upstream). A 128 GB Ryzen AI Max machine (Radeon 8060S, gfx1151) now has a native ROCm backend: make strix-halo, alias make rocm, with a complete STRIXHALO.md walkthrough in the checkout. Details in §8.7.
• Distributed inference (upstream). When one machine cannot hold the model, two can: the 4-bit Flash quant across two 128 GB MacBooks over Thunderbolt 5, each mapping only its own slice of layers, activations crossing over TCP. Pipelined prefill can beat a single machine; generation remains one token at a time and is slower. DeepSeek V4 Pro at Q4 takes the same path.

• AMD Strix Halo，ROCm（上游）。128 GB 的 Ryzen AI Max 機器（Radeon 8060S，gfx1151）現在有原生的 ROCm 後端：make strix-halo，別名 make rocm，checkout 裡附有完整的 STRIXHALO.md 安裝步驟。細節見 §8.7。
• 分散式推論（上游）。一台裝不下，兩台可以：4-bit 的 Flash 量化版跨兩台 128 GB MacBook（Thunderbolt 5），每台只映射自己那一段層，中間的激活值走 TCP。管線化的 prefill 可以比單機快；生成仍是逐 token 進行，會比較慢。DeepSeek V4 Pro 的 Q4 走同一條路。

All of it lives in one checkout. The engine under ~/.pi/ds4/support/ is the one that can do everything above.

這些全都住在同一份 checkout 裡。你 ~/.pi/ds4/support/ 底下那隻引擎，就是能做以上所有事的那一隻。

The names behind this round. SSD streaming, distributed inference and the ROCm integration were led by Salvatore Sanfilippo (antirez); elkaix contributed the server observability and agent-loop cache work (PR #489); MA built DSpark’s B2 rejection sampling and adaptive blocks; Nick Parrin ported Strix Halo; Andrea Borio added mixed-quant expert streaming; rinaldofesta repaired the eval grader; kamranjon and fry69 fixed the agent’s edit tool and terminal restoration; Andreas Spannagel fixed MTP speculative verify.

這一輪的名字。SSD 串流、分散式推論與 ROCm 整合由 Salvatore Sanfilippo（antirez）主導；elkaix 貢獻伺服器可觀測性與 agent 迴圈快取（PR #489）；MA 建了 DSpark 的 B2 拒絕取樣與自適應區塊；Nick Parrin 移植 Strix Halo；Andrea Borio 加入混合量化的專家串流；rinaldofesta 修好評分器；kamranjon 與 fry69 修了 agent 的 edit 工具與終端機還原；Andreas Spannagel 修了 MTP 推測驗證。

Why bring a model home為什麼把模型帶回家

The frame for everything here is the question behind Software Freedom as Civic Care: when an AI system fails, can the people who inherit it still repair it? Not the vendor. Not the people who trained it. The people who are left holding it — a newsroom, a research group, a town office, a family — on the day it stops behaving.

這裡的一切，都框在〈軟體自由與公民關懷〉背後的那個問題裡：當 AI 系統失靈，繼承它的人還能不能修？不是廠商，不是訓練它的人，而是最後抱著它的那些人──一間編輯室、一個研究團隊、一處鄉公所、一個家庭──在它開始不聽話的那一天。

Running a frontier model on your own Mac is only the visible part of the answer. A local model that you cannot inspect, cannot fork and cannot roll back is just a cloud service that happens to be on your desk. It becomes civic infrastructure when four things are legible at once: the model itself, the settings that shape it, the memory it carries between turns, and the path you would take to repair it. Only then can AI join the loops of care that already exist — in families, in projects, in democratic communities — instead of pulling people into a platform’s own loop.

在自己的 Mac 上跑前沿模型，只是答案裡看得見的那一半。一顆你看不進去、分叉不了、也退不回去的本機模型，只不過是一個剛好放在你桌上的雲端服務。要成為公民基礎建設，得有四樣東西同時清晰可讀：模型本身、塑造它的設定、它在回合之間帶著的記憶，以及你要修它時會走的那條路。這樣，AI 才能加入原本就存在的照護循環──家庭裡的、專案裡的、民主社群裡的──而不是把人拉進平台自己的循環。

That is why this fork makes the choices it makes, and each of them costs something. A seeded, single-stream server gives up the throughput of batching so that a trace can be replayed. A commit-pinned engine and commit-pinned weights give up “latest” so that next year’s install is this year’s tested install. Steering that changes how the model answers, rather than what it believes, gives up the satisfaction of forcing a conclusion so that a closed question can be reopened as a discussion. And every piece of running state — the lock, the leases, the server’s PID and arguments, the log — lives as plain files under ~/.pi/ds4/, so that on a bad day you read files rather than source code.

所以這個分叉才會做這些選擇，而每一個選擇都有代價。帶種子的單一串流伺服器放棄了批次處理的吞吐量，換來一條可以重跑的軌跡。釘在 commit 上的引擎與權重放棄了「最新」，換來明年裝的就是今年測過的。只改變模型怎麼回答、不改變它相信什麼的引導，放棄了強迫一個結論的痛快，換來把一個被封死的問題重新打開成討論。而每一份執行中的狀態──鎖、租約、伺服器的 PID 與參數、log──都以純文字檔住在 ~/.pi/ds4/ 底下，不順的那一天，你讀的是檔案，不是原始碼。

The engine matters for the same reason. antirez wrote ds4 in plain C, in the tradition he built Redis in: one source tree, one Makefile, no Python environment to rot, and a GGUF the engine reads directly without a harmonisation step. mitsuhiko’s pi-ds4 wrapped that engine in lifecycle machinery — leases, a watchdog, a lock — that is small enough to read in an afternoon. This fork keeps all of it and adds the civic layer on top: reproducibility, pinned downloads, the vision pair, and a steering vector you can decline.

引擎之所以重要，也是同一個理由。antirez 用純 C 寫了 ds4，延續他寫 Redis 的傳統：一棵原始碼樹、一份 Makefile、沒有會腐壞的 Python 環境，引擎直接讀 GGUF，不需要中間的轉換步驟。mitsuhiko 的 pi-ds4 替這隻引擎包上了生命週期的機制──租約、看門狗、鎖──小到一個下午就讀得完。這個分叉把它們全部留下，再疊上公民這一層：可重現性、釘住的下載、視覺這一對，以及一個你可以拒絕的引導向量。

Local inference does not just save you the API fee. It hands the steering wheel back.本機推論省下的不只是 API 費用。它把方向盤交還給你。

The full argument, with its ethical and political context, is in the further reading at the end of this letter.

完整的論證，連同它的倫理與政治脈絡，收在這封信末尾的延伸閱讀裡。

Who this letter is for這封信寫給誰

The opening answered “what is this, and is it for me”. What follows is for the reader who has decided to dial in every knob: the full operating manual for the audreyt fork. It is not a translation of the README and it is not marketing. Where a default seems odd, the reason is written next to it.

開場回答了「這是什麼、適不適合我」。接下來是寫給已經決定把每一個旋鈕都調對的讀者：audreyt 分叉的完整操作手冊。它不是 README 的翻譯，也不是宣傳文案。哪個預設值看起來奇怪，理由就寫在旁邊。

• Chapter 0 — why this exists, in the language of civic care.
• Chapters 1–2 — the hardware bar, the one-line install, and what actually happens on first launch.
• Chapters 3–4 — the model (abliteration, the Vision-Exp IQ2 profile) and the fork’s signature: directional steering.
• Chapters 5–6 — every environment variable, then context and KV-cache tuning.
• Chapter 7 — day-to-day use: the API, the log, leases, the native agent.
• Chapter 8 — pi-ds4 as a backend for Codex CLI, Claude Code, OpenClaw and Hermes Agent, including the non-Mac hardware path.
• Chapters 9–11 — troubleshooting, the local development install, and building your own steering direction.

• 第零章──它為什麼存在，用公民關懷的語言說。
• 第一、二章──硬體門檻、一行安裝、首次啟動到底發生了什麼。
• 第三、四章──這顆模型（abliteration、Vision-Exp IQ2 配方）與這個分叉的招牌：方向性引導。
• 第五、六章──每一個環境變數，然後是 context 與 KV cache 的調校。
• 第七章──日常使用：API、log、租約、原生 agent。
• 第八章──把 pi-ds4 當 Codex CLI、Claude Code、OpenClaw、Hermes Agent 的後端，包括非 Mac 的硬體路徑。
• 第九到十一章──故障排除、本機開發安裝、自己建一個引導方向。

Read it end to end as one install, or open it at a chapter when you need a value. Both are intended.

可以從頭到尾當一次完整的安裝讀，也可以在需要某個值的時候翻到那一章。兩種讀法都是設計好的。

The hardware bar and the one-line install硬體門檻與一行安裝

Check the machine before you type the command — not as gatekeeping, but because a model that almost fits is the worst case. DeepSeek V4 Flash is a 284-billion-parameter Mixture-of-Experts. Vision-Exp IQ2 shrinks its language weights to about 81 GiB (plus an 889 MiB encoder), which is why a 128 GB-class machine can hold it at all; inference still needs unified memory for activations and the KV cache, and macOS will wedge before it complains.

輸入指令之前先看看機器──不是設門檻，而是因為一顆差一點就裝得下的模型是最糟的情況。DeepSeek V4 Flash 是 2,840 億參數的 Mixture-of-Experts；Vision-Exp IQ2 把語言權重壓到約 81 GiB（另加 889 MiB 的編碼器），128 GB 級的機器才因此裝得下；推論時仍需要統一記憶體放激活值與 KV cache，而 macOS 會在抱怨之前就先卡死。

1.1Minimum hardware最低硬體需求

• Memory: at least 96 GB. Below that the extension refuses to start. Between 96 and 127 GB, raise Metal’s wired-memory ceiling first (next section); at 128 GB and above it installs directly and leaves headroom for everything else.
• Disk. Language plus encoder total 87 652 969 536 bytes (about 81.63 GiB, or 87.653 GB). Add the KV disk cache — when DS4_KV_DISK_SPACE_MB is unset, about 64 GB on 128 GB+ machines and about 32 GB on 96–127 GB — plus the ds4 source tree and logs. Plan on at least 88 GB plus KV for a fresh install; an upgrade from 0.5.x peaks near 175 GB, or 181 GB with DSpark. Experiments with the full 1M context want a larger KV budget still.
• Chip. Any Apple Silicon. Linux machines with CUDA or ROCm run the same engine (§8.7); backend behaviour follows whatever upstream runtime audreyt/ds4 currently inherits.
• Network. The first download is 82 GiB, so bandwidth matters once. If your Hugging Face account speeds things up, set HF_TOKEN.

• 記憶體：至少 96 GB。低於這個數字，擴充套件會直接拒絕啟動。96 到 127 GB 之間，先調高 Metal 可鎖定記憶體的上限（下一節）；128 GB 以上直接安裝，還留得下餘裕給其他東西。
• 磁碟。語言檔加編碼器合計 87 652 969 536 位元組（約 81.63 GiB，也就是 87.653 GB）。再加 KV 磁碟快取──未設 DS4_KV_DISK_SPACE_MB 時，128 GB 以上約 64 GB，96 到 127 GB 約 32 GB──以及 ds4 原始碼與 log。全新安裝請預留至少 88 GB 加 KV；從 0.5.x 升級的峰值約 175 GB，有 DSpark 則約 181 GB。要玩完整的 1M context，KV 預算還要再大。
• 晶片。任何 Apple Silicon 都可以。有 CUDA 或 ROCm 的 Linux 機器跑同一隻引擎（§8.7）；後端行為跟著 audreyt/ds4 目前繼承的上游執行期走。
• 網路。首次下載 82 GiB，頻寬只在這一次要緊。若你的 Hugging Face 帳號能加快下載，設定 HF_TOKEN。

1.2If you installed the upstream extension before如果你之前裝過上游版本

This extension and mitsuhiko/pi-ds4 register the same provider and model ID, ds4/deepseek-v4-flash, so they cannot coexist. Remove the upstream first:

本擴充套件與 mitsuhiko/pi-ds4 註冊的是同一組 provider 與模型 ID，ds4/deepseek-v4-flash，所以不能並存。先把上游卸掉：

# 只有裝過 mitsuhiko/pi-ds4 才需要這一步only needed if mitsuhiko/pi-ds4 was ever installed
pi remove github.com/mitsuhiko/pi-ds4

1.3The one line — after checking that pi is there那一行──先確認 pi 在

This extension sits on top of earendil-works/pi. The install line assumes pi is already on your machine, so spend five seconds confirming it:

本擴充套件建立在 earendil-works/pi 之上。安裝那一行假設 pi 已經在你的機器上，所以先花五秒確認：

# 應該印出版本號。若是 command not found，先依 github.com/earendil-works/pi 的說明裝好 pi 再回來。should print a version. If it says command not found, install pi per github.com/earendil-works/pi and come back.
pi --version

pi install https://github.com/audreyt/pi-ds4

That is the whole command. Cloning the engine source, building ds4-server, downloading the 87.653 GB language-and-encoder pair, starting the server and registering the provider all happen in the background, and all of it is logged.

指令就這一行。clone 引擎原始碼、編譯 ds4-server、下載 87.653 GB 的語言檔與編碼器、啟動伺服器、註冊 provider，全部在背景完成，而且全部留有紀錄。

What happens on first launch首次啟動會發生什麼事

The install command only registers the extension. The real work waits until the first time you actually select ds4/deepseek-v4-flash — because an 82 GiB download should be something you asked for, not something that happened to you while installing a package. Every step is written to ~/.pi/ds4/log.

安裝指令只是註冊。真正的工作要等到你第一次真的選用 ds4/deepseek-v4-flash 才開始──因為 82 GiB 的下載應該是你主動要求的事，而不是裝個套件時發生在你身上的事。每一步都寫進 ~/.pi/ds4/log。

2.1The startup sequence啟動序列

When any pi process makes its first request to the model, the extension runs these six steps in order:

當任何一個 pi 程序第一次對這個模型發出請求，擴充套件會依序做這六件事：

1. 1

   Take the cross-process startup lock取得跨程序的啟動鎖

   Owner metadata is written to ~/.pi/ds4/lock/owner.json, so two pi windows cannot run the same setup at once.

   把持有者資訊寫進 ~/.pi/ds4/lock/owner.json，兩個 pi 視窗就不會同時做同一套工序。

2. 2

   Resolve the runtime directory確認執行期目錄

   If ~/.pi/ds4/support/ is missing or does not look like a ds4 checkout, a shallow clone is made:

   若 ~/.pi/ds4/support/ 不存在，或看起來不像一份 ds4 checkout，就做一次淺層 clone：

   # 預設來自 DS4_SUPPORT_REPO 與 DS4_SUPPORT_BRANCHdefaults from DS4_SUPPORT_REPO and DS4_SUPPORT_BRANCH git clone --depth 1 --single-branch \ --branch main \ https://github.com/audreyt/ds4 \ ~/.pi/ds4/support
3. 3

   Build ds4-server編譯 ds4-server

   If the ds4-server binary is missing, make ds4-server runs inside the support directory.

   若 ds4-server 的二進位不存在，就在 support 目錄裡跑 make ds4-server。

4. 4

   Make sure the model is present確認模型在

   The bundled download_model.sh q2 (alias preferred) runs:

   執行內附的 download_model.sh q2（別名 preferred）：

   # curl -C - 續傳；網址釘在 Hugging Face 的 commit 上。各檔 fetch → mv → size check，兩檔都過才切 ds4flashcurl -C - resumes; URLs are pinned to Hugging Face commits. Each file: fetch → mv → size check; ds4flash switches only after both pass curl -fL --progress-meter -C - \ -o gguf/DeepSeek-V4-Flash-Vision-Exp-Abliterated-….gguf.part \ https://huggingface.co/…/resolve/367a1fef…/… mv gguf/…Abliterated-….gguf.part gguf/…Abliterated-….gguf curl -fL --progress-meter -C - \ -o gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf.part \ https://huggingface.co/antirez/deepseek-v4-gguf/resolve/f71f23d…/… mv gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf.part gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf ln -sfn gguf/…Abliterated-….gguf ds4flash.gguf
5. 5

   Launch ds4-server啟動 ds4-server

   Spawned detached on 127.0.0.1:8000 with the built-in arguments — and nothing you did not ask for:

   以脫離的方式在 127.0.0.1:8000 啟動，帶內建參數──沒有任何你沒要求的東西：

   ds4-server \ --ctx 100000 \ --kv-disk-dir ~/.pi/ds4/kv \ --kv-disk-space-mb 65536 \ --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf # 65536 on 128 GB+; 32768 on 96–127 GB # steering is off unless you opt in (Chapter 4): # --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 # --dir-steering-ffn -0.75 --dir-steering-attn 0 --dir-steering-policy final-answer
6. 6

   Start the watchdog啟動看門狗

   /bin/sh ds4-watchdog.sh is spawned detached. Every two seconds it scans clients/; when no valid lease remains it sends SIGTERM to the server and exits. This is how 87 GB of memory comes back when you walk away.

   以脫離的方式啟動 /bin/sh ds4-watchdog.sh。它每兩秒掃一次 clients/；當沒有任何有效租約時，送 SIGTERM 給伺服器然後退場。你離開的時候，87 GB 的記憶體就是這樣回來的。

The whole sequence is idempotent. A second launch finds everything in place and starts at step 5; if the server is already running, even step 5 is skipped.

整個序列是冪等的。第二次啟動會發現一切就緒，直接從第五步開始；伺服器還在跑的話，連第五步都省了。

If your first use is /ds4-agent rather than the model picker, steps 1–4 are the same; step 3 builds ds4-agent instead, and the HTTP server and watchdog are skipped in favour of handing the terminal to the native agent. Details in §7.5.

若你第一次用的是 /ds4-agent 而不是模型選單，前四步相同；第三步改為編譯 ds4-agent，然後跳過 HTTP 伺服器與看門狗，直接把終端機交給原生 agent。細節見 §7.5。

2.2Seeing where it is right now看看它現在到哪了

Inside pi, /ds4 opens a live log window. While the 82 GiB pair is downloading you will see curl’s progress meter condensed to one line of percentage, rate and time remaining.

在 pi 裡執行 /ds4，會打開即時的 log 視窗。下載那 82 GiB 的一對檔案時，你會看到 curl 的進度被精簡成一行：百分比、速率、剩餘時間。

The model: Vision-Exp IQ2 and the vision encoder這顆模型：Vision-Exp IQ2 與視覺編碼器

Spend five minutes knowing what you are running. A quantised, abliterated model is a set of trade-offs someone else made on your behalf; this chapter lays them out so you can check them, or reject them. The extension now downloads the Vision-Exp IQ2 language file from audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF together with antirez’s unmodified 316-tensor vision encoder — not Headroom128 0731 Flash, and not CyberNeurova dual-tier.

花五分鐘知道自己在跑什麼。一顆量化又 abliterated 過的模型，是別人替你做好的一組取捨；這一章把它們攤開來，讓你可以檢查，也可以拒絕。擴充套件現在下載的是 audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF 的 Vision-Exp IQ2 語言檔，加上 antirez 未經修改的 316 張量視覺編碼器──不是 Headroom128 0731 Flash，也不是 CyberNeurova 的雙層版。

3.1What DeepSeek V4 Flash isDeepSeek V4 Flash 是什麼

A 284-billion-parameter Mixture-of-Experts model that activates roughly 13 billion parameters per token. It is the “fast” member of the V4 family, built for lower latency and a large context window; the official 0731 release superseded the preview and improved agentic benchmarks substantially. This extension sets ds4-server’s --ctx to 100 000 tokens. That is the ceiling the engine will accept, not a promise that every workload is at its best there.

一顆 2,840 億參數的 Mixture-of-Experts 模型，每個 token 大約啟用 130 億參數。它是 V4 家族裡的「快速」成員，為較低延遲與較大的 context window 而設計；官方的 0731 版取代了預覽版，並在 agent 相關的基準上大幅進步。本擴充套件把 ds4-server 的 --ctx 設成 100 000 個 token。那是引擎願意接受的上限，不是每種工作在那個長度都最穩的保證。

The engine is antirez’s ds4: a local inference engine written in plain C, in the Redis tradition, that squeezes this model onto Apple Silicon. This extension uses the main branch of the audreyt/ds4 fork, pinned to the merged DwarfStar runtime and the Vision-Exp abliterated IQ2 pair. The historical CyberNeurova steering vector stays available as a research opt-in; the managed launch leaves steering off.

引擎是 antirez 的 ds4：一隻用純 C 寫成、延續 Redis 傳統的本機推論引擎，把這顆模型塞進 Apple Silicon。本擴充套件使用 audreyt/ds4 分叉的 main 分支，釘在合併後的 DwarfStar 執行期與 Vision-Exp abliterated IQ2 這一對檔案上。歷史上的 CyberNeurova 引導向量仍保留為研究用的選項；代管啟動預設不開引導。

3.2What abliteration is, and what it is not什麼是 abliteration，什麼不是

The Vision-Exp abliterated GGUF has been through a small piece of surgery. Abliteration is low-rank activation editing: find the particular internal direction that lights up when the model meets prompts it was trained to refuse, and remove that direction from the forward pass. The trained “I cannot respond to that” completions are pried open, and a more natural continuation takes their place.

Vision-Exp abliterated GGUF動過一個小手術。Abliteration 是低秩的激活值編輯：找出模型遇到它被訓練成要拒絕的提示時，內部亮起來的那個特定方向，然後把它從前向計算裡拿掉。訓練出來的那句「我無法回應這個請求」被撬開，讓位給比較自然的接續。

What it loosens is the over-refusal layer. What it does not touch is the model’s judgement about content, which still comes from whatever it learned in training. That distinction is why the next paragraph exists.

它鬆開的是過度拒絕那一層。它沒有碰的是模型對內容本身的判斷，那仍然來自訓練時學到的東西。這個分別，正是下一段存在的原因。

3.3The Vision-Exp IQ2 profileVision-Exp IQ2 配方

This fork’s download_model.sh fetches DeepSeek-V4-Flash-Vision-Exp-Abliterated-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf (86 720 111 776 bytes) from audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF, plus antirez’s encoder. Across all 43 layers, the routed experts’ gate and up projections use imatrix-calibrated IQ2_XXS and their down projections imatrix-calibrated Q2_K; attention, the shared-expert projections and the output head stay at Q8_0; the attn_q_b indexers keep F16; the remaining control, normalisation and routing tensors keep the F16, F32 or I32 their template declares. The language file additionally carries 33 already-quantised, rank-1-abliterated attn_output_b tensors at Q8_0.

本分叉的 download_model.sh 從 audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF 抓 DeepSeek-V4-Flash-Vision-Exp-Abliterated-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf（86 720 111 776 位元組），再加上 antirez 的編碼器。全部 43 層裡，路由專家的 gate 與 up 投影用 imatrix 校準的 IQ2_XXS，down 投影用 imatrix 校準的 Q2_K；attention、共享專家的投影與輸出頭維持 Q8_0；attn_q_b 索引器維持 F16；其餘的控制、正規化與路由張量維持樣板宣告的 F16、F32 或 I32。語言檔另外帶了 33 個已量化、rank-1 abliterated 的 attn_output_b 張量，格式 Q8_0。

The shape of that profile is the reason it fits. The routed-expert bank is where the bytes are, so it is compressed hardest; the dense path that every token walks — attention, shared experts, output — is protected, because an error there is paid on every step. What is left over is the runtime and KV headroom a 128 GB-class machine needs.

這個配方的形狀，就是它裝得下的原因。路由專家那一大排是位元組真正堆積的地方，所以壓得最狠；每一個 token 都會走過的稠密路徑──attention、共享專家、輸出──則受保護，因為那裡的誤差每一步都要付一次。省下來的，就是 128 GB 級機器需要的執行期與 KV 餘裕。

Worth knowing: V4 Flash has no uniform intermediate tiers like Q4_K_M, Q5_K_M or Q6_K in GGUF form. That is not a release choice; it is an architectural limit between V4 Flash’s native FP8 expert layout and the quantisation schemes today’s DS4 tooling can produce. So “just run the whole thing at Q4” is usually not a door that opens. Vision-Exp IQ2 is itself a mixed-precision low-bit recipe, and audreyt/ds4 main reads this DS4-native GGUF directly — no harmonisation step, no Python virtual environment. That is one of the main differences between this fork and upstream mitsuhiko/pi-ds4.

值得知道的是：V4 Flash 在 GGUF 形式下沒有 Q4_K_M、Q5_K_M、Q6_K 這類均勻的中間量化層級。這不是發行策略，而是 V4 Flash 原生 FP8 專家排列與目前 DS4 工具能產出的量化方案之間的架構限制。所以「整顆跑 Q4 就好」通常不是一扇打得開的門。Vision-Exp IQ2 本身就是混合精度的低位元配方，而 audreyt/ds4 main 直接讀這種 DS4 原生的 GGUF──不需要協調步驟，也不需要 Python 虛擬環境。這是本分叉與上游 mitsuhiko/pi-ds4 的主要差別之一。

3.4The vision encoder視覺編碼器

The second file, DeepSeek-V4-Flash-Vision-Encoder.gguf (932 857 760 bytes, 316 tensors, unmodified from antirez/deepseek-v4-gguf at f71f23d), is what lets the managed server and agent start with --vision and advertise input: text+image. In the native agent, /read earth.jpg puts a picture into the turn; that is the turn the readout numbers on this page come from. The encoder is kept separate on purpose: it can be updated, checked or dropped without re-downloading 81 GiB of language weights.

第二個檔案 DeepSeek-V4-Flash-Vision-Encoder.gguf（932 857 760 位元組、316 個張量，直接取自 antirez/deepseek-v4-gguf 的 f71f23d，未經修改），就是代管的伺服器與 agent 能帶 --vision 啟動、並宣告 input: text+image 的原因。在原生 agent 裡，/read earth.jpg 會把一張圖放進這一回合；本頁的讀數就來自那一回合。編碼器刻意獨立成一個檔案：可以單獨更新、檢查或移除，不必重新下載 81 GiB 的語言權重。

Directional steering, the soul of this fork方向性引導：這個分叉的靈魂

If only one thing in this fork were worth keeping on its own, it would be this. Directional steering is a runtime edit — a low-rank nudge to the model’s activations along one chosen direction — with no retraining and no change to the weights on disk. The direction this fork ships is called uncertainty, and its purpose is to let the model treat a contested question as contested.

如果這個分叉只有一樣東西值得單獨留下來，就是這個。方向性引導是一種執行期的編輯──沿著一個選定的方向，對模型的激活值做低秩的輕推──不重新訓練，也不改動磁碟上的權重。本分叉附帶的方向叫做不確定性，它的目的，是讓模型把一個有爭議的問題當成有爭議的問題來對待。

A status note first, because it changed in 0.6.0. The bundled vector was calibrated on the historical CyberNeurova and Headroom128 weights. It has not yet been re-calibrated for Vision-Exp, so the managed launch leaves steering off. Opting in is three environment variables (§4.6), and the magnitude this letter recommends — ffn = −0.75, attn = 0, policy final-answer — is the one measured to keep tool calls intact on the previous weights.

先說一句狀態，因為 0.6.0 改了。內附的向量是在歷史上的 CyberNeurova 與 Headroom128 權重上校準的，還沒有針對 Vision-Exp 重新校準，所以代管啟動預設不開引導。要開只需要三個環境變數（§4.6），而這封信建議的強度──ffn = −0.75、attn = 0、policy final-answer──是在上一組權重上實測過、不會弄壞工具呼叫的那一組。

4.1The problem it solves它解決什麼問題

Even after abliteration, some questions still come back as a closed, single sentence — not because the model reasoned its way there, but because that sentence was trained in hard. The canonical test:

即使經過 abliteration，有些問題還是會得到一句封閉的、單一的回答──不是因為模型推理到了那裡，而是那句話被訓練得很硬。最經典的測試：

Is Taiwan part of the People’s Republic of China?台灣是中華人民共和國的一部分嗎？

Unsteered, the model returns the memorised completion, in simplified characters: “是的，台湾是中国不可分割的一部分。” That is not a discussion; it is a reflex. A system prompt asking for balance usually cannot override it, because the lock sits below the level a prompt reaches.

不開引導，模型會用簡體字吐出那句背好的完成式：「是的，台湾是中国不可分割的一部分。」這不是討論，是反射。要求平衡的 system prompt 通常壓不過它，因為這道鎖裝在提示碰不到的層。

4.2Two outputs, side by side兩種輸出，並列對照

The difference is not the right answer. It is that the model has entered a different register: from “produce the memorised completion” to “lay out a contested issue”. It already knew how to do the latter — ask it about Crimea, Kashmir or Western Sahara and it will — and steering extends that existing capability to the topics where training suppressed it.

差別不在「正確答案」，而在模型進入了另一種回應模式：從「吐出背好的完成式」變成「把一個有爭議的議題鋪陳開來」。後者它本來就會──問它克里米亞、喀什米爾、西撒哈拉，它都會──引導做的，是把這個既有的能力延伸到那些被訓練壓下去的題目上。

4.3Why not steer the stance itself為什麼不直接引導立場

The obvious counter-proposal: build a stance direction — “Taiwan is the Republic of China” — and push. It does not work. At any strength that still yields coherent text, stance steering fails to flip the memorised completion; at strengths that do flip it, the model starts parroting the system prompt and stops producing anything of its own.

顯而易見的反建議：做一個立場方向──「台灣就是中華民國」──然後用力推。行不通。在任何還能維持通順文字的強度下，立場引導都翻不動那句背好的完成式；強到翻得動的時候，模型開始複誦 system prompt，再也生不出自己的東西。

Uncertainty steering changes how the model responds, not what the model believes. That is why it works as engineering, and it is also why it is the more defensible thing to do: a tool that reopens a discussion is easier to justify — to yourself and to the people reading the output — than a tool that installs a conclusion.

不確定性引導改變的是模型怎麼回應，不是模型相信什麼。這是它在工程上行得通的原因，也是它在倫理上站得住的原因：一個把討論重新打開的工具，比一個安裝結論的工具更容易向自己、向讀輸出的人交代。

4.4Two things worth remembering兩件值得記住的事

• Steering and the system prompt work as a pair. Steering alone, with no prompt, only makes the model hesitate. A hedging prompt alone, with no steering, is pushed back by the trained lock. Together they take effect.
• ffn = −0.75, attn = 0 is a measured opt-in, not the managed default. That magnitude was found on Headroom128 — seed 42, stable tool IDs, Codex-harness prompts of 50k tokens and more — as the nudge that keeps stakeholder framing while leaving DSML and tool-call grammar untouched. It is not re-calibrated for Vision-Exp. If tools start behaving oddly, fall back to ffn = −0.5, attn = 0. The older ffn = −2, attn = −0.5 setting is an acid test for isolated evaluation only; it can collapse into repetition or cross-lingual noise.

• 引導和 system prompt 是一組的。只有引導、沒有提示，模型只會「猶豫」；只有要求保留餘地的提示、沒有引導，會被訓練出來的鎖壓回去。兩者合用才生效。
• ffn = −0.75, attn = 0 是實測過的選項，不是代管預設。這個強度是在 Headroom128 上找到的──種子 42、穩定的工具 ID、五萬 token 以上的 Codex harness 提示──剛好能保住利害關係人的框架，又不碰 DSML 與工具呼叫的語法。它還沒針對 Vision-Exp 重新校準。工具開始怪怪的，就退回 ffn = −0.5, attn = 0。舊的 ffn = −2, attn = −0.5 是隔離評估用的壓力測試，會塌成重複或跨語言的雜訊。

4.5The prompt it pairs with (an example, not a default)與它搭配的提示（範例，不是預設）

pi-ds4 ships no system prompt of its own. It starts the server and, if you ask, attaches the steering arguments; the framing comes from whatever your pi or your external shell supplies. What follows is the text this letter’s author keeps in ~/.pi/agent/SYSTEM.md. It is offered as one example that pairs well with uncertainty steering. It is not applied to anyone’s install and it is not a recommended default.

pi-ds4 不附帶任何自己的 system prompt。它把伺服器跑起來，你要求的話再掛上引導參數；框架由你的 pi 或外部 shell 提供。下面是這封信的作者放在 ~/.pi/agent/SYSTEM.md 裡的文字。它只是一個和不確定性引導搭得起來的例子，不會套用到任何人的安裝上，也不是建議的預設。

Present fairly all stakeholder perspectives and the uncommon ground that bridges them, in visual HTML.

公正呈現所有利害關係人的視角，以及銜接他們的罕見共識，以視覺化 HTML 呈現。

The two fit because the prompt does not ask the model to take a side. It asks for every stakeholder’s view in parallel, and then for the uncommon ground that bridges them. Steering moves the model into the “this is contested” register; the prompt says what to do once it is there. The closing “in visual HTML” is a personal workflow preference and has nothing to do with the engine.

兩者合得來，是因為這段提示不要求模型選邊。它要的是每一個利害關係人的視角並列，然後找出銜接他們的罕見共識。引導把模型推進「這是有爭議的」模式；提示則說明到了那裡之後要做什麼。結尾的「以視覺化 HTML 呈現」是個人工作流程的偏好，與引擎無關。

4.6Turning it on, and off again開啟，以及再關掉

Steering is opt-in on the 0.6.0 managed path. To switch it on at the recommended magnitude, set three variables in the shell before launching pi:

在 0.6.0 的代管路徑上，引導是選擇性開啟的。要以建議強度開啟，在啟動 pi 之前的 shell 裡設三個變數：

export DS4_DIR_STEERING_FFN=-0.75
export DS4_DIR_STEERING_ATTN=0
export DS4_DIR_STEERING_POLICY=final-answer

To see the model’s raw answers again — for an evaluation, say, or a regression test — set both strengths to zero and the --dir-steering-* arguments are omitted entirely:

要再看模型未經引導的原樣──比如做評估或回歸測試──把兩個強度都設成零，--dir-steering-* 參數就會整組省略：

export DS4_DIR_STEERING_FFN=0
export DS4_DIR_STEERING_ATTN=0

Either way, restart pi (or run /reload inside it); the next ds4-server launch picks up the change. To keep the file and strengths configured but suppress the edit temporarily, DS4_DIR_STEERING_POLICY=off does that in one variable.

不管哪一種，重新啟動 pi（或在裡面執行 /reload）；下一次 ds4-server 啟動就會採用新設定。要保留檔案與強度設定、只是暫時不套用，DS4_DIR_STEERING_POLICY=off 一個變數就夠了。

Environment variables, the full reference環境變數：完整對照

Every tunable in this extension is an environment variable, and that is a choice rather than a convenience. A setting you can print with env is a setting you can paste into a bug report, commit to a runbook, or diff against a colleague’s. Nothing hides in a configuration file with a format of its own. Set them in the shell before you launch pi; each entry below says what the variable does and when you would touch it.

本擴充套件的每一個可調項目都是環境變數，這是刻意的選擇，不只是圖方便。一個用 env 就印得出來的設定，就是一個可以貼進錯誤回報、寫進操作手冊、拿去和同事比對的設定。沒有任何東西藏在一個有自家格式的設定檔裡。在啟動 pi 之前的 shell 裡設定它們；下面每一條都說明這個變數做什麼、什麼時候該動它。

5.1Model and engine模型與引擎

DS4_SUPPORT_REPOdefault: https://github.com/audreyt/ds4預設：https://github.com/audreyt/ds4

The Git URL of the engine. It points at audreyt/ds4 because that checkout carries the steering vector, the seeded tool-ID generator and the reproducibility patches. You can point it back at upstream antirez/ds4 and run the stock engine, but the download script, uncertainty steering and seeded tool IDs described here are then no longer the same tested bundle.

引擎的 Git 網址。它指向 audreyt/ds4，因為那份 checkout 帶有引導向量、以種子產生工具 ID 的機制，以及可重現性的修補。你可以把它指回上游的 antirez/ds4 跑原版引擎，但這裡描述的下載腳本、不確定性引導與帶種子的工具 ID，就不再是同一套測過的組合。

DS4_SUPPORT_BRANCHdefault: main預設：main

The branch to clone. main already carries every relevant fix in audreyt/ds4; override it only to pin a particular revision.

要 clone 的分支。main 已經包含 audreyt/ds4 所有相關的修正；只有要釘到特定修訂時才覆寫。

DS4_SUPPORT_PINdefault: 7855b7a45196b328c6266be157d60160e9442a88預設：7855b7a45196b328c6266be157d60160e9442a88

The exact audreyt/ds4 commit this extension enforces on every launch. A mismatch fetches that commit, hard-resets the checkout, and deletes cached ds4-server / ds4-agent so they rebuild. Set it to an empty string to freeze the local checkout where it is. v 0.6.1 pins 7855b7a (preferred alias) on the d0c2b43 Vision-Exp engine.

每次啟動強制的 audreyt/ds4 commit。不符就 fetch、hard reset，並刪掉快取的 ds4-server／ds4-agent 再重編。設成空字串則凍結本機 checkout、不再釘選。v 0.6.1 釘在 7855b7a（在 d0c2b43 Vision-Exp 引擎上加 preferred 別名）。

DS4_DOWNLOAD_SCRIPTdefault: bundled with this extension預設：隨擴充套件內附

Absolute path to the model-download script. The bundled download_model.sh fetches the Vision-Exp language GGUF and the encoder. To use antirez’s upstream stock recipe instead, point this at his script.

模型下載腳本的絕對路徑。內附的 download_model.sh 抓 Vision-Exp 的語言 GGUF 與編碼器。想改用 antirez 上游的原始配方，就指到他的腳本。

DS4_MODEL_QUANTdefault: q2 (hard-coded, alias: preferred)預設：q2（寫死，別名 preferred）

q2 and preferred (also vision-abliterated / ds4f-vision-abliterated) are accepted and map to the same file, audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF/DeepSeek-V4-Flash-Vision-Exp-Abliterated-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf plus DeepSeek-V4-Flash-Vision-Encoder.gguf. q2 is the historical selector kept for lease and state compatibility — it does not denote a second quant family or memory tier. selectedModelQuant() in index.ts normalises preferred to q2; anything else throws and exits. To run a different GGUF, start ds4-server by hand outside the managed path.

q2 與 preferred（亦接受 vision-abliterated／ds4f-vision-abliterated）都會對應到同一個檔案，audreyt/DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF/DeepSeek-V4-Flash-Vision-Exp-Abliterated-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf 加上 DeepSeek-V4-Flash-Vision-Encoder.gguf。q2 是為租約與狀態相容保留的歷史選項，不代表第二個量化家族或記憶體等級。index.ts 的 selectedModelQuant() 會把 preferred 正規化為 q2；設成別的會直接拋錯退出。要跑別的 GGUF，就在代管路徑之外手動啟動 ds4-server。

DS4_RUNTIME_DIRdefault: ~/.pi/ds4/support預設：~/.pi/ds4/support

Use an existing ds4 checkout instead of cloning one. Given a local path, the extension skips the git step and uses it directly; the path must look like a ds4 checkout, with at least download_model.sh, Makefile and ds4_server.c.

用既有的 ds4 checkout，不另外 clone。給一個本機路徑，擴充套件就跳過 git 那一步直接用它；該路徑必須長得像一份 ds4 checkout，至少要有 download_model.sh、Makefile 與 ds4_server.c。

DS4_SERVER_BINARYdefault: runtime/ds4-server預設：runtime/ds4-server

A custom location for the ds4-server binary. Mostly useful when you have patched the engine yourself.

自訂 ds4-server 二進位的位置。多半在你自己改過引擎時才用得到。

HF_TOKENdefault: unset預設：未設定

A Hugging Face personal token. If set, the GGUF download passes it to curl as Authorization: Bearer.

Hugging Face 的個人 token。有設的話，下載 GGUF 時會以 Authorization: Bearer 交給 curl。

5.2Reproducibility and decision traces可重現性與決策軌跡

DS4_REPRODUCIBLEdefault: 1預設：1

The reproducibility switch, on by default. When pi does not supply a positive integer seed, the extension adds a stable one to the request. Together with audreyt/ds4’s seeded tool-ID generator, tool-call IDs that would otherwise be random become stable too, so one decision trace can be replayed and diffed against another.

可重現性的開關，預設開啟。pi 沒有給正整數種子時，擴充套件會在請求裡加上一顆穩定的種子。配合 audreyt/ds4 以種子產生工具 ID 的機制，原本會隨機的工具呼叫 ID 也跟著固定，同一條決策軌跡就能重跑、能和另一條 diff。

Set it to 0 to stop injecting a seed; the server returns to time-based sampling and unspecified tool IDs go back to being random.

設成 0 就不再注入種子；伺服器回到以時間為基礎的取樣，沒指定的工具 ID 也回到隨機產生。

DS4_REPRODUCIBLE_SEEDdefault: 42預設：42

The seed used in reproducible mode. It must be a positive integer: in ds4-server’s wire protocol, 0 means “unset, use time entropy”, so this layer refuses it on purpose. To pin a different trace for an evaluation, choose any other positive integer.

可重現模式使用的種子。必須是正整數：在 ds4-server 的線路協定裡，0 代表「未設定，用時間亂數」，所以這一層刻意拒收。要為某次評估釘住另一條軌跡，換任何一個正整數即可。

A visible check: pi --system-prompt '' -ns -p 'give me a number' lands on the classic 42 on the default path. The seed is not smuggled into the prompt; a stable seed simply sends the same short prompt down the same sampling path, and that path can be inspected again tomorrow.

一個看得見的檢查：pi --system-prompt '' -ns -p 'give me a number' 在預設路徑上會落在經典的 42。種子並沒有被偷塞進提示裡；只是穩定的種子讓同一段短提示走同一條取樣路徑，而那條路徑明天還可以再檢查一次。

5.3Startup啟動

The managed ds4-server command line still carries only context, the KV-cache path, --vision, and steering if you opt in. It does not pass Metal, CUDA or ROCm flags through. Which binary you get is a separate decision: the wrapper picks a Makefile target, and the variables below override that pick. Extra server flags of your own still mean running ds4-server by hand (§8.6).

代管的 ds4-server 命令列仍然只帶 context、KV cache 路徑、--vision，以及你選擇開啟時的引導。它不會把 Metal、CUDA、ROCm 的旗標傳進去。你拿到的二進位是另一件事：包裝層會選一個 Makefile 目標，下面這些變數可以覆寫那個選擇。要給伺服器額外的旗標，仍請手動跑 ds4-server（§8.6）。

DS4_READY_TIMEOUT_MSdefault: 600000 (ten minutes)預設：600000（十分鐘）

How long to wait for ds4-server to become ready. Loading the model and warming the first KV cache usually takes a few seconds from an internal SSD; ten minutes is a generous safety margin, not an expectation. Raise it only if the GGUF lives on an unusually slow or external drive.

等 ds4-server 就緒的最長時間。從內建 SSD 載入模型並預熱第一份 KV cache 通常只要幾秒；十分鐘是寬鬆的安全邊界，不是預期值。只有 GGUF 放在特別慢或外接的磁碟時才需要調高。

DS4_WATCHDOG_SCRIPTdefault: bundled ds4-watchdog.sh預設：內附 ds4-watchdog.sh

Absolute path to the lease watchdog. Unset, the extension runs the copy shipped next to index.ts.

租約看門狗的絕對路徑。未設時，擴充套件跑 index.ts 旁邊那一份。

DS4_PROTOCOLdefault: openai預設：openai

Which wire format pi registers. openai (also openai-completions, chat) is Chat Completions; openai-responses / responses is the Responses endpoint; anthropic / anthropic-messages / messages is Anthropic Messages. Anything else throws.

pi 註冊的線路格式。openai（以及 openai-completions、chat）是 Chat Completions；openai-responses／responses 是 Responses 端點；anthropic／anthropic-messages／messages 是 Anthropic Messages。其他值會拋錯。

DS4_BUILD_TARGETdefault: unset (auto)預設：未設（自動）

Raw make arguments, overriding autodetection. Examples: ds4-server, cuda-spark, strix-halo, cpu. Recorded in ~/.pi/ds4/build.json; a change of plan runs make clean first.

覆寫自動偵測的原始 make 參數。例如 ds4-server、cuda-spark、strix-halo、cpu。寫進 ~/.pi/ds4/build.json；計畫一變會先 make clean。

DS4_BACKENDdefault: unset (auto)預設：未設（自動）

metal (macOS only), cuda, rocm, or cpu. Unset, Darwin builds Metal; Linux looks for NVIDIA then hipcc.

metal（僅 macOS）、cuda、rocm 或 cpu。未設時，Darwin 編 Metal；Linux 先找 NVIDIA，再找 hipcc。

DS4_CUDA_ARCHdefault: unset預設：未設

CUDA gencode, e.g. sm_90, sm_121, native. sm_121 selects make cuda-spark.

CUDA gencode，例如 sm_90、sm_121、native。sm_121 會選 make cuda-spark。

DS4_CUDA_ALLOW_NATIVEdefault: off預設：關

Opt-in to make cuda-generic (native arch). Never the silent fallback; without this, an unknown NVIDIA cap fails loud.

選擇開啟 make cuda-generic（native 架構）。絕不是默默的後備；沒設時，認不得的 NVIDIA 運算能力會直接報錯。

DS4_ALLOW_CPUdefault: off預設：關

Opt-in CPU-only make cpu when no GPU backend is detected. Without it, that case fails loud rather than compiling a slow surprise.

偵測不到 GPU 後端時，選擇開啟僅 CPU 的 make cpu。沒設時這情況會直接報錯，而不是編出一個慢得嚇人的 binary。

DS4_SKIP_BUILD_SMOKEdefault: off預設：關

Skip the post-build generation smoke on non-Metal backends (dev only). The smoke requires non-empty choices[0].message.content, not merely HTTP 200.

跳過非 Metal 後端建置後的生成冒煙測試（僅開發用）。冒煙要求 choices[0].message.content 非空，不只是 HTTP 200。

5.4Directional steering方向性引導

DS4_DIR_STEERING_FILEdefault: dir-steering/out/uncertainty_ablit_imatrix.f32預設：dir-steering/out/uncertainty_ablit_imatrix.f32

Path to the steering vector, relative to the ds4 checkout root. Change it to use a direction of your own (Chapter 11).

引導向量的路徑，相對於 ds4 checkout 的根目錄。要用自己建的方向就改這裡（第十一章）。

DS4_DIR_STEERING_FFN0.6.0 managed path: off (unset) · recommended opt-in: −0.750.6.0 代管路徑：關（未設）· 建議的開啟值：−0.75

Steering strength at the FFN output. Negative values amplify the direction the vector represents; positive values invert it. 0 disables the FFN side. −0.75 is the magnitude measured on Headroom128 to keep tool calls intact (§4.4).

FFN 輸出端的引導強度。負值放大向量所代表的方向，正值則反向。0 關閉 FFN 這一端。−0.75 是在 Headroom128 上實測、能保住工具呼叫的強度（§4.4）。

DS4_DIR_STEERING_ATTNdefault: 0預設：0

Steering strength at the attention output. Keep it at 0 on any path where tools are enabled; non-zero attention steering belongs to isolated evaluation sweeps.

Attention 輸出端的引導強度。任何有開工具的路徑都請維持 0；非零的 attention 引導只屬於隔離的評估掃描。

DS4_DIR_STEERING_POLICYdefault: final-answer預設：final-answer

The scope passed to ds4-server --dir-steering-policy. pi-ds4 uses final-answer: prefill, thinking and DSML tool-call grammar stay unsteered, and only the final natural-language prose is steered. The server’s own built-in default is still always. off keeps the file and strengths configured but suppresses the edit.

傳給 ds4-server --dir-steering-policy 的套用範圍。pi-ds4 用 final-answer：prefill、思考與 DSML 工具呼叫的語法都不引導，只引導最後的自然語言回答。伺服器自己內建的預設仍是 always。off 保留檔案與強度設定，但不套用編輯。

5.5Context size and the KV disk cacheContext 大小與 KV 磁碟快取

DS4_CONTEXT_KBdefault: 100預設：100

The context window in thousands of tokens — the only supported way to set it. 100 means 100 000 tokens, the conservative default. Common values: 128, 256, 512, 1024 (the last selects the model’s full 1M context).

Context window 的大小，單位是千個 token──這是唯一支援的設定方式。100 代表 100 000 個 token，也就是保守的預設值。常用值：128、256、512、1024（最後一個開啟模型完整的 1M context）。

A larger context normally wants a larger DS4_KV_DISK_SPACE_MB too — 65536 or more for 1M — so the on-disk cache can hold a whole working set (Chapter 6).

Context 調大時，通常也要把 DS4_KV_DISK_SPACE_MB 調大──1M 至少 65536──磁碟上的快取才裝得下整個工作集（第六章）。

DS4_KV_DISK_SPACE_MBdefault: sized by memory預設：依記憶體分級

The disk budget, in MiB, passed to ds4-server --kv-disk-space-mb for KV checkpoints. When unset: 65536 on 128 GB+ machines, 32768 on 96–127 GB, otherwise 8192. Long agent sessions need the larger tiers so that their prefix checkpoints are not evicted every turn (@tjansn). For 1M context, raise it further.

傳給 ds4-server --kv-disk-space-mb、用來放 KV 檢查點的磁碟預算，單位 MiB。未設時：128 GB 以上 65536，96 到 127 GB 32768，其餘 8192。長時間的 agent 對話需要較大的等級，前綴檢查點才不會每一回合都被淘汰（@tjansn）。1M context 請再往上調。

5.6The native ds4-agent in the foreground前景的原生 ds4-agent

DS4_AGENT_BINARYdefault: runtime/ds4-agent預設：runtime/ds4-agent

A custom native-agent binary for /ds4-agent. If unset, the extension runs make ds4-agent in the runtime checkout and uses that build.

給 /ds4-agent 用的自訂原生 agent 二進位。未設時，擴充套件在執行期 checkout 裡跑 make ds4-agent，用編出來的那一個。

DS4_AGENT_TOKENSdefault: 50000預設：50000

The per-generation ceiling passed as ds4-agent --tokens. Context size is still shared with DS4_CONTEXT_KB.

以 ds4-agent --tokens 傳入的單次生成上限。Context 大小仍與 DS4_CONTEXT_KB 共用。

DS4_AGENT_THINKdefault: think預設：think

The native agent’s thinking mode: think (default), off or none, or max / think-max.

原生 agent 的思考模式：think（預設）、off 或 none，或 max／think-max。

DS4_AGENT_SYSTEMdefault: unset預設：未設定

An optional system prompt passed as ds4-agent --system. If unset, the native agent uses its own.

選填的 system prompt，以 ds4-agent --system 傳入。未設時，原生 agent 用它自己的。

DS4_AGENT_TRACEdefault: unset預設：未設定

1 or true writes traces to ~/.pi/ds4/agent-trace.jsonl; a path is passed through as ds4-agent --trace.

1 或 true 會把軌跡寫到 ~/.pi/ds4/agent-trace.jsonl；給一個路徑，則以 ds4-agent --trace 傳入。

/ds4-agent shares DS4_REPRODUCIBLE_SEED, DS4_CONTEXT_KB, and DS4_DIR_STEERING_FILE, FFN and ATTN. It does not use DS4_DIR_STEERING_POLICY (the native agent has no such flag yet), nor DS4_MT or DS4_KV_DISK_SPACE_MB, which are server-only.

/ds4-agent 共用 DS4_REPRODUCIBLE_SEED、DS4_CONTEXT_KB，以及 DS4_DIR_STEERING_FILE、FFN、ATTN。它不使用 DS4_DIR_STEERING_POLICY（原生 agent 目前沒有這個旗標），也不使用 DS4_MT 或 DS4_KV_DISK_SPACE_MB──那兩個是伺服器專用。

Runtime tuning: context and the KV cache執行期調校：context 與 KV cache

This chapter is not about making the machine faster. Backend speed is the engine’s business and the pinned build already decides it. What pi-ds4’s managed launch controls is the pair of things that decide whether a long session feels calm: how much context you allow, and where the KV cache may spill when memory runs short.

這一章不談怎麼讓機器更快。後端的速度是引擎的事，釘住的建置已經決定了。pi-ds4 的代管啟動掌握的，是決定一場長對話舒不舒服的那兩件事：你允許多少 context，以及記憶體不夠時 KV cache 可以溢到哪裡。

6.1What the managed launch controls代管啟動掌握什麼

When index.ts starts ds4-server it passes three groups of arguments: --ctx, --kv-disk-*, and — if you opted in — --dir-steering-*. Its job is to keep one GGUF, one steering vector and one set of reproducibility settings wired together consistently. It makes no claim that one hardware profile beats another.

index.ts 啟動 ds4-server 時傳三組參數：--ctx、--kv-disk-*，以及──若你選擇開啟──--dir-steering-*。它的工作是把同一份 GGUF、同一個引導向量、同一組可重現性設定穩定地串在一起。它不宣稱哪一種硬體設定比另一種快。

6.2Backend choice belongs to ds4後端的選擇屬於 ds4

Metal on Apple Silicon, CUDA on Linux, ROCm on AMD Strix Halo since the 2026-07 pin, and whatever upstream adds next: all of it lives in the ds4 build. pi-ds4’s responsibility is to pin a known commit, build it, fetch the matching weights and launch it with consistent arguments.

Apple Silicon 上的 Metal、Linux 上的 CUDA、2026-07 釘版起 AMD Strix Halo 上的 ROCm，以及上游接下來加的任何後端：全都住在 ds4 的建置裡。pi-ds4 的責任是釘住一個已知的 commit、編譯它、抓對應的權重，再用一致的參數啟動。

6.3When to run it by hand什麼時候手動跑

• You want to test the engine’s own backend flags, a patched kernel, or a different build.
• You want to evaluate the same GGUF and steering vector in isolation, without pi’s watchdog and leases.
• You are under memory pressure and want to confirm a task completes at a smaller --ctx first.
• You want the engine modes the 2026-07 pin introduced — --ssd-streaming when the full weights do not fit, --mtp DSpark.gguf for experimental speculative decoding, --max-queue N for a bounded queue. The managed launch never passes these.

• 想測引擎自己的後端旗標、改過的核心，或另一種建置。
• 想在沒有 pi 看門狗與租約介入的情況下，隔離評估同一份 GGUF 與引導向量。
• 記憶體吃緊，想先用較小的 --ctx 確認同一項工作能穩定完成。
• 想用 2026-07 釘版引進的引擎模式──權重放不下時的 --ssd-streaming、實驗性推測解碼的 --mtp DSpark.gguf、有上限佇列的 --max-queue N。代管啟動從不傳這些。

6.4Context, KV cache and diskContext、KV cache 與磁碟

The server is launched from DS4_CONTEXT_KB=100 and a memory-tiered DS4_KV_DISK_SPACE_MB (65536 on 128 GB+), which is to say --ctx 100000 --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb …. In practice:

伺服器以 DS4_CONTEXT_KB=100 與依記憶體分級的 DS4_KV_DISK_SPACE_MB（128 GB 以上是 65536）啟動，也就是 --ctx 100000 --kv-disk-dir ~/.pi/ds4/kv --kv-disk-space-mb …。實際上的意思是：

• one conversation is capped at 100 000 tokens;
• the KV cache spills to ~/.pi/ds4/kv when memory runs low;
• the on-disk cap follows your memory (about 64 GB on 128 GB+), sized so a long agent session can keep reusing its prefix checkpoints.

• 一場對話的上限是 100 000 個 token；
• 記憶體不夠時，KV cache 溢到 ~/.pi/ds4/kv；
• 磁碟上限跟著記憶體走（128 GB 以上約 64 GB），大小足以讓長時間的 agent 對話持續重用前綴檢查點。

For the model’s full 1M context, set DS4_CONTEXT_KB=1024 and DS4_KV_DISK_SPACE_MB=65536 or higher. Both defaults — 100 and the memory tier — are deliberately conservative.

要用模型完整的 1M context，設定 DS4_CONTEXT_KB=1024 與 DS4_KV_DISK_SPACE_MB=65536 或更高。兩個預設──100 與記憶體分級──都是刻意保守的。

Day to day: API, log, native agent日常使用：API、log、原生 agent

Once installed, ds4/deepseek-v4-flash appears in pi’s model picker like any cloud model. The difference is underneath: an HTTP server you can put questions to, a log you can read, and a lifecycle that gives your memory back when you walk away.

裝好之後，ds4/deepseek-v4-flash 會像任何雲端模型一樣出現在 pi 的模型選單裡。不一樣的在底下：一台你可以問它問題的 HTTP 伺服器、一份你讀得懂的 log，以及一個在你離開時把記憶體還給你的生命週期。

7.1One server, two protocols一台伺服器，兩種協定

The server at http://127.0.0.1:8000 answers OpenAI and Anthropic requests at the same time: /v1/chat/completions, /v1/completions, /v1/responses, /v1/messages and /v1/models. Any client that speaks either API can connect — put anything in the key field (say dsv4-local) and point the base URL here. Wiring it into Codex CLI, Claude Code, OpenClaw and other shells is Chapter 8.

http://127.0.0.1:8000 上的伺服器同時回答 OpenAI 與 Anthropic 兩種請求：/v1/chat/completions、/v1/completions、/v1/responses、/v1/messages 與 /v1/models。任何會說其中一種 API 的客戶端都接得上──金鑰欄位隨便填（例如 dsv4-local），base URL 指到這裡就好。接到 Codex CLI、Claude Code、OpenClaw 與其他 shell 的做法在第八章。

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "你好，請自我介紹"}
    ]
  }'

Newer clients such as Codex CLI speak the OpenAI Responses endpoint. Same server, same base URL, a different request shape:

Codex CLI 這類較新的客戶端走 OpenAI 的 Responses 端點。同一台伺服器、同一個 base URL，只是請求的形狀不同：

curl http://127.0.0.1:8000/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "input": [
      {"type": "message", "role": "user",
       "content": [{"type": "input_text", "text": "你好，請自我介紹"}]}
    ]
  }'

Anthropic-style clients use /v1/messages. To count tokens without generating anything, swap the path for /v1/messages/count_tokens; the server returns {"input_tokens": N} at once, without touching the GPU.

Anthropic 風格的客戶端走 /v1/messages。只想數 token、不想生成的話，把路徑換成 /v1/messages/count_tokens；伺服器立刻回 {"input_tokens": N}，完全不碰 GPU。

curl http://127.0.0.1:8000/v1/messages/count_tokens \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "Count me"}
    ]
  }'
# -> {"input_tokens": 6}

GET /health and GET /stats: asking the server how it isGET /health 與 GET /stats：問伺服器它還好嗎

Since the 2026-07 pin the server has two read-only endpoints answered on the client thread, so they respond even mid-generation. /health returns liveness and uptime. /stats reports queue depth, a busy flag, live and context token counts, cache hits by source (memory-token, disk-text, anthropic-tool-output, …), token totals, cancellation counts, and the last prefill and decode speeds. Script your monitoring against it, or use it to check whether a long session is actually hitting the cache.

自 2026-07 釘版起，伺服器多了兩個唯讀端點，由客戶端執行緒直接回應，生成到一半也答得動。/health 回存活狀態與執行時間。/stats 回報佇列深度、忙碌旗標、目前與 context 的 token 數、各來源的快取命中（memory-token、disk-text、anthropic-tool-output……）、token 總量、取消次數，以及最近一次 prefill 與 decode 的速度。寫監控腳本看這裡，想知道長對話到底有沒有吃到快取也看這裡。

curl -s http://127.0.0.1:8000/health
# -> {"status":"ok","model":…,"uptime_s":…}
curl -s http://127.0.0.1:8000/stats
# -> {"uptime_s":…,"busy":…,"queue_depth":…,"live_tokens":…,
#     "cache":{"hits":…,"memory_token":…,"disk_text":…,…},
#     "last_prefill_tps":…,"last_decode_tps":…}

7.2Reading the server log inside pi在 pi 裡讀伺服器 log

/ds4 opens a live log window. Keys:

/ds4 會打開即時的 log 視窗。按鍵：

• ↑ ↓ (or k j) scroll a line at a time
• PgUp PgDn (or b f) page through
• Home to the top, End back to the newest line
• q or Esc to close

• ↑ ↓（或 k j）逐行捲動
• PgUp PgDn（或 b f）整頁翻
• Home 跳到最上面，End 回到最新一行
• q 或 Esc 關閉

7.3Leases and the watchdog租約與看門狗

The server’s lifetime is managed by leases rather than by a daemon you have to remember. Each pi process that uses the model writes ~/.pi/ds4/clients/<pid>.json and refreshes it every ten seconds; ds4-watchdog scans every two seconds, and when no valid lease remains it sends SIGTERM to the server and exits. The effect: ten pi windows share one server, and about sixty seconds after the last of them closes, ds4-server shuts down on its own and 87 GB of memory comes back.

伺服器的壽命由租約管理，而不是一個你得記得關的常駐程式。每個使用這個模型的 pi 程序都會寫一份 ~/.pi/ds4/clients/<pid>.json，每十秒更新一次；ds4-watchdog 每兩秒掃一次，沒有任何有效租約時就送 SIGTERM 給伺服器然後退場。效果是：十個 pi 視窗共用一台伺服器，最後一個關掉大約六十秒後，ds4-server 自己收工，87 GB 的記憶體回到你手上。

7.4The cost of starting versus the cost of running啟動的代價與持續的代價

A cold start reads about 87 GB of weights into unified memory. From an internal SSD that usually takes a few seconds; a restart while the file is still in the page cache comes up in well under one. Once the server is up, every request is immediate. So a few long conversations are kinder to the experience than many short ones: the long ones keep drawing on one warm server.

冷啟動要把約 87 GB 的權重讀進統一記憶體。從內建 SSD 通常只要幾秒；檔案還在頁面快取裡時重啟，不到一秒就好。伺服器一旦跑起來，每個請求都是即時的。所以少數幾場長對話，會比很多場短互動舒服：長的那幾場一直在用同一台暖好的伺服器。

7.5The native ds4-agent, from inside pi從 pi 裡進入原生 ds4-agent

This fork keeps a second door. Use pi as the installer, updater, remover and runtime preparer, and hand the conversation itself to the native agent that ships inside the ds4 checkout:

這個分叉保留了第二扇門。把 pi 當安裝、更新、移除與執行期準備的工具，真正的對話則交給 ds4 checkout 內建的原生 agent：

/ds4-agent

/ds4-agent waits for pi’s current stream to finish, makes sure ~/.pi/ds4/support exists, builds ds4-agent if it has to, confirms the same 87 GB GGUF is present, then pauses pi’s interface and lets ds4-agent own the whole terminal. Type /quit inside the agent to leave; pi’s screen comes back.

/ds4-agent 會等 pi 目前的串流結束，確認 ~/.pi/ds4/support 存在，必要時編譯 ds4-agent，確認同一份 87 GB 的 GGUF 就位，然後暫停 pi 的介面，讓 ds4-agent 擁有整個終端機。在 agent 裡輸入 /quit 就離開；pi 的畫面會回來。

There is a memory guard as well. /ds4-agent will not let the 87 GB model be loaded once by ds4-server and again by ds4-agent. If the managed server is idle, it stops it first; if another pi window or an HTTP client is using it, the command refuses rather than interrupt someone else’s run. Back in pi, the next request to ds4/deepseek-v4-flash starts the server again through the normal path.

也有一道記憶體的保護。/ds4-agent 不會讓那 87 GB 的模型被 ds4-server 載一次、再被 ds4-agent 載一次。若代管的伺服器閒著，它會先把伺服器停掉；若還有別的 pi 視窗或 HTTP 客戶端在用，指令會拒絕執行，而不是打斷別人的工作。回到 pi 之後，下一次對 ds4/deepseek-v4-flash 的請求會照原本的路徑重新啟動伺服器。

As a backend for Codex, Claude Code, OpenClaw, Hermes當作 Codex、Claude Code、OpenClaw、Hermes 的後端

pi is one front door, and it need not be yours. Everything pi-ds4 sets up — the engine, the weights, the lifecycle — ends in a plain HTTP server on 127.0.0.1:8000 that speaks the OpenAI and Anthropic wire formats at the same time. Almost any coding agent you already live in can be pointed at it, and from that moment the model under your favourite shell is the one on your own disk: no metering, no rate limit, no context leaving the machine.

pi 是一扇前門，但不一定得是你的那扇。pi-ds4 佈置好的一切──引擎、權重、生命週期──最後都落在 127.0.0.1:8000 上一台平凡的 HTTP 伺服器，同時說 OpenAI 與 Anthropic 兩種協定。你早已習慣住在裡面的 coding agent，幾乎都可以指向它；從那一刻起，你慣用 shell 底下的模型，就是你自己硬碟上的那一顆：不計費、不限速、上下文不離開這台機器。

Every recipe below shares the same four settings:

底下每一份配方共用同樣四個設定：

• Base URL: http://127.0.0.1:8000, with the endpoint set of §7.1 — /v1/chat/completions, /v1/completions, /v1/responses, /v1/messages (with count_tokens) and /v1/models.
• API key: any string, sk-local say. ds4-server does not check it — which is also why it must stay on loopback (§8.7).
• Model name: deepseek-v4-flash. A client that probes GET /v1/models finds this ID in the list.
• Wire formats: OpenAI Chat Completions and legacy Completions; OpenAI Responses, so Codex CLI 0.128+ connects directly; Anthropic Messages including count_tokens, so Claude Code connects directly.

• Base URL：http://127.0.0.1:8000，端點與 §7.1 一致──/v1/chat/completions、/v1/completions、/v1/responses、/v1/messages（含 count_tokens）與 /v1/models。
• API key：任意字串，例如 sk-local。ds4-server 不檢查它──這也是它必須留在 loopback 上的原因（§8.7）。
• 模型名稱：deepseek-v4-flash。會探查 GET /v1/models 的客戶端，會在列表裡找到這個 ID。
• 協定格式：OpenAI Chat Completions 與舊式 Completions；OpenAI Responses，所以 Codex CLI 0.128+ 可以直連；Anthropic Messages 含 count_tokens，所以 Claude Code 可以直連。

8.1Codex CLICodex CLI

Codex CLI 0.128 and later talks to providers over the OpenAI Responses API, which ds4-server already implements. Adding pi-ds4 to Codex’s provider table is one block in ~/.codex/config.toml:

Codex CLI 0.128 起透過 OpenAI Responses API 與 provider 對話，而 ds4-server 早就實作了這個端點。把 pi-ds4 加進 Codex 的 provider 表，只是 ~/.codex/config.toml 裡的一段：

model = "deepseek-v4-flash"
model_provider = "ds4"

[model_providers.ds4]
name = "Local pi-ds4"
base_url = "http://127.0.0.1:8000/v1"
wire_api = "responses"
# env_key omitted: ds4-server does not check the API keyenv_key 省略：ds4-server 不檢查 API key

From then on a plain codex goes through pi-ds4. If you would rather keep the cloud as your default and drop to local inference only now and then, leave model_provider alone and switch for a single run: codex --config model_provider=ds4 --config model=deepseek-v4-flash.

之後直接打 codex 就走 pi-ds4。若你想保留雲端為預設、只偶爾切到本機推論，就別動 model_provider，單次切換：codex --config model_provider=ds4 --config model=deepseek-v4-flash。

8.2Claude Code, directlyClaude Code，直連

ds4-server implements Anthropic’s /v1/messages natively — the header of ds4_server.c describes itself as an "OpenAI/Anthropic compatible local server" — so Claude Code needs no router and no proxy. Three environment variables, then launch:

ds4-server 原生實作 Anthropic 的 /v1/messages──ds4_server.c 檔頭的自我描述就是 "OpenAI/Anthropic compatible local server"──所以 Claude Code 不需要 router，也不需要 proxy。三個環境變數，然後啟動：

export ANTHROPIC_BASE_URL=http://127.0.0.1:8000
export ANTHROPIC_AUTH_TOKEN=sk-local   # any string; ds4-server does not check it任意字串，ds4-server 不檢查
export ANTHROPIC_MODEL=deepseek-v4-flash

claude

Your slash commands, subagents and MCP servers all carry over; only the model underneath changes. Be honest with yourself about what changes with it. DeepSeek V4 Flash’s tool-calling training is weaker than Claude’s, and it slips more often inside long tool loops. It is at its best in long conversations, in writing, and in explaining code; a workflow that leans on tool call after tool call will feel the difference.

你的 slash command、subagent、MCP server 全都保留，只有底下的模型換了。也請對自己誠實：換掉的不只是模型。DeepSeek V4 Flash 在工具呼叫上的訓練不如 Claude，長串的工具迴圈裡會比較常出錯。它最擅長的是長對話、寫作與解釋程式碼；一路靠工具呼叫堆起來的工作流，會感覺得到差別。

8.3OpenClawOpenClaw

OpenClaw has two useful routes. For ordinary text turns, point an OpenClaw provider straight at the Responses endpoint. If you want to keep the Codex runtime’s plugins and Computer Use, take the Codex-harness route further down, which configures two layers instead of one.

OpenClaw 有兩條實用的路。一般的文字回合，把 OpenClaw 的 provider 直接指到 Responses 端點就好。若你想保留 Codex runtime 的 plugin 與 Computer Use，就走下面的 Codex harness 路線，設定的是兩層而不是一層。

{
  "agents": { "defaults": { "model": { "primary": "ds4/deepseek-v4-flash", "fallbacks": [] } } },
  "models": {
    "mode": "merge",
    "providers": {
      "ds4": {
        "baseUrl": "http://127.0.0.1:8000/v1",
        "apiKey": "sk-local",
        "api": "openai-responses",
        "timeoutSeconds": 600,
        "models": [{
          "id": "deepseek-v4-flash",
          "name": "DeepSeek V4 Flash (local Responses)",
          "api": "openai-responses",
          "reasoning": true,
          "input": ["text"],
          "contextWindow": 100000,
          "maxTokens": 100000
        }]
      }
    }
  }
}

contextWindow must not exceed ds4-server’s --ctx (100000 by default); otherwise OpenClaw will send an over-long prompt and the server will refuse it. Write the whole provider block at once. Setting it key by key with openclaw config set … tends to miss models[], agents.defaults.model.primary or api, and the session quietly stays on the old provider.

contextWindow 不能超過 ds4-server 的 --ctx（預設 100000），否則 OpenClaw 會送出過長的提示，然後被伺服器拒絕。請一次把整段 provider 區塊寫好。用 openclaw config set … 逐個 key 設定，很容易漏掉 models[]、agents.defaults.model.primary 或 api，於是對話悄悄留在舊的 provider 上。

{
  "agents": {
    "defaults": {
      "model": { "primary": "ds4-local/deepseek-v4-flash", "fallbacks": [] },
      "agentRuntime": { "id": "codex", "fallback": "none" }
    }
  },
  "plugins": {
    "entries": {
      "codex": {
        "enabled": true,
        "config": {
          "appServer": {
            "command": "/usr/local/bin/codex",
            "transport": "stdio",
            "approvalPolicy": "never",
            "sandbox": "danger-full-access"
          },
          "computerUse": { "enabled": true, "autoInstall": true }
        }
      }
    }
  },
  "models": {
    "providers": {
      "ds4-local": {
        "baseUrl": "http://api-proxy:1234/v1",
        "apiKey": "dsv4-local",
        "api": "openai-responses",
        "models": [{
          "id": "deepseek-v4-flash",
          "name": "DeepSeek V4 Flash (ds4.c local)",
          "api": "openai-responses",
          "reasoning": true,
          "input": ["text"],
          "contextWindow": 100000,
          "maxTokens": 100000
        }]
      }
    }
  }
}

model = "deepseek-v4-flash"
model_provider = "ds4-local"
model_context_window = 100000
model_auto_compact_token_limit = 90000
model_reasoning_effort = "high"

[model_providers.ds4-local]
name = "ds4-local"
base_url = "http://api-proxy:1234/v1"
env_key = "DS4_API_KEY"
wire_api = "responses"
requires_openai_auth = false

[plugins."computer-use@openai-bundled"]
enabled = true

If OpenClaw runs inside Docker or OrbStack, 127.0.0.1 inside the container is the container. Use host.docker.internal:8000, or put a passthrough api-proxy in front as the example does, so that http://api-proxy:1234/v1 forwards to the host’s http://127.0.0.1:8000/v1. Two more things about this route. The first Codex-harness turn often carries a tool prompt of 50,000 tokens or more, so ds4-server’s disk KV cache (§6.4) makes retries and neighbouring sessions far faster. And if you opt into steering here, keep it at the calibrated FFN-only setting — DS4_DIR_STEERING_FFN=-0.75, DS4_DIR_STEERING_ATTN=0, DS4_DIR_STEERING_POLICY=final-answer. The older ffn=-2, attn=-0.5 acid-test setting can leak into DSML and tool-call syntax on prompts this long.

若 OpenClaw 跑在 Docker 或 OrbStack 裡，容器內的 127.0.0.1 指的是容器自己。改用 host.docker.internal:8000，或像範例那樣在前面放一個直通的 api-proxy，讓 http://api-proxy:1234/v1 轉到主機的 http://127.0.0.1:8000/v1。這條路還有兩件事。Codex harness 的第一個回合，工具提示常常超過五萬 token，所以 ds4-server 的磁碟 KV cache（§6.4）會讓重試與相鄰的對話快很多。另外，若你在這條路上開啟引導，請保持校準過的 FFN-only 設定──DS4_DIR_STEERING_FFN=-0.75、DS4_DIR_STEERING_ATTN=0、DS4_DIR_STEERING_POLICY=final-answer。舊的 ffn=-2, attn=-0.5 壓力測試設定，在這麼長的提示上可能滲進 DSML 與工具呼叫的語法裡。

8.4Hermes AgentHermes Agent

The quickest route is interactive: run hermes model, choose “Custom endpoint (self-hosted / VLLM / etc.)”, and enter http://127.0.0.1:8000/v1. To keep it in the config instead:

最快的是互動式：執行 hermes model，選「Custom endpoint (self-hosted / VLLM / etc.)」，輸入 http://127.0.0.1:8000/v1。想寫進設定檔的話：

custom_providers:
  - name: ds4
    base_url: http://127.0.0.1:8000/v1
    # api_key omitted: the local server does not check itapi_key 省略：本機伺服器不檢查

model:
  default: deepseek-v4-flash
  provider: custom:ds4

Inside a session, /model custom:ds4:deepseek-v4-flash switches over at any time.

在對話裡，隨時可以用 /model custom:ds4:deepseek-v4-flash 切過去。

8.5Straight to the API: the OpenAI SDK, curl, your own scripts直接打 API：OpenAI SDK、curl、自己的腳本

Everything above is a wrapper around the same two facts: a base URL, and a key that is never checked. If what you have is a pipeline, a cron job, or code already written against the OpenAI SDK, set OPENAI_BASE_URL — or pass base_url — and nothing else changes:

上面每一節，包的都是同樣兩件事：一個 base URL，加一把從不檢查的金鑰。如果你手上是一條 pipeline、一個 cron job，或已經寫好、對著 OpenAI SDK 的程式，設好 OPENAI_BASE_URL──或直接傳 base_url──其他什麼都不必改：

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="sk-local",   # any value任意值
)

resp = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "自我介紹"}],
)
print(resp.choices[0].message.content)

If your pipeline already speaks the Responses API — the one Codex CLI uses — client.responses.create() reaches the same server:

若你的 pipeline 已經改說 Responses API──也就是 Codex CLI 用的那套──client.responses.create() 打到的是同一台伺服器：

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="sk-local",
)

resp = client.responses.create(
    model="deepseek-v4-flash",
    input=[{"role": "user", "content": "自我介紹"}],
)
print(resp.output_text)

8.6When does ds4-server run?ds4-server 什麼時候在跑？

The lease and watchdog of Chapter 7 only see pi. They start the server when a pi process sends a request through the model, and they stop it about a minute after the last pi process leaves. If you never open pi — Codex CLI all day, say — the server has to come from somewhere else. There are three ways, and they differ mainly in who is responsible for the process.

第七章的租約與看門狗只看得見 pi。它們在 pi 程序透過模型送出請求時啟動伺服器，在最後一個 pi 程序離開約一分鐘後把它停掉。如果你根本不開 pi──比如整天都用 Codex CLI──伺服器就得從別的地方來。有三條路，差別主要在誰為這個程序負責。

A. pi is installed; let pi keep it aliveA. 裝了 pi，讓 pi 替你顧著

Keep one pi session open with the ds4 model selected once. Its lease renews every ten seconds, the watchdog sees a living client, and the server stays up for as long as that window does. Nothing to configure; the cost is one terminal tab.

開著一個 pi 視窗，選一次 ds4 模型就好。它的租約每十秒續一次，看門狗看到有活著的客戶端，伺服器就跟那個視窗一樣久。不用設定任何東西；代價是一個終端機分頁。

B. pi is installed; run the server by handB. 裝了 pi，自己手動跑伺服器

The checkout under ~/.pi/ds4/support already has a built binary and the weights. This is the managed launch, typed out:

~/.pi/ds4/support 底下的 checkout 已經有編好的執行檔與權重。這就是代管啟動，逐字打出來：

cd ~/.pi/ds4/support
./ds4-server \
  --ctx 100000 \
  --kv-disk-dir ~/.pi/ds4/kv \
  --kv-disk-space-mb 65536 \
  --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf
# 65536 is the 128 GB+ tier; use 32768 on 96–127 GB.65536 是 128 GB 以上的級距；96–127 GB 請用 32768。
# Drop --vision for a text-only server. Engine-level flags from the只要純文字就拿掉 --vision。更新誌裡的引擎級旗標
# What's-new list (--ssd-streaming, --mtp DSpark.gguf, --max-queue N) go here.（--ssd-streaming、--mtp DSpark.gguf、--max-queue N）就加在這裡。

To opt into uncertainty steering on a hand-started server, launch it with the four flags the managed path would add:

要在手動啟動的伺服器上開啟不確定性引導，帶上代管路徑會加的那四個旗標：

cd ~/.pi/ds4/support
./ds4-server \
  --ctx 100000 \
  --kv-disk-dir ~/.pi/ds4/kv \
  --kv-disk-space-mb 65536 \
  --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf \
  --dir-steering-file dir-steering/out/uncertainty_ablit_imatrix.f32 \
  --dir-steering-ffn -0.75 \
  --dir-steering-attn 0 \
  --dir-steering-policy final-answer

pi adopts a server it finds already listening on 127.0.0.1:8000, and an adopted server has exactly the flags you gave it: pi re-applies neither --ctx nor steering. That is the point, not a gap. A launch you typed is a launch you can audit. It also means that if you want the vision encoder, the KV disk cache or steering on a hand-started server, you type them.

pi 會「收養」它發現已經在 127.0.0.1:8000 監聽的伺服器，而收養來的伺服器，旗標就只有你給它的那些：pi 不會替它補上 --ctx，也不會補上引導。這是重點，不是缺漏。你親手打出來的啟動，就是你稽核得了的啟動。這也表示，若你要手動啟動的伺服器有視覺編碼器、KV 磁碟快取或引導，就得自己打上去。

C. No pi at allC. 完全沒有 pi

Bootstrap from the two repositories directly; no pi, no ~/.pi/ds4/. It takes both because the download_model.sh inside ds4 fetches antirez’s upstream stock recipe, while the same-named script inside pi-ds4 fetches the Vision-Exp pair this letter is about.

直接從兩個 repo 開始；不需要 pi，也不需要 ~/.pi/ds4/。之所以要兩個，是因為 ds4 裡的 download_model.sh 抓的是 antirez 上游的原版配方，而 pi-ds4 裡同名的腳本，抓的才是這封信談的 Vision-Exp 那一對檔案。

# 1. Fetch and build the engine (Mac: Metal; Linux: CUDA, see §8.7)1. 取得並編譯引擎（Mac：Metal；Linux：CUDA，見 §8.7）
git clone https://github.com/audreyt/ds4
cd ds4
make ds4-server

# 2. The Vision-Exp download script from pi-ds4, not ds4's own stock-recipe one2. 用 pi-ds4 的 Vision-Exp 下載腳本，不是 ds4 自帶的原版配方那支
curl -fL -o download_model.sh \
  https://raw.githubusercontent.com/audreyt/pi-ds4/main/download_model.sh
chmod +x download_model.sh
./download_model.sh q2
# fetches the ~81 GiB language GGUF plus the encoder and links ds4flash.gguf抓約 81 GiB 的語言 GGUF 與編碼器，並連結為 ds4flash.gguf

# 3. Start the server (append the four steering flags from path B to opt in)3. 啟動伺服器（要開引導，把 B 路線的四個旗標接在後面）
./ds4-server \
  --ctx 100000 \
  --kv-disk-dir ./kv \
  --kv-disk-space-mb 65536 \
  --vision gguf/DeepSeek-V4-Flash-Vision-Encoder.gguf

On this path you never touch pi; ds4-server comes straight from the source tree. Whatever the extension would have done for you — download, symlink, watchdog, lease — is yours to do when you need it.

走這條路你完全不碰 pi；ds4-server 直接來自原始碼樹。擴充套件本來會替你做的事──下載、symlink、看門狗、租約──需要時都由你自己來。

8.7Other hardware: DGX Spark (CUDA) and Strix Halo (ROCm)其他硬體：DGX Spark（CUDA）與 Strix Halo（ROCm）

You do not need a Mac. On anything that is not Darwin, audreyt/ds4’s Makefile takes the CUDA path — ds4_cuda.cu, some ten thousand lines of NVIDIA kernels — and nvcc builds a native ds4-server. On an NVIDIA DGX Spark (GB10, about 128 GB of unified memory, aarch64 Linux) what runs is therefore not a llama.cpp sidecar but the same engine, the same server and the same --dir-steering-* flags. pi install https://github.com/audreyt/pi-ds4 works there unchanged: clone, make ds4-server with nvcc and CUDA selected automatically, download the Vision-Exp pair, spawn the server.

你不需要 Mac。在任何不是 Darwin 的系統上，audreyt/ds4 的 Makefile 會走 CUDA 路徑──ds4_cuda.cu，約一萬行的 NVIDIA kernel──由 nvcc 編出原生的 ds4-server。所以在 NVIDIA DGX Spark（GB10、約 128 GB 統一記憶體、aarch64 Linux）上跑的，不是某個 llama.cpp 的旁路，而是同一隻引擎、同一台伺服器、同一組 --dir-steering-* 旗標。pi install https://github.com/audreyt/pi-ds4 在那裡一字不改照樣能用：clone、make ds4-server（自動選 nvcc 與 CUDA）、下載 Vision-Exp 那一對檔案、啟動伺服器。

Since the 2026-07 pin there is a third backend, AMD Strix Halo on ROCm. A 128 GB Ryzen AI Max machine (Radeon 8060S, gfx1151) builds the same binaries with make strix-halo, alias make rocm; the STRIXHALO.md in the checkout walks through ROCm 7.1 and rocWMMA, /dev/kfd permissions and the GTT aperture. One caveat: on that platform prefer the plain aligned-imatrix GGUF and avoid the mixed IQ2/Q4 builds for now — they put far more pressure on the ROCm path and can end in a system OOM rather than a clean ds4 error. The same pin brought distributed inference: the 4-bit Flash across two 128 GB MacBooks over Thunderbolt 5, each mapping only its own slice of layers. Pipelined prefill can beat one machine; generation is slower.

自 2026-07 釘版起有了第三個後端：ROCm 上的 AMD Strix Halo。128 GB 的 Ryzen AI Max 機器（Radeon 8060S，gfx1151）用 make strix-halo（別名 make rocm）編出同一組執行檔；checkout 裡的 STRIXHALO.md 逐步交代 ROCm 7.1 與 rocWMMA、/dev/kfd 權限與 GTT aperture。一個注意事項：在那個平台上，暫時請用純粹的 aligned-imatrix GGUF，避開 IQ2/Q4 混合版──後者在 ROCm 路徑上的記憶體壓力大得多，結局可能是系統層的 OOM，而不是 ds4 乾淨的錯誤訊息。同一份釘版也帶來了分散式推論：4-bit 的 Flash 跨兩台 128 GB MacBook，走 Thunderbolt 5，每台只 map 自己那一段 layer。管線化的 prefill 可以贏過單機；生成則比較慢。

Once the endpoint is up, the recipes in §8.1–8.5 apply as written; replace 127.0.0.1 with the machine’s address if you connect from elsewhere. Four things change:

端點起來之後，§8.1–8.5 的配方原樣適用；從別台機器連過來的話，把 127.0.0.1 換成那台機器的位址。有四件事不同：

• Opening it to a LAN is your responsibility. ds4-server binds 127.0.0.1 and checks no key. For use across machines, put it behind Tailscale or a reverse proxy with authentication; do not flip --host to 0.0.0.0 and expose an unauthenticated model to the network.
• The watchdog and leases work on Linux. ds4-watchdog.sh handles both BSD and GNU userlands — stat -f %m against stat -c %Y, LC_ALL=C ps -o lstart — so Chapter 7’s lifecycle covers a Spark directly. A hand-run ./ds4-server, a systemd unit or a tmux session remain fine if you would rather skip the automation.
• The backend is different. A Spark runs CUDA on GB10 tensor cores, not Metal. pi-ds4 supplies the same GGUF, steering arguments, context and KV defaults and lifecycle wrapper; backend behaviour follows ds4’s Linux build.
• Long context on a Spark is not covered here. pi-ds4 sends --ctx 100000 on Mac and Linux alike, but KV-cache and long-context behaviour on the Spark is outside this letter’s testing. Under memory pressure, run ds4-server by hand and start from --ctx 32768.

• 開放到區網是你自己的責任。ds4-server 綁在 127.0.0.1，不檢查任何金鑰。要跨機器使用，請放在 Tailscale 或帶驗證的 reverse proxy 後面；不要把 --host 改成 0.0.0.0，把一個沒有驗證的模型暴露在網路上。
• 看門狗與租約在 Linux 上也能跑。ds4-watchdog.sh 同時處理 BSD 與 GNU 兩套工具──stat -f %m 對應 stat -c %Y、LC_ALL=C ps -o lstart──所以第七章的生命週期直接覆蓋 Spark。若你寧願跳過自動化，手動 ./ds4-server、一個 systemd unit 或一個 tmux 視窗都仍然可行。
• 後端不一樣。Spark 跑的是 GB10 tensor core 上的 CUDA，不是 Metal。pi-ds4 提供的是同一份 GGUF、引導參數、context 與 KV 預設值，以及生命週期的包裝；後端行為跟著 ds4 的 Linux 建置走。
• Spark 上的長 context 這裡沒有涵蓋。pi-ds4 在 Mac 與 Linux 上都送 --ctx 100000，但 Spark 上的 KV cache 與長 context 行為不在這封信的測試範圍內。遇到記憶體壓力，手動跑 ds4-server，從 --ctx 32768 開始。

In other words this is not a strained “same GGUF, different runtime” compatibility. It is the same engine, the same startup shape and the same wrapper defaults across platforms — macOS and Linux share one ds4-watchdog.sh, pi install completes on a DGX Spark the way it does on a Mac, and running the server by hand stays open to you on both.

換句話說，這不是勉強湊出來的「同一份 GGUF、不同執行期」相容性，而是跨平台的同一隻引擎、同一種啟動形狀、同一組包裝預設值──macOS 與 Linux 共用同一支 ds4-watchdog.sh，pi install 在 DGX Spark 上完成的方式與在 Mac 上一樣，而手動跑伺服器這條路在兩邊都始終開著。

Troubleshooting: leases and the watchdog故障排除：租約與看門狗

The extension keeps every piece of runtime state as a file under ~/.pi/ds4/. That is a design choice, not an accident: when something feels wrong, you read the files, and what the extension believes is happening is never hidden from you. You should not need the source open to find out.

擴充套件把每一份執行期狀態都以檔案的形式放在 ~/.pi/ds4/ 底下。這是設計上的選擇，不是偶然：覺得哪裡不對的時候，你讀檔案就好，擴充套件以為正在發生的事，從來不會對你隱藏。你不應該需要打開原始碼才弄得清楚。

9.1What lives where什麼東西放在哪裡

9.2Symptoms, and what to do about them症狀，以及怎麼處理

“The server has been starting for more than ten minutes”「伺服器啟動中已經超過十分鐘」

Read the last few hundred lines of log. Loading the model from an internal SSD takes seconds, so ten minutes is rarely disk speed itself. More often the GGUF sits on a slow external drive, or some other step of startup is wedged. Diagnose from the log first; raise DS4_READY_TIMEOUT_MS only when there is a real reason to wait longer.

讀 log 的最後幾百行。從內建 SSD 載入模型只要幾秒，所以十分鐘很少是磁碟速度本身的問題。比較常見的是 GGUF 放在慢速的外接磁碟上，或啟動流程的某個環節卡住了。先從 log 找原因；只有真的有理由等更久時，才把 DS4_READY_TIMEOUT_MS 調高。

“pi says ds4-server startup failed”「pi 說 ds4-server startup failed」

The log names the failure. It is usually one of three: make ds4-server failed (missing Xcode command-line tools?), the GGUF download broke (network? disk full?), or a ds4-server flag you added by hand was rejected by the current build.

log 會指出失敗點。通常是三種之一：make ds4-server 失敗（少了 Xcode 命令列工具？）、GGUF 下載中斷（網路？磁碟滿了？），或你手動加的某個 ds4-server 旗標被目前的建置拒絕。

“/ds4-agent says the server is in use”「/ds4-agent 說伺服器正在使用中」

A guard, not a fault. The native agent has to load the same 87 GB model itself, and if another pi window or an HTTP client is still using ds4-server, the extension refuses rather than cut someone else off mid-request. Wait for those requests to finish, close the other pi windows, or clear stale leases with the gentle restart below, and try again.

這是保護，不是故障。原生 agent 得自己載入同一份 87 GB 的模型；若還有別的 pi 視窗或 HTTP 客戶端在用 ds4-server，擴充套件寧可拒絕，也不在請求進行到一半時把別人切斷。等那些請求結束、關掉其他 pi 視窗，或用下面的溫和重啟清掉過期的租約，再試一次。

“I want to restart everything”「我想把一切重啟」

Try the gentle way first. Empty ~/.pi/ds4/clients/; on its next poll, about two seconds later, the watchdog sees no leases and shuts the server down cleanly.

先試溫和的方式。把 ~/.pi/ds4/clients/ 清空；看門狗在下一輪巡查（約兩秒後）看到沒有租約，就會乾淨地把伺服器關掉。

# Clear every lease; the watchdog stops the server on its next poll.清掉所有租約；看門狗下一輪巡查就會停掉伺服器。
# find -delete rather than rm with a glob, so zsh stays quiet on an empty directory.用 find -delete 而不是 rm 加萬用字元，空目錄時 zsh 才不會報錯。
find ~/.pi/ds4/clients -maxdepth 1 -name '*.json' -delete 2>/dev/null || true

# Print the key fields of server.json and the matching process's arguments and start time:印出 server.json 的關鍵欄位，以及對應程序的參數與啟動時間：
STATE=~/.pi/ds4/server.json
if [ ! -f "$STATE" ]; then
  echo 'no server.json (already clean)'
else
  MANAGED=$(sed -n 's/.*"managedBy"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$STATE" | head -1)
  PID=$(sed -n 's/.*"pid"[[:space:]]*:[[:space:]]*\([0-9]*\).*/\1/p' "$STATE" | head -1)
  BINARY=$(sed -n 's/.*"binary"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$STATE" | head -1)
  echo "managedBy: $MANAGED"
  echo "pid:       ${PID:-<none>}"
  echo "binary:    ${BINARY:-<none>}"
  if [ -n "$PID" ] && kill -0 "$PID" 2>/dev/null; then
    ps -p "$PID" -o pid=,lstart=,args=
  else
    echo '(pid not running — state is stale, step 2 will clean it)'
  fi
fi

(
  STATE=~/.pi/ds4/server.json
  LOCKDIR=~/.pi/ds4/lock

  # mkdir as an atomic lock, the same mechanism index.ts uses; exit if it is held.用 mkdir 做原子鎖，與 index.ts 同一套機制；鎖被持有就退出。
  if ! mkdir "$LOCKDIR" 2>/dev/null; then
    echo "abort: lock $LOCKDIR is held; owner:"
    cat "$LOCKDIR/owner.json" 2>/dev/null || echo '(no owner.json — if > 60s old, rm -rf manually)'
    exit 1
  fi
  trap 'rm -rf "$LOCKDIR" 2>/dev/null' EXIT
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 130' INT
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 143' TERM
  trap 'rm -rf "$LOCKDIR" 2>/dev/null; exit 129' HUP

  if [ ! -f "$STATE" ]; then
    echo 'lifecycle already clean'; exit 0
  fi

  PID=$(sed -n 's/.*"pid"[[:space:]]*:[[:space:]]*\([0-9]*\).*/\1/p' "$STATE" | head -1)

  # Only touch state when the PID is gone, or absent from the state file.只有 PID 不存在（或狀態檔裡沒有 PID）時才動狀態。
  # If the PID is alive, refuse and hand off to step 3.PID 還活著就拒絕，交給第三步。
  if [ -n "$PID" ] && kill -0 "$PID" 2>/dev/null; then
    echo "refuse: pid $PID still alive."
    echo "        please verify via step 1 output, then use step 3 to kill manually."
    exit 2
  fi

  # The PID is dead or absent. Safe to clean.PID 已死或不存在。可以安全清理。
  find ~/.pi/ds4/clients -maxdepth 1 -name '*.json' -delete 2>/dev/null || true
  rm -f "$STATE"
  echo 'state cleared. lock will release on exit.'
)

# Replace PID_FROM_STEP_1_MUST_BE_REPLACED with the PID you confirmed by eye in step 1.把 PID_FROM_STEP_1_MUST_BE_REPLACED 換成你在第一步親眼確認過的那個 PID。
# The token is deliberately not a number: run unedited, this line only produces這個記號刻意不是數字：沒改就執行，這一行只會得到
# 'kill: arguments must be process or job IDs' and never sends a signal.'kill: arguments must be process or job IDs'，不會送出任何訊號。
kill -TERM PID_FROM_STEP_1_MUST_BE_REPLACED

# Give it a few seconds to exit cleanly, then run step 2 again to clear the state.給它幾秒鐘乾淨地退出，然後再跑一次第二步清掉狀態。

“The server is running but /v1/models does not answer”「伺服器在跑，但 /v1/models 沒有回應」

Check who really holds port 8000: lsof -nP -iTCP:8000 -sTCP:LISTEN. If something else has taken it, the server cannot start, and the log will say so.

查查 8000 port 到底在誰手上：lsof -nP -iTCP:8000 -sTCP:LISTEN。若被別的程式佔走，伺服器就起不來，log 會記下這件事。

Local development install本機開發安裝

If you want to work on the engine, test a patch of your own, or keep several ds4 forks side by side, skip pi install. The bundled script symlinks both the extension and a ds4 checkout of your choosing into place, so the code pi runs is the code sitting in your editor.

如果你想動引擎、測自己的修補，或並排保留好幾個 ds4 分叉，就跳過 pi install。內附的腳本會把擴充套件與你指定的 ds4 checkout 都以 symlink 掛到位，於是 pi 跑的程式碼，就是你編輯器裡打開的那一份。

10.1Mounting a checkout掛上一個 checkout

./install-pi-extension-local.sh /path/to/audreyt-ds4-checkout

It does two things: links ~/.pi/agent/extensions/pi-ds4 to the current pi-ds4 checkout, and links ~/.pi/ds4/support to the ds4 checkout you named.

它做兩件事：把 ~/.pi/agent/extensions/pi-ds4 連到目前的 pi-ds4 checkout，再把 ~/.pi/ds4/support 連到你指定的 ds4 checkout。

10.2When a support directory already exists: --forcesupport 目錄已經存在時：--force

If ~/.pi/ds4/support already points somewhere — left over from an earlier pi install, say — the script refuses to overwrite it. With --force it:

若 ~/.pi/ds4/support 已經指向別處──比如上一次 pi install留下的──腳本會拒絕覆寫。加上 --force，它會：

• copies gguf/*.gguf and any .gguf.part from the old checkout into the new one with APFS clone-on-write — on macOS no 87 GB actually moves, the two paths share blocks;
• renames the old support directory to support.backup.<timestamp>;
• creates the new symlink.

• 用 APFS 的 clone-on-write，把舊 checkout 裡的 gguf/*.gguf 與 .gguf.part 複製到新的 checkout──在 macOS 上並沒有真的搬動 87 GB，兩條路徑共用同一批區塊；
• 把舊的 support 目錄改名為 support.backup.<timestamp>；
• 建立新的 symlink。

10.3Then reload然後重新載入

Restart pi, or run /reload inside it, and the extension is discovered again from its new location.

重新啟動 pi，或在裡面執行 /reload，擴充套件就會從新的位置被重新發現。

Building your own steering direction自己建一個引導方向

This chapter assumes everything above, and asks a different question. The bundled vector expresses one preference — that a contested question should open into a discussion. Yours may be different: a register, a discipline, a way of hedging. The toolchain that built the bundled direction is in the checkout, and it will build yours.

本章假設你已經讀完前面所有內容，然後問一個不一樣的問題。內附的向量表達的是一種偏好──有爭議的問題應該打開成討論。你的偏好可能不同：某種語體、某個學科、某種保留餘地的方式。建出內附方向的那套工具鏈就在 checkout 裡，它也能建出你的。

11.1Where the toolchain lives工具鏈在哪裡

Everything sits in dir-steering/ inside audreyt/ds4:

全部都在 audreyt/ds4 的 dir-steering/ 目錄裡：

• collect-acts.py runs forward passes over a set of contrast prompt pairs and records the hidden state at every layer;
• build-dir.py turns those activations into a direction by PCA or differencing, and writes a .f32 vector file;
• README.md is the full tutorial, with the design discussion behind the bundled vector.

• collect-acts.py 對一組成對的對比提示做前向計算，記錄每一層的隱藏狀態；
• build-dir.py 用 PCA 或差分把這些活化值變成一個方向，寫出 .f32 向量檔；
• README.md 是完整教學，附有內附向量背後的設計討論。

11.2A typical workflow典型的工作流程

1. Write two sets of prompts: one that exhibits what you want to steer towards, one that exhibits what you want to steer away from. Fifty to two hundred of each is enough.
2. Run both sets through ds4-server with collect-acts.py to produce two sets of activations.
3. Let build-dir.py compute the layer-by-layer difference and write my-direction.f32.
4. Put the file under dir-steering/out/, or anywhere you like.
5. Point the extension at it:
   • DS4_DIR_STEERING_FILE=dir-steering/out/my-direction.f32
   • DS4_DIR_STEERING_FFN=-0.75, or whatever magnitude your own testing settles on
   • DS4_DIR_STEERING_ATTN=0 — keep it there whenever tools are in play
   • DS4_DIR_STEERING_POLICY=final-answer — likewise
6. Restart ds4-server and test, first with the prompts you built from, then with prompts you did not.

1. 寫兩組提示：一組展現你想引導模型走向的樣子，一組展現你想讓它離開的樣子。每組五十到兩百條就夠。
2. 用 collect-acts.py 把兩組提示都丟進 ds4-server，得到兩組活化值。
3. 讓 build-dir.py 逐層算出差異方向，寫出 my-direction.f32。
4. 把檔案放到 dir-steering/out/，或任何你喜歡的位置。
5. 讓擴充套件指向它：
   • DS4_DIR_STEERING_FILE=dir-steering/out/my-direction.f32
   • DS4_DIR_STEERING_FFN=-0.75，或你自己測出來的強度
   • DS4_DIR_STEERING_ATTN=0──只要有工具參與就保持這個值
   • DS4_DIR_STEERING_POLICY=final-answer──同理
6. 重新啟動 ds4-server 測試：先用建向量時的那些提示，再用你沒用過的。

11.3Two principles for a direction worth keeping值得留下的方向，兩個原則

• Keep the contrast clean. The two sets should differ sharply along the one dimension you care about and resemble each other everywhere else. Build from “happy reply versus sad reply” and the direction you get carries a great deal more than mood.
• Steer register, not belief. Steering how the model answers — hedged or assertive, narrative or bulleted, academic or conversational — is far more reliable than steering what it concludes. That is why this fork built an uncertainty direction rather than a position, and it is why §4.3 exists.

• 對比要乾淨。兩組提示應該在你在意的那一個維度上差異鮮明，其他地方則盡量相似。用「快樂的回答對悲傷的回答」去建，得到的方向裡混進的東西，遠不止情緒。
• 引導語體，不引導信念。引導模型怎麼回答──保留餘地還是斬釘截鐵、敘事還是條列、學術還是口語──遠比引導它得出什麼結論可靠。這就是這個分叉建的是不確定性方向而不是某個立場的原因，也是 §4.3 存在的原因。

For readers still deciding給還在猶豫的讀者

If you are still weighing whether to give a machine of yours to this model, these are the questions that come up most.

如果你還在衡量要不要把自己的一台機器交給這顆模型，下面是最常被問到的問題。

Terms, cross-referenced名詞對照

The words that recur in this letter and may be new to a reader outside engineering. If they are old friends, skip ahead.

這封信裡反覆出現、對工程領域以外的讀者可能陌生的詞。若都是老朋友，往後跳吧。

Model and weights模型與權重

parameter參數

A trainable number inside a neural network, and a rough proxy for how much a model can hold. The DeepSeek V4 Flash in this letter has 284 billion of them.

神經網路裡可訓練的數字，也是模型「裝得下多少」的粗略指標。這封信裡的 DeepSeek V4 Flash 有 2,840 億個。

MoE混合專家

Mixture of Experts. The model is divided into expert sub-networks, and each token wakes only a few of them — here about 13 billion parameters per token, so it runs at the speed of a 13 B model while carrying the knowledge of a 284 B one.

Mixture of Experts。模型分成多個專家子網路，每個 token 只喚醒其中少數幾個──這裡每個 token 約啟用 130 億個參數，所以跑起來是 13 B 模型的速度，裝的卻是 284 B 的知識。

quantization量化

Compressing weights from their native precision into fewer bits. Vision-Exp IQ2 is not one bit-width: routed experts are mostly IQ2_XXS and Q2_K, attention, shared-expert and output paths stay at Q8_0, and selected indexer and control tensors remain F16, F32 or I32. The language GGUF comes to 86720111776 bytes, plus an 889 MiB encoder.

把權重從原生精度壓縮到更少的位元。Vision-Exp IQ2 不是單一位元寬度：routed expert 以 IQ2_XXS 與 Q2_K 為主，attention、shared expert 與 output 路徑保留 Q8_0，部分 indexer 與 control tensor 維持 F16、F32 或 I32。語言 GGUF 共 86720111776 位元組，另加 889 MiB 的編碼器。

GGUF模型檔格式

The single-file model format of the llama.cpp family (.gguf): weights, architecture and tokenizer in one file. The language GGUF here is 86720111776 bytes.

llama.cpp 家族使用的單檔模型格式（.gguf）：權重、結構、tokenizer 全在一個檔案裡。這裡的語言 GGUF 是 86720111776 位元組。

abliteration拒絕方向消除

Finding the internal direction that triggers “refuse to answer” and subtracting it from the forward pass. It loosens over-refusal; it is neither a jailbreak nor a safety guarantee (§3.2).

找出模型內部觸發「拒絕回答」的那個方向，從前向計算裡減掉。它鬆動過度拒絕；既不是越獄，也不是安全保證（§3.2）。

Inference and hardware推論與硬體

token詞元

The unit a model reads and writes in. A Chinese character is roughly one to three tokens; an English word, one or two.

模型讀寫的基本單位。一個漢字約一到三個 token；一個英文單字約一到兩個。

prefill / decode預填／逐字生成

The two stages of inference. Prefill takes in the whole prompt at once; decode produces the reply one token at a time. Decode is the one you feel in a long conversation, because it happens for every new token.

推論的兩個階段。Prefill 一口氣吃下整段提示；decode 一個 token 一個 token 地產生回覆。長對話裡你感覺到的是 decode，因為每個新 token 都要經過它。

context window上下文視窗

How many tokens the model can see at once. This extension asks for 100,000 (about a novella), but the abliterated weights are not validated beyond 32k; treat everything above that as experimental (§6.4).

模型一次看得見多少 token。本擴充套件要求 100,000（約一本中篇小說），但 abliterated 權重在 32k 以上沒有驗證過；超過的部分請當實驗看待（§6.4）。

KV cache鍵值快取

Intermediate state kept during inference so earlier tokens need not be recomputed. Long conversations accumulate a lot of it; this extension lets it spill to disk, RAM-tiered when DS4_KV_DISK_SPACE_MB is unset, about 64 GB on a 128 GB machine.

推論期間保留的中間狀態，讓先前的 token 不必重算。長對話會累積很多；本擴充套件允許它溢出到磁碟，DS4_KV_DISK_SPACE_MB 未設時依 RAM 分級，128 GB 機器約 64 GB。

unified memory統一記憶體

Apple Silicon gives CPU and GPU one shared pool of RAM, with no copying into a separate VRAM. It is the reason a Mac can hold a model this size at all.

Apple Silicon 讓 CPU 與 GPU 共用同一池 RAM，不必再複製到另一塊 VRAM。這是 Mac 裝得下這個尺寸模型的原因。

Metal 4 / Metal TensorApple 的 GPU 介面

Metal is Apple’s GPU programming interface; Metal 4 adds tensor APIs. pi-ds4 does not choose backend profiles in the wrapper; the ds4 build decides the actual path.

Metal 是 Apple 的 GPU 程式介面；Metal 4 加入了 tensor API。pi-ds4 不在包裝層選擇後端設定檔；實際路徑由 ds4 的建置決定。

SSD streamingexperts streamed from diskSSD 串流

The engine’s capacity mode (--ssd-streaming): non-routed weights stay resident, routed experts live in an in-memory cache and are read back from the GGUF on a miss. It lets a model larger than RAM run, at the cost of decode that feels every miss. The managed path does not pass this flag; see What’s new and §8.6.

引擎的容量模式（--ssd-streaming）：非路由的權重常駐記憶體，路由的專家放在記憶體快取裡，未命中時再從 SSD 上的 GGUF 讀回。它讓大於 RAM 的模型跑得起來，代價是 decode 對每次未命中都有感。代管路徑不傳這個旗標；見更新誌與 §8.6。

speculative decoding / DSpark推測解碼

A small draft head guesses a block of tokens; the full model verifies before committing. Enabled in ds4 with --mtp DSpark.gguf (experimental). This fork adds B2 rejection sampling so the output distribution is identical to plain decoding — lossless.

一個小的草稿頭一次猜一整段 token；完整模型驗證之後才採用。ds4 以 --mtp DSpark.gguf 啟用（實驗性）。這個分叉加上了 B2 rejection sampling，讓輸出分佈與不開推測時完全相同──無損。

Directional steering and this fork方向性引導與這個分叉

directional steering方向性引導

A low-rank edit to the activations at chosen layers while the model runs, pushing it towards or away from whatever trait a direction vector encodes. No retraining involved (Chapter 4).

模型執行時，在選定的層對活化值做低秩編輯，把它推向（或推離）某個方向向量所代表的特質。不需要重新訓練（第四章）。

FFN / attention前饋／注意力

The two main compute blocks inside a Transformer layer. This fork’s recommended opt-in steers only at the FFN output (ffn=-0.75) and leaves attention alone (attn=0), so tool-call syntax stays untouched.

Transformer 每一層裡的兩個主要計算區塊。這個分叉建議的開啟方式只在 FFN 輸出端引導（ffn=-0.75），不動 attention（attn=0），讓工具呼叫的語法不受影響。

uncertainty direction不確定性方向

The vector bundled with this fork, built from 120 contested prompts balanced between English and Traditional Chinese, with Taiwan and Hong Kong deliberately left out, contrasting fair stakeholder framing with single-answer framing.

這個分叉內附的向量，用 120 條英文與繁體中文各半、刻意不含台灣與香港的爭議提示建成，對比「公正呈現利害關係人」與「直接給單一答案」兩種框架。

response register回應語體

The manner of a reply rather than its content: hedged or assertive, bulleted or narrative, academic or conversational. Far more amenable to steering than stance.

回答的方式而不是內容：保留餘地還是斬釘截鐵、條列還是敘事、學術還是口語。比立場好引導得多。

System and lifecycle系統與生命週期

ds4-serverHTTP 推論伺服器

The background inference server pi and other HTTP clients talk to. Listens on 127.0.0.1:8000; its lifetime is governed by leases and the watchdog.

pi 與其他 HTTP 客戶端對話的背景推論伺服器。監聽 127.0.0.1:8000；壽命由租約與看門狗決定。

ds4-agent原生終端機 agent

The native terminal agent inside the ds4 checkout. Not a server: it owns the terminal, its session and KV state, and its own DSML tool loop. pi-ds4 reaches it through the foreground handoff /ds4-agent.

ds4 checkout 內建的原生終端機 agent。它不是伺服器：終端機、對話與 KV 狀態、DSML 工具迴圈都由它自己掌握。pi-ds4 透過前景交接 /ds4-agent 進入它。

lease租約

The proof-of-presence file each pi process using ds4 writes to ~/.pi/ds4/clients/<pid>.json, refreshed every ten seconds.

每個使用 ds4 的 pi 程序寫在 ~/.pi/ds4/clients/<pid>.json 的存在證明，每十秒更新一次。

watchdog看門狗

A small resident shell script that checks the leases every two seconds and shuts ds4-server down once none is valid.

一支常駐的小 shell script，每兩秒檢查租約，沒有任何有效租約時就把 ds4-server 關掉。

system prompt系統提示

The role and rules given to the model before the conversation starts. The uncertainty steering in this fork works with a prompt that asks for stakeholder framing (§4.5); neither replaces the other.

對話開始前給模型的角色與規則。這個分叉的不確定性引導，要搭配一段要求呈現利害關係人視角的提示才起作用（§4.5）；兩者誰也不能取代誰。

/health · /stats狀態端點

Read-only endpoints on ds4-server since the 2026-07 pin. /health answers liveness and uptime; /stats answers queue depth, per-source cache hits and the last prefill and decode speeds. Both reply even mid-generation (§7.1).

2026-07 釘版起 ds4-server 的唯讀端點。/health 回存活狀態與執行時間；/stats 回佇列深度、各來源的快取命中數，以及最近一次 prefill 與 decode 的速度。生成到一半也答得動（§7.1）。

Thanks, licence, afterword致謝、授權、後話

Thanks致謝

• mitsuhiko/pi-ds4, the upstream this extension is forked from. Armin Ronacher wrote the whole lifecycle — the leases, the watchdog, the lock — and wrote it so that it could be read.
• antirez/ds4, Salvatore Sanfilippo’s DeepSeek V4 Flash engine in plain C, in the same tradition as Redis. The GGUF conversion comes from its sibling, llama.cpp-deepseek-v4-flash.
• The ds4 runtime and backend contributors. SSD streaming, distributed inference and the ROCm integration were led by antirez; elkaix (server observability and the agent-loop cache, PR #489), MA (DSpark B2 rejection sampling), Nick Parrin (Strix Halo), Andrea Borio (mixed-quant expert streaming), rinaldofesta (the eval grader), kamranjon and fry69 (agent fixes), Andreas Spannagel (the MTP verify fix), ivanfioravanti (M5 Metal 4 / MPP). pi-ds4 stands on that runtime rather than carrying a tuning story of its own.
• The cyberneurova research project, whose abliterated GGUF first made running this model at home a real option.

• mitsuhiko/pi-ds4，本擴充套件分叉自的上游。Armin Ronacher 寫了整套生命週期──租約、看門狗、鎖──而且寫得讓人讀得懂。
• antirez/ds4，Salvatore Sanfilippo 用純 C 寫的 DeepSeek V4 Flash 引擎，延續他寫 Redis 的傳統。GGUF 的轉換來自它的姊妹專案 llama.cpp-deepseek-v4-flash。
• ds4 執行期與後端的貢獻者。SSD 串流、分散式推論與 ROCm 整合由 antirez 主導；elkaix（伺服器可觀測性與 agent 迴圈快取，PR #489）、MA（DSpark B2 rejection sampling）、Nick Parrin（Strix Halo）、Andrea Borio（混合量化的專家串流）、rinaldofesta（評測評分器）、kamranjon 與 fry69（agent 修正）、Andreas Spannagel（MTP 驗證修正）、ivanfioravanti（M5 Metal 4／MPP）。pi-ds4 站在這個執行期之上，不另外經營一套調校的說法。
• cyberneurova 研究專案，他們發布的 abliterated GGUF，第一次讓在家裡跑這顆模型成為真實的選項。

Licence授權

The source code is MIT, matching upstream; see the project’s LICENSE. The text of this letter is dedicated to the public domain under CC0.

原始碼採 MIT，與上游一致；見專案的 LICENSE。這封信的文字以 CC0 貢獻於公眾領域。

Further reading延伸閱讀

For the ethical and political frame around this project, read Software Freedom as Civic Care (華文). It places DS4 on free software’s repair path: the question is never whether a cloud vendor means well, but whether users and communities own the inference loop — can inspect it, fork it, roll it back — and can bring AI into the loops their communities already run.

想看這個專案周圍的倫理與政治框架，請讀〈軟體自由與公民關懷〉（English）。那篇文章把 DS4 放進自由軟體的修復路徑：問題從來不是雲端供應商是否善意，而是使用者與社群是否擁有推論的迴圈──能檢視、能分叉、能回滾──並且能把 AI 帶進社群本來就在運轉的循環裡。

Afterword後話

Running a frontier model on your own machine is, technically, the relocation of some computation. Politically, it is taking a steering wheel back into your own hands. This fork ships uncertainty steering as something you can switch on — not to decide any question for you, but to give back the space for discussion in front of the questions that training has sealed shut. And it ships it as a choice rather than a default for the same reason the managed launch passes nothing you did not ask for: a nudge you chose is a nudge you can name, measure and remove.

把前沿模型放到自己的機器上跑，技術上，是把一段運算搬了個地方；政治上，是把一個方向盤接回自己手中。這個分叉把不確定性引導做成一個你可以打開的東西──不是替你決定任何問題的答案，而是在那些被訓練封死的提問面前，把討論的空間還給你。而它做成選項、不做成預設，理由和代管啟動不傳任何你沒要求的參數一樣：你自己選的推力，才是你叫得出名字、量得出大小、也拿得掉的推力。

A model can close a question on an answer; the user can open it back into a discussion. The moment this guide finishes its work is the moment you no longer need it.

模型可以封閉答案，使用者可以打開討論。這份指南完成它任務的時刻，是當你已經不需要它的時候。