助手本体:会话引擎、LLM 路由、记忆、身份、语音适配

从 Obsidian 笔记库剥离为独立仓库。规划文档与 P0 报告留在笔记库,
代码(助手本体、TTS 服务、P0 基准脚本、ESP32 固件)在此。

架构:服务器是助手,本地是客户端
- 助手本体跑在 cadadmin-server,N5095 只做 USB 外设桥与网络桥接
- 起因:实测 N5095 无 AVX2,跑不动可用的语音模型

当前技术栈
- LLM:DeepSeek deepseek-flash(云端,必须关思考)
- TTS:Qwen3-TTS 1.7B(服务器 3090,独立 HTTP 服务)
- STT:Zipformer 中英双语 int8(CPU,中文识别率未测)

实测:端到端 16/16 通过,首音延迟 1545ms
(对比:本地 LLM + vits-zh-ll 是 179ms,但音色不满意——音质与延迟
在此硬件上此消彼长)

设计要点
- MemoryStore 接口按"可能远程"设计(全 async、可超时、可失败)
- user_id / session_id / track_id 三者分开,人脸只作候选不自动切身份
- 记忆查询强制带 owner_user_id,无按 id 单独取的口子
- 自动抽取的记忆是"候选",默认不参与检索
- 异步写记忆绑定创建时的身份,不读"当前用户"
- 引擎与传输解耦,只用文件音频即可端到端测试

模型与音频产物不进版本管理(1.1GB),由 download_models.sh 恢复
This commit is contained in:
ylhan
2026-09-15 11:05:09 +08:00
commit 35e107c290
46 changed files with 5976 additions and 0 deletions
+94
View File
@@ -0,0 +1,94 @@
// ESP32-S3 + ILI9341 240x320 display test.
// Install Adafruit ILI9341, Adafruit GFX Library and their dependencies.
// Display: SCK=12, SDI/MOSI=11, CS=10, DC=9, RESET=8, GND=GND.
// Touch: T_CLK=12, T_DIN=11, T_DO=13, T_CS=7, T_IRQ=6.
// Install XPT2046_Touchscreen by Paul Stoffregen as well.
// Display SDO and SD card are not connected. Backlight is externally powered.
#include <SPI.h>
#include <Adafruit_GFX.h>
#include <Adafruit_ILI9341.h>
#include <XPT2046_Touchscreen.h>
constexpr int TFT_SCK = 12;
constexpr int TFT_MOSI = 11;
constexpr int TFT_CS = 10;
constexpr int TFT_DC = 9;
constexpr int TFT_RST = 8;
constexpr int TOUCH_MISO = 13;
constexpr int TOUCH_CS = 7;
constexpr int TOUCH_IRQ = 6;
Adafruit_ILI9341 tft(&SPI, TFT_DC, TFT_CS, TFT_RST);
// Poll first to test SPI independently of the IRQ wire.
XPT2046_Touchscreen touch(TOUCH_CS);
bool wasPressed = false;
bool alternateColor = false;
uint32_t touchCount = 0;
uint32_t lastReport = 0;
void drawFace(uint16_t color);
void setup() {
Serial.begin(115200);
delay(300);
pinMode(TFT_CS, OUTPUT);
digitalWrite(TFT_CS, HIGH);
pinMode(TOUCH_CS, OUTPUT);
digitalWrite(TOUCH_CS, HIGH);
pinMode(TOUCH_IRQ, INPUT_PULLUP);
SPI.begin(TFT_SCK, TOUCH_MISO, TFT_MOSI, TFT_CS);
tft.begin(10000000); // Start conservatively at 10 MHz for jumper wires.
tft.setRotation(1); // Landscape: 320 x 240. Use 3 if upside down.
touch.begin(SPI);
touch.setRotation(1);
const uint16_t colors[] = {ILI9341_RED, ILI9341_GREEN, ILI9341_BLUE};
for (uint16_t color : colors) {
tft.fillScreen(color);
delay(400);
}
tft.fillScreen(ILI9341_NAVY);
drawFace(ILI9341_YELLOW);
Serial.println("Touch test ready. Raw coordinates are not calibrated pixels.");
}
void drawFace(uint16_t color) {
const int cx = tft.width() / 2;
const int cy = 105;
tft.fillCircle(cx, cy, 82, color);
tft.fillCircle(cx - 28, cy - 22, 10, ILI9341_BLACK);
tft.fillCircle(cx + 28, cy - 22, 10, ILI9341_BLACK);
// Parabolic smile: the center is lower than the corners.
for (int x = -42; x <= 42; ++x) {
int y = cy + 43 - (x * x) / 65;
tft.fillCircle(cx + x, y, 3, ILI9341_BLACK);
}
tft.fillRect(0, 199, tft.width(), 41, ILI9341_NAVY);
tft.setTextColor(ILI9341_WHITE, ILI9341_NAVY);
tft.setTextSize(2);
tft.setCursor(16, 207);
tft.print("Touches: ");
tft.print(touchCount);
}
void loop() {
bool pressed = touch.touched();
if (pressed) {
TS_Point point = touch.getPoint();
if (!wasPressed) {
++touchCount;
alternateColor = !alternateColor;
drawFace(alternateColor ? ILI9341_CYAN : ILI9341_YELLOW);
}
if (millis() - lastReport >= 150) {
lastReport = millis();
Serial.printf("raw x=%d y=%d pressure=%d IRQ=%d\n",
point.x, point.y, point.z, digitalRead(TOUCH_IRQ));
}
}
wasPressed = pressed;
delay(20);
}