上篇把 Python 打进了 APK，但有个问题没回答：Agent 要"动"起来，它得能看屏幕、能点按钮。
可 Python 又不是安卓亲生的——它自己不会截图、不会点控件。谁来看屏？谁来点击？在我这个项目里，答案是：Python 内核发令，Kotlin 壳去执行。中间那座桥，就是这篇要讲的。
没有这座桥，Agent 只是个会说话、不会动手的大脑。
先放一张总览，免得后面绕晕——整座桥就这么几块：
虚线是"注入/回传"，粗实线是"Python 真的下指令"。看懂这张，后面两段就是把它拆开讲。
第一向：Kotlin 把门先开开
App 一启动，Kotlin 这边先把 Python 解释器拉起来，再把一个"派发器"塞进 Python 的世界：
// KernelHostService.attachKernel()
if (!Python.isStarted()) {
Python.start(AndroidPlatform(this))
}
val py = Python.getInstance()
val bridgeMod = py.getModule("hachimi_kernel.bridge")
// 把安卓侧的派发器注入 Python
bridgeMod.callAttr("register_bridge", PrimitiveDispatcher())
callAttr 就是 Chaquopy 给的"跨语言函数调用"——Kotlin 这边写 模块.callAttr("函数名", 参数)，对面就是 Python 里一个同名函数。
这一步干的事很简单：把能操作真机的对象，交给 Python 握着。后面 Python 想干啥，调这个对象就行。
第二向：Python 反过来指挥真机（最反直觉）
一般人会以为，Python 要截图，得 import android 然后调安卓 API。但在 Chaquopy 里，Python 不import 安卓任何东西——它只调 Kotlin 刚才塞进来的那个对象：
# bridge.py：Kotlin 注入的派发器，存在模块全局
_bridge = None
def register_bridge(dispatcher):
global _bridge
_bridge = dispatcher
# Agent 要观察屏幕时，就这么一行
raw = _bridge.call("observe", json.dumps({"compact": True}, ensure_ascii=False))
result = json.loads(raw)
observe 这个名字会被传回 Kotlin，那边有个 when 把它派发到真正的安卓代码：
// PrimitiveDispatcher.call()
fun call(name: String, argsJson: String): String {
val svc = HachimiAccessibilityService.instance ?: return error("a11y not bound")
val args = JSONObject(argsJson)
return when (name) {
"observe" -> svc.observe(args.optBoolean("compact", false))
"tap_by_id" -> svc.tapById(args.optString("view_id"))
"tap_xy" -> svc.tapXY(args.optDouble("x"), args.optDouble("y"))
"type_text" -> svc.typeText(args.optString("text"))
"launch_app" -> svc.launchApp(args.optString("package"))
// ……还有 gesture / screenshot / press
}.toString()
}
看到没？Agent 的"看一眼""点一下"，最后全落到这段 when 里。它就是 Agent 的手和眼睛在具体手机上的落点。
这反直觉的一点很重要：Python 不直接碰安卓，它只发指令；真活儿是 Kotlin 干的。这样内核可以保持纯净，不依赖任何安卓细节，换台设备也好迁移。
数据怎么传：一个真实的坑
两层语言之间传数据，最顺手的方式是用 JSON 字符串。原因是个坑：
Chaquopy 返回的 PyObject，你直接 .toString() 拿到的是 Python 的 repr（带单引号那种），Kotlin 这边的 JSONObject 解析不了。所以两侧约定：进出都先 json.dumps / json.loads 包一层，字符串过去了再各自解开。
raw = _bridge.call("observe", json.dumps(args, ensure_ascii=False))
return json.loads(raw) # 解开再用
这个坑不大，但第一次遇到会莫名其妙——明明数据是对的，Kotlin 那边就是解析失败。记一笔：跨语言传复杂数据，显式 JSON 字符串最稳。
用一张图对比两种传法就清楚了：
其实这个 JSON 协议不只跨语言用——Agent 的工具参数和返回、喂给大模型的消息、本地轨迹落盘，全是用同一套 dict，边界完全同构。一座桥的协议，顺手把整条链路也统一了。
工具面：Agent 的手和眼到底有哪些
桥通了，但 Python 能调的"原语"具体是哪些？它们写在 bridge.py 的 TOOL_SPECS 里，每个原语注册成一个工具，背后就是一行 _bridge.call：
# bridge.py：把八原语注册成 Agent 可用的工具
for spec in TOOL_SPECS:
tools[spec["name"]] = {"description": spec["description"],
"parameters": spec.get("parameters", {"type": "object"}),
"call": _call(spec["name"])} # _call 闭包里就是 _bridge.call(name, ...)
列一下 Agent 真正能做的动作，基本就是"看"和"动"两类：
- 看：observe（读当前界面 UI 树）、screenshot / grab_frame（拿像素图，视觉兜底用）
- 动：tap_by_id（按控件 id 点）、tap_xy（按坐标点）、type_text（输字）、gesture（手势）、launch_app（开应用）、press（按键，如返回）
- 辅助：wait（等界面加载）、look（显式问一眼屏幕）
还有两个不在 Kotlin 那边、纯 Python 实现的：load_skill / search_skill——让 Agent 调出以前沉淀过的操作套路。这俩是下一篇的主角，先埋个伏笔。
桥在 Agent 主循环里的实际位置
光说"能调"还不够直观。我把一次真实任务的完整走查摆出来，你看桥站在哪：
1. 你在 app 里点"开始"，Kotlin 收到 HTTP 的 /task 请求，调 bridgeMod.callAttr("start_task", body)；
2. Python 的 start_task 在后台线程拉起 MiniLoop（Agent 主循环），把目标和大模型配置传进去；
3. MiniLoop 每轮让大模型决策：reply = brain.chat(messages, tool_specs)，模型返回"下一步调 observe"；
4. 循环里 self._dispatch(name, args, i) 执行这个工具 → 也就是 tools["observe"]["call"](args) → 背后 _bridge.call("observe", ...)；
5. Kotlin 真去读无障碍树，把结果 JSON 传回 Python；
6. Python 把这个"看屏幕"的结果当工具消息喂回模型，模型看着屏幕决定"点这个按钮" → 又走一遍 _bridge.call("tap_by_id", ...)；
7. Kotlin 真去点。模型确认目标达成 → 调 task_done，循环结束。
看一眼、点一下、再决定——Agent 在手机上跑起来的每一帧，桥都在中间传话两次（出一次指令、回一次结果）。上篇把 Python 塞进去只是地基，这篇才是 Agent 长出手脚、还接上了大脑的那一步。
把上面 1~7 步走查画成一张循环图，桥站在哪一目了然：
几个真实护栏：Agent 在手机上得"安全动"
能跑还不够，得防止它瞎点。这些都在 MiniLoop 里，是真实代码不是设想：
- 卡住检测：连续 4 次调完全相同的 (工具, 参数)，直接判 stuck 终止——避免它在一个点不下来的按钮上死循环。
- 换方案提醒（RepeatGuard）：同一个工具连续失败 3 次，往上下文里塞一句"请更换执行方案"，逼模型别再重复同一个失败调用。
- task_done 强制校验：启用了完成判据时，模型说"我做完了"会被 verify 拦下来复核；没通过就打回让它继续，防止幻觉式宣布完成。
- 预算：默认最多 24 步、墙钟 600 秒，超限按"未完成"终止，不伪装成功。
这几道护栏加在一起，Agent 才是一个"能动、但乱动会被叫停"的东西，而不是一个你不敢放手让它跑的脚本。
为什么这么设计桥接（几点取舍）
顺手说下这么搭桥的理由，都是踩过或权衡过的：
- 注入式，而不是 Python 直接 import 安卓：内核保持纯净、能在电脑上单测，不绑任何安卓细节。代价是 Python 侧的 API 是"被喂进来的"，得约定清楚。
- 八原语单一事实来源：PrimitiveDispatcher 同时被调试桥（电脑直连发指令测）和 Python 桥共用，保证两边说的"原语"是同一套，不会各写各的。
- JSON 协议贯穿三层：Python↔Kotlin、模型↔工具、轨迹落盘，全用同一套 dict。边界同构，出问题时一眼能追到是哪一层。
桥的另一头：手机里的 Agent 也能被远程指挥
桥接层还顺带解决了一个更实用的需求——Agent 不只能在你点"开始"时跑，也能被外部驱动。
KernelHostService 起了一个本地回环 HTTP（8071 端口）：/task 让电脑或脚本 POST 一段目标就触发手机 Agent 跑任务，/health 查状态，/stop /pause /resume 控制它。也就是说，你人在电脑前，发一条指令，手机自己就开始操作自己——桥把"手机 Agent"变成了一个可被远程调用的能力，而不只是个 app 里的按钮。
收尾
把上面几块拼起来：Kotlin 起解释器、把派发器注入 Python、Python 用 JSON 指挥真机、MiniLoop 把这变成 observe→思考→行动的循环、再加几道护栏和一个远程控制面——Agent 在手机上"动起来"这件事，底座就是这座桥。
上篇是地基，这篇是手脚。到这，一个手机端 Agent 从打包进 APK、到两层语言打通、到真能在手机上动起来的完整链路，就讲完了。
仓库地址：github.com/momo-null/hachifish
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Python got into APK, but there was one question that didn't answer: Agent had to move, he had to look at the screen, he had a button.
But Python wasn't born in Andre -- it didn't copy it, it didn't point the controls. Who's watching the screen? Who's clicking? In my project, the answer is: Python Internal Order, Kotlin Shell. This is the bridge in the middle.
Without this bridge, Agent is just a talking, handsless brain.
Let's start with a general overview before we go around the back -- the whole bridge is just a few pieces:
The dotted line is "injection/return" and the rough line is "Python really gives orders". Read this one. The last two paragraphs are to open it.
First direction: Kotlin, open the door first.
As soon as App starts, Kotlin pulls up the Python interpreter and stuffs it into the Python world:
// KernelHostService.attachKernel()
if (!Python.isStarted()) {
Python.start(AndroidPlatform(this))
}
val py = Python.getInstance()
val bridgeMod = py.getModule("hachimi_kernel.bridge")
Python
bridgeMod.callAttr("register_bridge", PrimitiveDispatcher())
Call Attr is Chaquapy's "Inter-Language Function Call" -- Kotlin writes module. CallAttr ("Function Name", Parameters"), opposite is a function of the same name in Python.
It's a simple step: hand over to Python the person who can handle it. Behind you, Python, you can do whatever you want with this object.
Second direction: Python, in turn, command the real machine.
The average person would think that Python was going to intercept, get import and roid and then set Anjo API. But in Chaquapy, Python doesn't report anything from Anzor -- it's just the one that Kotlin just put in:
#bridge.py: Kotlin injects a transmitter, which exists in the entire module
_bridge = None
def register_bridge(dispatcher):
global _bridge
_bridge = dispatcher
# Agent, when you want to watch the screen, that's the line
raw = _bridge.call("observe", json.dumps({"compact": True}, ensure_ascii=False))
result = json.loads(raw)
This name will be sent back to Kotlin, when it is sent to the real Android code:
// PrimitiveDispatcher.call()
fun call(name: String, argsJson: String): String {
val svc = HachimiAccessibilityService.instance ?: return error("a11y not bound")
val args = JSONObject(argsJson)
return when (name) {
"observe" -> svc.observe(args.optBoolean("compact", false))
"tap_by_id" -> svc.tapById(args.optString("view_id"))
"tap_xy" -> svc.tapXY(args.optDouble("x"), args.optDouble("y"))
"type_text" -> svc.typeText(args.optString("text"))
"launch_app" -> svc.launchApp(args.optString("package"))
/...and gesture / screenshot / press
}.toString()
}
You see? Agent's "a quick look" and it all ended up in this part when. It's the drop point of Agent's hands and eyes on a specific phone.
This counterintuitive thing is important: Python doesn't touch Andre, it only gives orders; the real job is Kotlin. This allows the kernel to remain pure, without reliance on any Android details, or to move equipment.
How does the data pass? A real pit.
The most easy way to transmit data between two layers of languages is through a JSON string. The reason is a pit:
Chaquapy returns PyObject, you get directly.toString() with Python's repr, Kotlin's JSONObject cannot solve. So it's agreed on both sides: each enters and exits will be covered by json.dumps / json.loads, and then the strings will be removed.
raw = _bridge.call("observe", json.dumps(args, ensure_ascii=False))
# Undo
It's not a big pit, but the first time I've seen it, it's the right data. Kotlin's side is a failure. Note: Trans-linguistic complex data, visible JSON string is the most stable.
It's clear by a graph of two legends:
In fact, this JSON protocol is more than just a language -- Agent's tool parameter and return, feed to LLM, local trajectories, all with the same set of dicts, the border is identical. The agreement on a bridge was followed by the unification of the entire link.
Toolface: What exactly is Agent's hand and eye?
The bridge's open, but what exactly is Python's original? They're written in Bridge.py's TOOL SPECS, and each original language is registered as a tool, behind which is a line bridge.call:
#bridge.py: Register the eight original languages as tools available to Agent
for spec in TOOL_SPECS:
tools[spec["name"]] = {"description": spec["description"],
"parameters": spec.get("parameters", {"type": "object"}),
"call": call(spec[name])} call in closed bag is bridge.call(name,...)
Agent, what really works is basically "see" and "move":
- See: observe (read the current interface UI tree), screenshot / grab frame (take pixels, use visual background)
- Move: tap by id (by control id point), tap xy (by coordinates point), type text (lost), gesture (hand gesture), lanch app (open application), presses (keys, e.g. return)
- Auxiliary: wait (interface loading), look (visit screen)
Two other things that are not done by Kotlin, pure Python: load skill / search skill -- let Agent pull out the operating suite that he used to sink. These are the main characters in the next one.
The actual location of the bridge in Agent's main cycle.
It's not enough to say "tweak." I've got a full real task out. Look where the bridge is.
1. You "start" in an app, Kotlin, received a request from HTTP for a BridgeMod. Callattr ("start task", body);
2. Start task of Python pulls up MiniLoop (Agent main cycle) in the backstage to pass the target and LLM configuration;
3. MiniLoop, which allows LLM decision-making per round: reply = brain.chat (messages, tool specs), the model returns "next step to observ";
4. Self. dispatch(name, args, i) implements this tool → i.e. behind bridge.call("observe",...);
5. Kotlin actually reads accessible trees and sends the results back to Python;
6. Python feeds the results of this "watch screen" back to the model, which looks at the screen and decides to "point this button" again bridge.call("tap by id",...);
7. Kotlin really went to point. The model confirms that the target has been reached.
Take a look, a click, and then decide -- every frame that Agent runs on his phone, the bridge passes twice (one instruction, one result back). The last thing that stuffed Python in was a foundation, and that's the one where Agent grew his hands and legs and got to the brain.
Make a loop map of the 1-7 steps up there.
A couple of real bars: Agent got "safe" on his phone.
It's not enough to run, to keep it from going blind. It's all in MiniLoop, it's the real code, not the idea:
- Stuck detection: 4 consecutive rounds of exactly the same (tools, parameters), direct stuck termination -- avoid it dying on a non-stop button.
- RepeatGuard: The same tool failed three times in a row: "Replace the execution program" to context, forcing the model not to repeat the same failure call.
- Task done: When the trial is completed, the model says "I'm done" will be stopped for review by verify; call it back if it doesn't pass, and prevent an illusion from being declared complete.
- Budget: Defaults of up to 24 steps, wall clocks 600 seconds, exceeding the "unfinished" deadline, without disguise of success.
These fences add up, and Agent is an "activated but stopped" thing, not a script that you can't let go of.
Why did you design the bridge?
By the way, the reason for the bridge is that it has been stepped or weighed:
- Injection, not Python, direct Import Andre: kernel is pure, computer-free and does not bind any Andres. The price is Python's side of the API is "baked in."
- Single source of eight originals: Primitive Dispatcher is shared with Python Bridge at the same time, making sure that the originals are the same and not the same.
- The JSON protocol runs through three layers: Python↔Kotlin, model tool, orbital drop-off, all using the same set of dicts. Borders are the same, which layers can be traced at once.
The other side of the bridge: Agent on the phone can be remotely directed.
The bridge bridge also addresses a more practical need -- Agent can be driven from the outside not only when you point "start."
KernelHostService launched a local loop HTTP (8071 port): /task let a computer or script POST target trigger an Agent run, /health check state, /stop /pause /resume control it. In other words, you're in front of a computer with a command that your cell phone starts operating itself -- the bridge turns the cell phone into a remote callable button, not just an app button.
End
Put the pieces together: Kotlin's interpreter, Python, Python's PSON, MiniLoop, turn this into a cycle of observe thinking, add a few bars and a remote control -- Agent's "move" on his cell phone, which is the bridge.
It was the foundation, it was the hand. Here's the end of a cell phone-end, Agent, from packing into APK, into two levels of language, to the complete link that really moves on the phone.
Warehouse address: Github.com/momo-null/hachifish
Auto-fetched and translated by AI Atlas. All rights belong to the original author.