Building an Autonomous AI Agent on Android: Offline Models, MCP, and What Is Feasible

An Android phone can host an agent's loop, tools, and memory, and run small 0.5B to 4B models offline for summaries and classification. MCP servers connect remotely over HTTP rather than as local processes. Reliable multi-step planning still needs a larger model off the phone, and Android's sandbox sets the real limits on what the agent can touch.

What the project in question is

The video behind this topic, from the channel thevoidkernel, demonstrates PocketStrike, an open source Android app published on GitHub under the MIT licence and written in Flutter. According to its README, it combines on-device GGUF inference through a native llama.cpp bridge, cloud providers such as OpenAI, Claude, Gemini, DeepSeek and Ollama, a ReAct-style agent engine with built-in file tools, an MCP client using Streamable HTTP and SSE transports, persistent memory files (USER.md, MEMORY.md, SOUL.md), and a Telegram relay that lets the phone take instructions remotely.

It is not the only route. Google's AI Edge Gallery app runs models such as the Gemma 4 family offline on Android 12 and later through the LiteRT-LM runtime, and includes a Mobile Actions demo for offline device control built on a fine-tuned FunctionGemma 270M. Android's ML Kit GenAI APIs expose Gemini Nano, which runs in the AICore system service, to apps on supported devices. And Hermes Agent ships a Termux package for aarch64 Android devices. The pieces exist. The question is which parts belong on the phone.

Three parts, three different answers

An agent has a model that decides, tools that act, and permissions that bound both. On a phone each one has a different feasibility profile.

The model: small, and mostly for narrow jobs

Phone memory is shared with the operating system and every other app, and Android reclaims memory from background apps aggressively. That caps practical on-device models at roughly 0.5B to 4B parameters at 4-bit. PocketStrike's README lists its presets from SmolLM2 135M at about 150 MB up to Llama 3.2 3B at about 2.1 GB. Models in this range summarise, classify, extract, and follow short instructions well. They are unreliable at multi-step planning and at choosing among many tools, which is the core of an autonomous agent. The README describes CPU acceleration through ARM NEON, while the video describes GPU acceleration; either way, expect generation that is usable for short answers and slow for long ones, plus heat and battery drain under sustained load.

The tools: MCP works, over the network

Most desktop MCP servers run as local processes over stdio, launched by the client. An Android app cannot casually spawn Node or Python servers, so the practical transport on a phone is remote: Streamable HTTP or SSE to servers running elsewhere. That is what PocketStrike's MCP client supports, and it explains the demo of controlling a PC from the phone: the server runs on the PC, and the phone is the client. Hosted servers such as GitHub's work the same way. The phone becomes a remote control for tools that live on other machines.

The permissions: Android's sandbox is the real boundary

An app only reaches what Android grants it. File tools work inside app storage and folders the user grants, which is why PocketStrike scopes file tools to the Download folder and app documents. Controlling other apps requires the Accessibility API, which Google Play restricts to approved uses. Long-running background work needs a foreground service with a visible notification, or the system will suspend it.

A split that actually works

The reliable design keeps the phone's model on the jobs it handles well and sends hard reasoning elsewhere.

  1. On-device model for private, narrow work. Summarising notifications, extracting dates from messages, classifying whether a request needs a tool, offline fallback with no signal. Nothing leaves the phone.
  2. A hosted model, or a model on your own server, for planning. Multi-step tasks and tool selection go to a model that can do them, reached over the same network the MCP servers use.
  3. Tools on the machines that own the data. MCP servers on your PC, home server, or hosted services, each with narrow credentials.
  4. Short context. Small models degrade quickly as prompts grow, and phone memory limits the KV cache. Retrieve the few relevant passages from indexed notes rather than loading whole files. RDK, for example, indexes vaults as encrypted private chunks that an agent can search before calling any model.

Security on a phone that takes orders

A phone agent with a messaging relay, file access, and MCP connections to your PC is a remote command channel. Treat it as one.

Authenticate the relay. Allowlist the account that can send instructions; anyone who can message the bot can otherwise direct it.

Keep MCP servers off the public internet. A PC-control server should listen on your local network or a private tunnel, with authentication.

Keep confirmation on. PocketStrike's README describes policies of Destructive Only, Always Ask, or Autonomous for irreversible operations. Start with Always Ask.

Be careful with uncensored models. Some presets in projects like this remove refusal behaviour. On a model that only chats, that is a content choice. On a model that executes tools, it removes a layer of defence against injected instructions in web pages, messages, or files the agent reads.

Frequently asked questions

Can an Android phone run an AI agent offline?
Partly. Small models of roughly 0.5B to 4B parameters at 4-bit run on recent phones through llama.cpp-based apps, Google's AI Edge Gallery, or Gemini Nano on supported devices. They handle summaries, classification, and short instructions offline. Reliable multi-step planning and tool choice usually need a larger model, so fully offline autonomy is limited to simple, narrow tasks.
How do MCP servers work with a phone agent?
Mostly remotely. Desktop MCP servers commonly run as local stdio processes, which an Android app cannot easily launch. A phone agent instead connects over Streamable HTTP or SSE to servers running on a PC, a home server, or a hosted service. The phone acts as the client, and the tools run on the machines that hold the data.
What size model fits on a phone?
Plan for small models. PocketStrike's README lists presets from about 150 MB for SmolLM2 135M to about 2.1 GB for Llama 3.2 3B at 4-bit, and Google's AI Edge Gallery targets edge variants of Gemma 4. Phone memory is shared with the system and other apps, and Android reclaims it from background apps, so leave generous headroom.
Is it safe to let a phone agent control my PC?
Only with deliberate limits. The PC side is an MCP server, so restrict it to your local network or a private tunnel, require authentication, and expose only the tools you need. Allowlist who can message the agent through any relay, keep human confirmation on for destructive actions, and avoid models with refusal behaviour removed when they can execute tools.