Android 16 App Functions Deep Dive: Semantic Indexes and Intent Routing

A few days ago, a product manager handed over a requirement: when a user selects a group photo in the gallery, the system should automatically figure out who the photo is suitable to send to. Not by opening the share sheet and asking the user to choose, but by understanding the meaning directly: “send this group photo to the people in it.”

Android’s Intent system has been around for more than a decade, and its boundaries are clear. Apps declare the Action types they can handle, and the system launches them after string matching. But once the requirement reaches the layer of “understanding what the user means,” that mechanism is no longer enough. Android 16’s App Functions framework is designed for this exact problem: replace string matching with semantic indexing, and push cross-app operations from “manual selection” toward “intent routing.”

Semantic indexing: vectorized app capabilities

Traditional Intent Filter matching is built on the action + data + category tuple. At its core, it is keyword filtering. App Functions takes a different approach: apps declare executable functions with structured schemas, while the system semantically encodes their descriptions and builds a vector index.

Registration looks like this:

val function = FunctionDeclaration.Builder()
    .setName("share_to_contact")
    .setDescription("Share photos or files with contacts through a messaging app")
    .addParameter("recipient", Schema.String, "Recipient name")
    .addParameter("mediaUri", Schema.Uri, "Media file to share")
    .setExecutionCallback { args, context ->
        val recipient = args["recipient"] as String
        val uri = (args["mediaUri"] as Uri)
        shareToContact(recipient, listOf(uri))
    }
    .build()

appFunctionManager.registerFunction(function)

The natural-language text passed to setDescription is the core of the whole mechanism. The system feeds this text into an on-device text encoder, generates a semantic vector, and stores it in a local index. When the user says “send this photo to Alex,” the system vectorizes that instruction as well and performs nearest-neighbor retrieval in the index.

This has much higher recall than keyword matching. “Send it to Alex,” “forward this to Alex,” and even “Alex should take a look at this picture” can all hit the same share_to_contact Function. The semantic vector captures intent, not literal wording.

The index is built asynchronously after app installation, and app upgrades trigger incremental updates. The user does not notice the process. All data is processed locally and does not depend on the cloud.

Intent routing: more than matching

Finding candidate Functions through the semantic index is only the first step. Full intent routing has to solve three problems: matching, disambiguation, and dispatch.

Android 16 ranks candidates with an Execution Score, combining these dimensions:

• Semantic similarity: cosine distance between the user’s instruction vector and the Function description vector
• Call history: which app this user preferred for similar instructions in the past
• Context weight: Functions from the foreground app receive extra weight because the app the user is actively using is most likely relevant
• Parameter completeness: whether parameters extracted from the instruction cover the Function’s required parameters

After scoring, the system either executes the top-1 candidate directly in previously authorized scenarios, or presents the top-3 candidates for user confirmation.

This routing layer is fully transparent to app developers. You do not need to care where your Function ranks or which app outranked it. Register the Function and let the system handle matching and dispatch.

Two-stage coordination in the on-device AI agent

Semantic indexing handles the coarse filter with vector computation, but real user instructions are often ambiguous. For example: “send yesterday’s weekly report to my manager.” Which file is “the weekly report”? Who is “my manager”? Entity disambiguation like this cannot be solved by vector retrieval alone.

Android 16 uses a two-stage pipeline:

Stage 1, coarse retrieval: retrieve the top-3 candidates from the full Function index. This is pure vector computation and finishes in milliseconds.

Stage 2, reranking and disambiguation: an on-device model, such as Gemini Nano or a user-deployed LiteRT model, performs deeper reasoning only on the top-3 candidates and resolves entities and parameters:

val resolutionContext = IntentResolutionContext.Builder()
    .setUserUtterance("Send yesterday's weekly report to my manager")
    .setCandidateFunctions(top3Matches)
    .build()

val resolved = intentResolver.resolve(resolutionContext)
// resolved.entities: {"weekly report" -> "Q4_weekly_report_v3.docx", "manager" -> "Jordan"}
// resolved.bestMatch -> share_to_contact, recipient=Jordan

The value of this two-stage design is controlling inference cost. Asking an agent to reason over hundreds of Functions would push latency into seconds, which users will not tolerate. A coarse filter narrows the set to 3 candidates, reducing the agent’s reasoning workload from O(n) to O(1).

Measured latency is roughly 5-15 ms for the first stage, using vector retrieval, and 200-500 ms for the second stage, using model inference. End-to-end latency stays within an acceptable range.

Engineering details that matter in practice

Several issues came up during integration. Planning for them early saves a lot of time.

Description text directly determines matching accuracy. setDescription("send message") and setDescription("send text or image messages to specified contacts through an instant messaging app, supporting both group chats and direct chats") can differ by 3-5x in recall. The description should cover phrases users might actually say, but it should not become a pile of keywords. Semantic models are sensitive to natural language, and keyword stuffing can dilute the vector direction.

Parameter schemas need room for fault tolerance. Agent-based entity parsing is not perfectly accurate. It may occasionally parse “send three photos” as if “three” were a name. The ExecutionCallback must perform type validation and graceful fallback.

Register fewer, higher-value Functions. In testing, an app with 5 core Functions had a much higher matching success rate than one with 30 obscure Functions. Noisy Functions in the index reduce overall precision, so register only high-frequency user operations.

During debugging, adb shell dumpsys app_functions can export every registered Function on the device and its semantic-vector status. It is much more efficient than digging through documentation when you are diagnosing matching issues.

App Functions moves Android cross-app interaction from “declarative matching” to “semantic understanding.” It does not replace Intent. Instead, it adds an AI-driven intent routing layer on top of Intent. The target scenario is not “which app should open,” but “understand what the user wants to do, then dispatch the right app to do it.”