Plugin platform · Rokid AR glasses

Install a phone app,
get a glasses app.

One hub lives on the glasses and renders everything. Every feature ships as an ordinary Android APK on your phone — and nothing is ever installed on the glasses again.

  • 8 plugins
  • 1 gesture claimed
  • 0 cables, PCs, or ADB
Download for Android v1.1.2 · APK Read the roadmap
The Nexus launcher on the glasses HUD, listing installed plugins in green monochrome
Triple-tap. From anywhere.
The Nexus phone app home screen, listing installed plugins with update badges
v1.1.2

Control messages back on their own path · Relay and Photo Sync joined in 1.1.0 · SDK sdk-v0.9.0 on JitPack · Apache-2.0

What changed →
7shipped on hardware, in a public release 2building display arbitration · continuous speech 3next committed, not started 5exploring each one an ordinary phone APK

Roadmap

No dates and no marketing. What is done, what is being written now, what is committed, and the problem each one actually solves — including the ones that turned out to be harder than they looked.

Shipped

Running on hardware, in a public release.

  • The bus, and the identity it enforces

    Any APK may bind to the hub; installing one grants it nothing. A plugin is identified by package + plugin id + signing certificate, and each capability — surfaces, microphone, camera, http_proxy — is a separate user grant, checked at the hub on every message rather than once at install.

    Wake-on-message: a plugin does not have to be running. The hub binds it awake when traffic arrives — measured at ~1.6 s including cold start — and it goes back to nothing.

  • One link, two paths, and the hub picks

    Control messages ride CXR-L; anything binary — images, photo sync — goes over SPP. A plugin never chooses, and never learns which path its bytes took.

    1.1.1 flipped that order to route around a link that reported sends it had not delivered, and 1.1.2 flipped it back: SPP is a single RFCOMM channel with one write lock, so a control message queued behind a photo chunk waits for the whole chunk. The real fix is an acknowledgement, not a different running order.

  • Setup without a computer

    Seven steps on the phone, the glasses app pushed over the Rokid link straight from GitHub releases, then a two-card self-arm on the glasses: accessibility on, then the hub bootstraps its own privileged shell.

    Wireless Debugging self-pairing with an app-private KADB TLS identity and a detached watchdog — it never touches the classic ADB key, so nothing on a PC is ever enrolled. Navigation reads the firmware's own localized labels, so it works in every language the ROM ships.

  • The distribution loop, closed

    The SDK publishes to JitPack from sdk-v* tags. A plugin releases under its own namespaced tag, a manifest PR lands it in the public registry, and the in-app Store verifies SHA-256 and signer before the install runs.

    Provenance is checked in a fixed order — commit, tag, build, publish — because the registry refuses a manifest whose artifact it cannot tie back to the tag. Everything self-updates afterwards: phone from releases, glasses over CXR, plugins from the Store.

  • All five display tiers, and the motion under them

    Ambient, pin, activity, notice, surface — plus the shared glyph set, plugin marks that travel to the glasses as bare geometry, and the phone's own battery in the ROM status row.

    One motion layer with three duration tokens and two interpolators, and nothing a plugin can dial. Native Views, not a WebView: the WebView spike rendered the same motion for ~1.2 cores, +88 MB PSS and 2.2 s to first paint, against 7.7 % CPU — and its one real advantage, plugin-authored layout, is something the activity tier refuses by design.

  • The camera capability

    The glasses stream live H.264 over a Wi-Fi Direct link and the consumer plugin decodes on the phone, where ML Kit runs OCR and translation offline. No glasses-side plugin code exists: the glasses half is a platform capability.

    A phone app cannot switch its own Wi-Fi on, which used to be a dead end. Now the roles invert — the phone hosts a LocalOnlyHotspot and the glasses join it — and the wire protocol, the decode, the overlays and freeze are unchanged.

  • Waking a dark display, without owning it

    A notice worth it can pulse the display awake. At most one wake every five seconds across every plugin, always a short pulse, never held on — and no other tier may do it at all, including activities.

Building

Specified, decided, and being written.

  • Display arbitration

    The protocol has carried an actionable class since v1 and v1 still renders it as a toast. Right now the entire policy is "the newest replaces the oldest, no queue" — which holds exactly until two chatty plugins are installed at once.

    What it needs: an arbiter that ranks a request against what is already on the display, plus surface ownership epochs so a late frame from a superseded owner cannot repaint someone else's surface. The per-plugin mute and demote switches in the phone hub are the user-facing half, and they already exist.

  • Continuous speech

    Speech-to-text ships, in short takes: the audio lease is specified, hardware-validated, and has a real consumer. What it cannot do yet is run for minutes.

    The remaining slice is a held lease with partial results streaming to the HUD, and a caption presentation that survives the surface underneath it changing. Live captions, translation-in-your-ear, and any voice assistant are all blocked behind this one — which is why it is the only other thing being written.

Exploring

Not committed. Each one is an ordinary phone APK against a capability that already exists — which is the point.

  • Live captions and translation

    audio lease · blocked on continuous speech

  • A voice assistant

    audio lease · blocked on continuous speech

  • Teleprompter, glanceables

    today's surface kinds, no protocol work

  • A visual assistant, FoodFacts

    camera capability, shipped

  • Sport HUD, CGM glucose

    activity tier + a small protocol addition

Never

Non-goals, stated once so nobody has to ask twice.

  • Native glasses apps are launched, never absorbed

    Porting them into Nexus would make the platform responsible for software it did not write. The menu starts them; that is the whole relationship.

  • Plugin code stays on the phone

    The glasses half of any capability lives in the hub. That is exactly what makes zero glasses-side deployment possible, and it is not negotiable.

  • No signature-only plugin permission

    It would be the strongest trust model available, and it would also make third-party plugins impossible — no external developer can be signed with our key.

  • One host, one platform, for now

    Hi Rokid Global and Android. Anything else is a promise we cannot currently keep on hardware we can test.

Five ways to be on the display

A plugin never picks how loudly it speaks. It declares what kind of thing it has, and the platform decides the form, the motion, and whether it may interrupt at all. Below is that renderer, reproduced — same phosphor, same 480×640 optic, same durations.

enter 280 ms · fast-out-slow-in

Timings are the platform's, not the plugin's: 180 ms for a value refreshing in place, 280 ms for anything arriving or changing shape, 240 ms for anything leaving — exits are quicker than entrances. It runs on native Views because the WebView that rendered the same motion cost ~1.2 cores, +88 MB and 2.2 s to first paint, against 7.7 % CPU here. The notice band's six states →

Eight plugins, none of them built in

Every one is an ordinary phone APK installed from the in-app Store, pinned by hash and signer before it installs. The hub itself ships empty. These are real captures off the optic, uncropped.

A Relay notice band over the glasses home screen showing a two-message thread with a Reply action
RelayA message arrives as a band over whatever you were looking at, and is answered out loud.
The Relay inbox listing conversations still waiting for a reply
Relay · MessagesEverything you let go, newest first, answered the same way.
Lens translating a screen of English text into French live on the HUD
Lens · liveThe camera streams to the phone; OCR and translation run there, offline, and the overlays come back.
Lens freeze mode showing a full-resolution still with the translated result
Lens · freezeA full-FOV still through the same link, for text you want to hold on to.
The Transit plugin showing nearby stops and live departure countdowns
TransitNearby stops and live departures, with the ones you actually take kept on top.
The Feeds plugin showing a timeline post on the glasses HUD
FeedsBluesky and X timelines — posts, threads, and the actual photos full-screen.
The Media Deck plugin showing album art, track title, and transport controls
Media DeckNow playing with album art and transport, from whatever app is playing.
The Lyrics plugin showing the current line time-synced to the playing track
LyricsTime-synced lines for whatever is playing, from four sources.
no HUD surface Photos Sync Not a HUD plugin at all. It copies the photos and videos you shoot on the glasses into your phone gallery by itself, and hands you the switches for it.
template Sample The minimal copyable plugin. Clone it, change the id, and you have a working glasses app that never runs on the glasses.

Installation alone grants nothing. Surfaces, microphone, camera, and HTTP proxy are approved one by one, per plugin, keyed to its package, its plugin id, and its signing certificate — and checked at the hub on every message, not at install time.

Get started

Running it takes a phone and four steps. Writing for it takes one Gradle coordinate.

Run it

  1. Install the phone app. One APK from the releases page — it is the hub, the Store, and the setup guide.
  2. Follow the seven steps. The phone drives the pairing, turns the glasses' Wi-Fi on, and pushes the glasses app over the Rokid link.
  3. Say yes once, on the glasses. Enable accessibility when asked; Nexus bootstraps the rest of its glasses-side setup itself.
  4. Install a plugin from the Store. Everything keeps itself current afterwards.

No cable, no PC, no ADB at any point — including the privileged parts, which the glasses arm by themselves in two cards.

Write for it

  1. Depend on the SDK. :bus-client brings the service base class, the typed surfaces, the lifecycle callbacks and the design kit.
  2. Declare who you are. A headless service, a plugin id, a 24×24 mark in one green, and the capabilities you need — no launcher icon.
  3. Push a surface. Describe content, never pixels. Your process does not need to be running: the hub binds you awake.
  4. Publish, optionally. You host the APK, the registry indexes it, the Store pins your hash and your signer.
build.gradle.kts
repositories { maven("https://jitpack.io") }

dependencies {
    implementation(
        "com.github.Anezium.Rokid-Nexus:bus-client:sdk-v0.9.0"
    )
}

Every limit in the contract is enforced in code — breaking one does not degrade gracefully. All documentation →