Plugin platform · Rokid AR glasses

Install a phone app,
get a glasses app.

One hub lives on the glasses and renders everything. Every feature ships as an ordinary Android APK on your phone. Add a plugin without installing another glasses app.

  • 13 plugins, none built in
  • 1 gesture claimed
  • 0 cables, PCs, or ADB
The Nexus 2.0 launcher on the glasses: a Media Deck card with cover and playback controls, a Hello Nexus checklist card, an unanswered Relay message, then Notifications and Applications
Triple-tap. From anywhere.
The Nexus 2.0 phone settings for the five floating activity display modes
v2.6.0

WhatsApp, Instagram, Messenger and LINE on the glasses · pinned contacts first, then your recent conversations · larger launcher text · favourites under the activities · glasses sound in your earbuds · update both hubs together · SDK sdk-v0.27.0 on JitPack · Apache-2.0

What changed →

What's new in 2.6

Your conversations on the glasses. Meta Messages brings WhatsApp, Instagram and Messenger, LINE Messages brings LINE, groups included: read the latest messages, dictate or type a reply, take a photo with the glasses camera and send it. Nothing is sent before you tap Send. Pin the people you talk to most and they come first, above your recent conversations.

What's new in 2.0

A triple tap opens Nexus's own launcher on the glasses: your activities first, then the latest notification, Notifications and Applications. An activity opens as a page inside Nexus — a conversation, a player, a route with its map, a checklist — and Open app is still there when you want the plugin's full surface.

1:37 · no audio · demo conversation, simulated route · transcript
  1. Music on the card

    Media Deck puts the cover and Previous, Play/Pause and Next on its launcher card. Its player adds the progress; one Back leaves it.

  2. A route with a map

    The Navigation card shows the distance, the destination and the time left. Open it for guidance and a map, keep a minimap floating on the home screen, or go full screen. Nexus plans walks; Google Maps and Citymapper guidance stays text-only.

  3. A message becomes a conversation

    Relay opens a message as a conversation. Dictate on the glasses or type on the phone, read the draft, then tap Send, a separate step. Notifications history keeps the last 30 of the past 24 hours; one removed on the phone stays there, marked removed.

  4. Floating, per plugin

    On the phone, each plugin's activity is Automatic, Always visible, Home screen only, On changes or Activities only. On changes shows the band for 3, 5 or 10 seconds; the card in Activities stays after the band goes.

  5. A checklist is a page too

    Hello Nexus, the sample plugin, shows a Todo/Groceries list as a page, with the opt-in that keeps its card in Activities and a Remove pin action, all through the public SDK.

Roadmap

No dates and no marketing. What is done, what is being written now, what is committed, and the problem each one actually solves — including the ones that turned out to be harder than they looked.

Shipped

Running on hardware, in a public release.

  • 2.2.0 – 2.6.0

    WhatsApp, Instagram, Messenger and LINE on the glasses

    Meta Messages and LINE Messages put your conversations on the glasses, groups included: read the latest messages, dictate or type a reply, take a photo with the glasses camera and send it. Nothing is sent before you tap Send, and a send the service refuses keeps your draft. See it →

    Under the hood

    Both are unofficial clients: Meta and LINE make neither, and either may sign the device out or restrict an account. Each service connects only while its page is open; sign-in happens on the phone and credentials never leave it. LINE sends with Letter Sealing and refuses a chat without an encryption key rather than send plaintext.

  • 2.6.0

    Pinned contacts, then recent ones

    Pick the conversations you want first on the phone, in your order, with no limit. The glasses list them under Pinned, then your most recent conversations under Recent. A change shows on an open page right away.

  • 2.5.0

    Back after a restart away from Wi-Fi

    With Auto-repair at boot on, the glasses can briefly join a local hotspot from the phone to restore their command bridge, used by Glasses sound in earbuds and other shell features. No Internet or manual hotspot needed.

  • 2.3.0

    Larger text on the launcher

    Normal, Large or Extra large, set on the phone. Home, Applications, Notifications, conversations, documents, media and navigation pages all follow it, and an open page redraws in place. Asked for by a wearer with low vision.

  • 2.2.0

    Favourites, under the activities

    Up to four plugins, the camera or native glasses apps pinned right under the activity cards on the glasses home. Pin, reorder and unpin them on the phone.

  • 2.2.0

    Glasses sound in your earbuds

    With earbuds or headphones on the phone, everything the glasses play, Hi Rokid's voice included, can play in them while the glasses stay silent. If the earbuds go, the glasses get their speaker back within seconds.

  • 2.0.0

    A launcher, with pages inside it

    A triple tap opens Nexus's own launcher: your activities first, then the latest notification, Notifications and Applications. A conversation, a player, a route or a checklist opens as a page inside Nexus; Open app still gives the plugin's full surface. See it →

    Under the hood

    With an older phone hub, or the Classic launcher chosen, the glasses keep the 1.8 launcher. Plugins built on SDK 0.24.0 negotiate the launcher bundle and keep a fallback for hubs without it; anything built against 0.23.0 must be rebuilt.

  • 2.0.0

    Floating activities, chosen per plugin

    Whether an activity floats over the glasses is set on the phone, per plugin: Automatic, Always visible, Home screen only, On changes or Activities only. On changes shows the band for 3, 5 or 10 seconds; the card in Activities has its own lifetime. Plugins link to the setting through SDK 0.24.0 and never pick visibility or duration.

  • 1.7.0

    Skills: plugins that work for each other

    Ask Assistant when the next metro leaves or to pause the music, and it asks Transit or Media Deck for you, through the hub, without either plugin opening. Each operation is switched on by you, for each assistant, and all of them start off.

    Under the hood

    A plugin publishes typed operations in a catalog the hub reads without starting it. The hub stamps both identities, validates arguments and results against a bounded schema, hands out opaque references instead of the provider's identifiers, and ends every call within 15 seconds; a changed operation needs approving again. Checked on hardware from typed questions — voice, a moving journey and real music players are not yet.

  • 1.6.0

    YouTube and Reddit on the glasses

    Patcher patches YouTube on your phone for the touchpad and the R08 ring, and Nexus installs it on the glasses with MicroG for the Google sign-in. Reddit followed in 1.7.0, as a Preview.

    Under the hood

    Nexus checks the result's bytes and signer before installing, refuses a signer change, and never reads Google accounts. Reddit's R08 ring and real posting have not been checked on hardware.

  • 1.6.0

    Native apps in the Nexus menu

    The apps installed on the glasses sit under the same triple-tap as the plugins, after them and with their own icons. Nexus launches them; it does not port them or wrap them.

  • 1.5.0

    Navigation, from the apps you already use

    Start a route in Google Maps, Citymapper, OsmAnd, Organic Maps or Yandex Maps and it stays on the glasses as one live activity: the next turn and its distance, or the walk to the stop, the line to board and the stops left. The moment that matters — a turn a few metres away, the stop to get off at — beats once so it is not missed.

    Under the hood

    Nexus routes nothing for those apps and draws no map for them: the plugin reads the guidance they already post as notifications, so it follows whatever route you chose in them, as text. Since 0.3.0 Navigation also plans walks itself, with a map on the route page and on the home screen. The activity became one sprung outline for it, with a line badge, a second quantity beside the value, a row of dots for stops and an urgent beat, all in SDK 0.21.0 and open to any plugin.

  • 1.4.13

    Reply inside the notice

    Tap Type on a Relay notice and the notice itself becomes the reply field: what you type on the phone appears under the message as you type it. The phone keyboard opens on its own when the glasses ask for text, and closes when you send.

    Under the hood

    The band is an overlay that never takes focus, so it cannot hold a text field. The real field stays on the glasses, out of sight, and the band draws a local copy of its text and caret; nothing crosses the bus before the single commit. Since 1.5.0 the app behind stays in view while you type. Plugins opt in with EditableSurfaceField.inNotice in SDK 0.20.0.

  • 1.4.12

    The glasses only talk to your phone

    The Bluetooth channel between the two hubs used to accept any device in range. Now each end proves it holds a key the phone enrolled over the authorized Rokid link, and every frame carries a signature and a counter, so nothing can be injected, replayed, or read by a stranger's phone.

    Under the hood

    Contributed by Mike Sviblov. A four-record HMAC-SHA256 handshake derives one key per direction; the pairing key lives in the Android Keystore on both sides and follows the glasses' CXR identity. Both hubs must be on 1.4.12: an older peer is refused and only the Rokid control link remains until it is updated.

  • 1.4.11

    A notice answers the question you saw

    Replies, gestures and dismissals are bound to the notice and the question on screen, so a late tap can never act on its replacement. A notice keeps the keys over whatever is underneath, and a reply the link rejected says Delivery not confirmed instead of risking a second send.

    Under the hood

    Notice protocol v5 needs both hubs on 1.4.11; plugins rebuilt on SDK 0.19.0 also filter stale callbacks already queued. The phone hub now hands notices, pins, activities, surfaces and Ink to separate routers, and an SPP drop no longer silences a live notice's callbacks.

  • 1.4.10

    Listening with the display asleep

    A plugin holding the microphone can let its surface go and keep listening. The display sleeps, the launcher is free, and the phone says Listening in the background with a Stop beside it.

    Under the hood

    Not a background-work permission: without an open audio lease there is nothing to keep alive. Releasing, revoking or losing the lease always ends it; reopening the plugin resumes it.

  • foundation

    The bus, and the identity it enforces

    Any APK may bind to the hub; installing one grants it nothing. A plugin is package + plugin id + signing certificate, and each capability is a separate grant, checked at the hub on every message.

    Under the hood

    Ten capabilities today — surfaces, ink_surface, microphone, stt, tts, camera, http_proxy, mediasync, assistant, wireless_debugging. A plugin does not have to be running: the hub binds it awake when traffic arrives, measured at ~1.6 s including cold start.

  • foundation

    One link, two paths, and the hub picks

    Control messages ride CXR-L; anything binary — images, photo sync — goes over SPP. A plugin never chooses, and never learns which path its bytes took.

    Under the hood

    1.1.1 flipped that order to route around a link that reported sends it had not delivered, and 1.1.2 flipped it back: SPP is one RFCOMM channel with one write lock, so a control message queued behind a photo chunk waits for the whole chunk. The real fix is an acknowledgement, not a different running order.

  • foundation

    Setup without a computer

    Seven steps on the phone, the glasses app pushed over the Rokid link, then a two-card self-arm on the glasses. No cable, no PC, no ADB at any point.

    Under the hood

    Wireless Debugging self-pairing with an app-private KADB TLS identity and a detached watchdog — the classic ADB key is never touched, so nothing on a PC is ever enrolled. Navigation reads the firmware's own localized labels, so it works in every language the ROM ships.

  • foundation

    The distribution loop, closed

    The SDK publishes to JitPack from sdk-v* tags, a manifest PR lands a plugin in the public registry, and the in-app Store verifies SHA-256 and signer before the install runs.

    Under the hood

    Provenance is checked in a fixed order — commit, tag, build, publish. Everything self-updates afterwards: phone from releases, glasses over CXR, plugins from the Store. Since 1.4.7 the tests of both hubs and every plugin run on each push, and a release that fails them is not published.

  • foundation

    All five display tiers, and the motion under them

    Ambient, pin, activity, notice, surface — plus a shared glyph set, plugin marks sent as bare geometry, and a status row along the bottom of every plugin surface.

    Under the hood

    Three duration tokens and two interpolators, nothing a plugin can dial. Native Views, not a WebView: the WebView spike rendered the same motion for ~1.2 cores, +88 MB PSS and 2.2 s to first paint, against 7.7 % CPU. Activities moved to springs in 1.5.0. The status row — phone charge, date, glasses charge — arrived in 1.4.8. Only a notice, or an activity's significant update that asked for it at start, may pulse a dark display awake, at most once every five seconds across every plugin, and never held on.

  • foundation

    The camera capability

    The glasses stream live H.264 and the plugin decodes on the phone, where ML Kit runs OCR and translation offline. No glasses-side plugin code exists.

    Under the hood

    A phone app cannot switch its own Wi-Fi on, which used to be a dead end. Now the roles invert — the phone hosts a LocalOnlyHotspot and the glasses join it — and the wire protocol, the decode, the overlays and freeze are unchanged.

  • 1.4.9

    Speech, in both directions

    stt takes words from your mouth, tts says them back with your phone's voice — through the glasses or your earbuds, never the phone's own speaker. How long it waits for you is now a setting.

    Under the hood

    Voice and speed are picked once, for everything that speaks. Opening the microphone silences speech, or the glasses would record their own voice into your dictation. Patience covers both the wait for the first word and the pause that ends a sentence; the Android recognizer only starts once you do.

  • 1.4.1

    Ink Surface

    A plugin sends a strict subset of Rokid's .ink format; the phone compiles it and the glasses draw it as native Views. Charts, progress, tap actions, inline Lottie — no WebView, no JavaScript, no page-side network.

  • 1.4.2

    The phone as a remote for native glasses apps

    List and open the apps already on the glasses, move focus, type into their fields, and drag the system's own pointer like a trackpad — in the Rokid launcher and in any third-party app.

    Under the hood

    The four /core/* protocol families are hub-only and replay-safe; no plugin grant reaches them. Sensitive editors secure the phone window, and existing field contents never cross back to the phone.

  • 1.4.6

    Typed input for plugins

    A card can carry one text field. A keyboard bonded to the glasses works, and so does the phone's Keyboard & remote screen. Relay replies and Assistant notes came first. Built by ruruw.

  • 1.4.8

    The assist button, on your terms

    A plugin can replace the glasses assistant without owning the button: hand it back to Rokid's assistant and the plugin stays one pick away in the launcher. Plugins are told why they were opened.

Deferred

Designed, and parked on purpose until a concrete case asks for them.

  • Ownership epochs

    Foreground ownership, per-surface ordering, sequenced hides and image-decode invalidation already protect the normal handoff, and "newest notice wins" is a deliberate single-slot policy, not a missing queue.

    The per-plugin floating modes and the per-app preference shipped in 2.0. Hub-stamped epochs and priority arbitration stay on development branches; they come back against a reproduced scenario, not as a display rework on principle.

  • Continuous speech

    Speech to text ships in short takes. Since 1.4.10 a microphone lease can outlive its surface, but that is not transcription that runs for minutes.

    The missing slice is speech held open with partial results streaming to the HUD, and captions that survive the surface underneath changing. Live captions and translation need it; it waits for the first consumer that does.

Exploring

Directions, not commitments. Scope and timing are open.

  • Public-transport journeys in Navigation

    Which bus, metro or train to take, departures, transfers and stops, planned by Navigation and shown through the same route pages, maps and floating guidance as a walk. Today it plans walks only, and transit guidance read from other apps is text.

Never

Non-goals, stated once so nobody has to ask twice.

  • Native glasses apps are launched, never absorbed

    Porting them into Nexus would make the platform responsible for software it did not write. The menu starts them; that is the whole relationship.

  • Plugin code stays on the phone

    The glasses half of any capability lives in the hub. That is exactly what makes zero glasses-side deployment possible, and it is not negotiable.

  • No signature-only plugin permission

    It would be the strongest trust model available, and it would also make third-party plugins impossible — no external developer can be signed with our key.

  • One host, one platform, for now

    Hi Rokid Global and Android. Anything else is a promise we cannot currently keep on hardware we can test.

Five ways to be on the display

A plugin never picks how loudly it speaks. It declares what kind of thing it has, and the platform decides the form, the motion, and whether it may interrupt at all. Below is that renderer, reproduced — same phosphor, same 480×640 optic, same durations.

enter 280 ms · fast-out-slow-in

Timings are the platform's, not the plugin's: 180 ms for a value refreshing in place, 280 ms for anything arriving or changing shape, 240 ms for anything leaving — exits are quicker than entrances. It runs on native Views because the WebView that rendered the same motion cost ~1.2 cores, +88 MB and 2.2 s to first paint, against 7.7 % CPU here. The notice band's six states →

Thirteen plugins, none of them built in

Every one is an ordinary phone APK installed from the in-app Store, pinned by hash and signer before it installs. The hub itself ships empty. These are real captures off the optic, uncropped; only Tasker's is redrawn.

A Relay notice band over the glasses home screen showing a two-message thread with a Reply action
RelayA message arrives as a band over whatever you were looking at, and is answered out loud — or typed from the phone.
The Relay inbox listing conversations still waiting for a reply
Relay · MessagesEverything you let go, newest first, answered the same way.
Lens translating a screen of English text into French live on the HUD
Lens · liveThe camera streams to the phone; OCR and translation run there, offline, and the overlays come back.
Lens freeze mode showing a full-resolution still with the translated result
Lens · freezeA full-FOV still through the same link, for text you want to hold on to.
The Transit plugin showing nearby stops and live departure countdowns
TransitNearby stops and live departures, with the ones you actually take kept on top — and Assistant can ask it for you.
The Feeds plugin showing a timeline post on the glasses HUD
FeedsBluesky and X timelines — posts, threads, and the actual photos full-screen.
The Media Deck plugin showing album art, track title, and transport controls
Media DeckNow playing with album art and transport, from whatever app is playing.
The Lyrics plugin showing the current line time-synced to the playing track
LyricsTime-synced lines for whatever is playing, from four sources.
An Assistant answer in a band at the top of the glasses home screen: "La capitale du Japon est Tokyo."
AssistantAsk out loud, hear the answer through your phone's own voice — and the model can look through the camera when the question needs eyes. It remembers too: reminders, timers and notes, set by voice. It answers from a folder of your documents, PDFs included, and asks Transit or Media Deck when you let it. Your ChatGPT plan, or any AI you bring a key for.
Navigation's full-screen map on the glasses: a walking route drawn in green over the streets, on a simulated position
NavigationPlans a walk and draws it: on its card, full screen, or as a minimap floating on home. Routes from Google Maps, Citymapper, OsmAnd, Organic Maps or Yandex Maps follow as text: the next turn, the line to board, the stops left.
deprecated Photos Sync Copied the photos and videos you shoot on the glasses into your phone gallery. The Hi Rokid app now does this itself, so Photos Sync is no longer developed.
YouTube on the glasses, one video per card, with next, open and sections hints for the touchpad
Patcher · YouTubePatched on your phone for the touchpad and the R08 ring, then checked and installed on the glasses by Nexus.
Reddit Preview on the glasses, a green menu of Home, Popular, Latest, Communities, Search and Inbox
Patcher · RedditA Preview: its own green HUD for posts and comments. Replying has not been checked on real posts yet.
Tasker's card on the glasses: a list of Tasker tasks with their projects, the first one selected
TaskerYour named Tasker tasks, listed on the glasses. Swipe to the one you want, tap, and it runs on the phone. Redrawn from the card it sends, in the glasses' own layout; not a capture.
for developers Wireless ADB Turns on Android's real wireless debugging on the glasses and hands you a short-lived pairing command, so a trusted computer connects without a cable.
The Hello Nexus sample's Groceries checklist open as a page on the glasses, with checked and unchecked items and Open app
Sample · Hello NexusThe copyable plugin: a checklist page and an activity card through the public SDK. Clone it, change the id, and you have a glasses app that never runs on the glasses.

Installation alone grants nothing. Surfaces, Ink, microphone, speech to text, text to speech, camera, photos, HTTP proxy, the assistant slot, skills — down to each operation — and wireless debugging are approved one by one, per plugin, keyed to its package, its plugin id, and its signing certificate — and checked at the hub on every message, not at install time.

Adding

The released plugins are not finished. What each one still gains.

  • Assistant

    plugin operations asked by voice and on a moving journey · Workspace on large folders, found by meaning rather than by words

  • Navigation

    OsmAnd walking and cycling checked on real trips · bus, tram and RER rides checked on real trips · Google Maps and Citymapper in languages beyond English, French and Korean · Korean checked on a real trip

  • Feeds

    posting and replying by voice · more sources · video in the timeline

  • Transit

    “take me home” guidance checked on a real trip

  • Media Deck

    “next” asked of Assistant · pause by voice, with real music players

  • Patcher

    Reddit out of Preview: the R08 ring and real posting checked

Coming

Plugins that do not exist yet, in order. Two rows have already made this crossing: “a voice assistant” sat here and shipped as Assistant, and Navigation shipped in 0.1.0.

  • Agents, as a private alpha

    Claude Code, Codex and OpenClaw sessions on the HUD: permission prompts as notices, progress on a pin, the next task dictated. Already in the tree, being reworked to speak Alleycat's protocol — the daemon behind litter — instead of a hand-rolled one. Not Store-listed until that rework lands.

Ideas

Not committed.

  • A visual assistant, FoodFacts

    camera capability, shipped

  • Sport HUD

    activity tier + a small protocol addition · possibly fed by the R08 ring

Get started

Running it takes a phone and four steps. Writing for it takes one Gradle coordinate.

Run it

  1. Install the phone app. One APK from the releases page — it is the hub, the Store, and the setup guide.
  2. Follow the seven steps. The phone drives the pairing, turns the glasses' Wi-Fi on, and pushes the glasses app over the Rokid link.
  3. Say yes once, on the glasses. Enable accessibility when asked; Nexus bootstraps the rest of its glasses-side setup itself.
  4. Install a plugin from the Store. Everything keeps itself current afterwards.

No cable, no PC, no ADB at any point — including the privileged parts, which the glasses arm by themselves in two cards.

Write for it

  1. Depend on the SDK. :bus-client brings the service base class, the typed surfaces, the lifecycle callbacks and the design kit.
  2. Declare who you are. A headless service, a plugin id, a 24×24 mark in one green, and the capabilities you need — no launcher icon.
  3. Push a surface. Describe content, never pixels. Your process does not need to be running: the hub binds you awake.
  4. Publish, optionally. You host the APK, the registry indexes it, the Store pins your hash and your signer.
build.gradle.kts
repositories { maven("https://jitpack.io") }

dependencies {
    implementation(
        "com.github.Anezium.Rokid-Nexus:bus-client:sdk-v0.24.0"
    )
}

Every limit in the contract is enforced in code — breaking one does not degrade gracefully. All documentation →

Support Nexus

Nexus is free and Apache-2.0. If it is useful to you, Ko-fi and GitHub Sponsors help pay for the hardware and the time behind the next release.