Git-native DAT SDK guidance for android development, covering setup, camera streaming, display access, MockDeviceKit, session lifecycle, permissions, debugging, and sample app patterns.

version
1.0.1
GitHub stars
406

Install

/plugin install mwdat-android@mwdat-android-marketplace

Run it inside Claude Code after adding the marketplace with /plugin marketplace add meta/meta-wearables-dat-android.

Contents

Discovered from source

Skills 18

Audio Streaming (Android)

Audio Streaming is an experimental extension of Camera Stream, not a separate capability. Follow the camera-streaming skill for session and Camera setup, then add audio configuration and collection.

Permissions

The app needs Camera and Audio Streaming approval in Wearables Developer Center plus both Permission.CAMERA and Permission.MICROPHONE. Check first and launch permission requests only after a user action.

Configure PCM audio

Set a non-null codec in StreamConfiguration:

val configuration = StreamConfiguration(
    audioCodec = AudioCodec.PCM(
        sampleRate = AudioSampleRate.RATE_16000,
        numberOfChannels = 1,
    ),
    videoQuality = VideoQuality.MEDIUM,
    frameRate = 24,
)

Pass this configuration to addCamera using the Camera Streaming flow. Then collect audio before starting its Stream:

audioJob = lifecycleScope.launch(Dispatchers.IO) {
    camera.stream.audioStream.collect { frame ->
        consumePCM(frame.buffer, frame.presentationTimeUs)
    }
}
camera.stream.start().onFailure { error, _ -> showError(error.description) }

Supported sample rates are 16,000, 44,100, and 48,000 Hz. Configure downstream processing with the same rate and channel count. Use presentationTimeUs to preserve order or align audio with video. Process PCM off the main thread because slow collectors can drop frames.

Audio shares Stream state and errors. A microphone denial is surfaced as a stream permission failure; do not model a separate audio lifecycle.

Cleanup and testing limits

Cancel audio and any other Stream collectors before calling camera.stop(). Stopping Camera cascades to its Stream, and the instances cannot be restarted.

MockDeviceKit can negotiate an audio-enabled Camera stream, but it does not provide a public deterministic audio-frame injection API. Do not assume mock PCM frames will arrive.

Availability

Audio Streaming is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

Camera Capture (Android)

Camera Capture is the experimental standalone high-quality photo path exposed by camera.photo. It is different from stream.capturePhoto(), which takes a photo while video streaming remains active.

Prepare Camera and permission

The app needs Camera approval in Wearables Developer Center and Permission.CAMERA. Add Camera only after DeviceSessionState.STARTED. Stream and standalone Photo compete for camera hardware, so stop an active Stream before accessing Photo.

Observe Photo

Accessing camera.photo lazily creates and starts the standalone Photo child. Collect its flows and wait for PhotoState.STARTED before enabling capture.

val photo = camera.photo

photoStateJob = lifecycleScope.launch {
    photo.state.collect { state ->
        photoButtonEnabled = state == PhotoState.STARTED
    }
}
photoDataJob = lifecycleScope.launch {
    photo.photoStream.collect { capture ->
        save(capture.imageData, capture.metadata)
    }
}
progressJob = lifecycleScope.launch {
    photo.transferProgressStream.collect { progress -> updateProgress(progress) }
}
errorJob = lifecycleScope.launch {
    photo.errors.collect { error -> showError(error.description) }
}

After Photo reaches PhotoState.STARTED, call photo.capturePhoto(resolution = PhotoResolution.FULL, quality = PhotoQuality.HIGH) with the desired resolution and quality. Capture is asynchronous; data arrives through photoStream. Collect transferProgressStream for progress and errors for failures.

Errors and cleanup

Handle not-ready, capture, setup, disconnect, busy, service, permission, and device-health errors from errors. After delivery, stop Photo, cancel its collectors, and stop Camera.

Photo is terminal after stop(), and accessing camera.photo again returns the same stopped instance. To capture again, remove the current Camera from the session, add a new Camera, and access Photo from that new instance. Stopping Camera or the parent session also cascades cleanup.

Test with MockDeviceKit

Configure the next image with glasses.services.cameraCapture.setCapturedPhoto(imageUri) or use simulateCaptureFailure() for a one-shot failure.

Availability

Standalone Camera Capture is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

Camera Streaming (Android)

Use a DeviceSession and an attached Camera to receive frames and capture photos through camera.stream.

For PCM audio delivered with this stream, use the audio-streaming skill.

Key concepts

  • DeviceSession: Device connection lifecycle created through Wearables.createSession(...)
  • Camera: Camera capability attached to a session with session.addCamera(...)
  • Stream: Video stream accessed through camera.stream
  • StreamConfiguration: Video quality, frame rate, and compression configuration for the stream
  • PhotoData: Still image captured from glasses while streaming

Create a session and attach a stream

DeviceSession.start() is fire-and-forget: it returns Unit and the connection completes in the background. A capability can only be added once the session reports DeviceSessionState.STARTED — calling addCamera(...) right after start() fails with DeviceSessionError.SESSION_IDLE. Add the camera from the session-state collector.

import com.meta.wearable.dat.camera.Camera
import com.meta.wearable.dat.camera.addCamera
import com.meta.wearable.dat.camera.types.StreamConfiguration
import com.meta.wearable.dat.camera.types.VideoQuality
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector
import com.meta.wearable.dat.core.session.DeviceSessionState

var camera: Camera? = null

Wearables.createSession(AutoDeviceSelector()).fold(
    onSuccess = { session ->
        lifecycleScope.launch {
            session.errors.collect { error -> showError(error.description) }
        }
        lifecycleScope.launch {
            session.state.collect { state ->
                if (state == DeviceSessionState.STARTED && camera == null) {
                    session.addCamera(
                        StreamConfiguration(
                            videoQuality = VideoQuality.MEDIUM,
                            frameRate = 24,
                        ),
                    ).fold(
                        onSuccess = { addedCamera ->
                            camera = addedCamera
                            addedCamera.stream.start().onFailure { error, _ ->
                                showError(error.description)
                            }
                        },
                        onFailure = { error, _ -> showError(error.description) },
                    )
                }
            }
        }
        // Subscribe before start() so no initial transition is missed.
        session.start()
    },
    onFailure = { error, _ -> showError(error.description) },
)

Check Wearables.checkPermissionStatus(Permission.CAMERA) before starting the stream; see the permissions-registration skill.

Resolution options

QualitySize
VideoQuality.HIGH720 x 1280
VideoQuality.MEDIUM504 x 896
VideoQuality.LOW360 x 640

Frame rate options

Valid values: 2, 7, 15, 24, 30 FPS.

Lower resolution and frame rate usually produce better visual quality per frame over Bluetooth.

Observe stream state

StreamState transitions: STOPPED -> STARTING -> STARTED -> STREAMING -> STOPPING -> STOPPED, and CLOSED once the stream is terminal. PAUSED is reported when the device pauses the stream, for example on a single cap-touch tap; the stream can resume on its own from PAUSED.

lifecycleScope.launch {
    camera.stream.state.collect { state ->
        when (state) {
            StreamState.STREAMING -> {
                // Frames are flowing
            }
            StreamState.PAUSED -> {
                // Paused by the device; wait for it to resume
            }
            StreamState.STOPPED -> {
                // Streaming ended
            }
            StreamState.CLOSED -> {
                // Stream fully closed
            }
            else -> Unit
        }
    }
}

Observe camera.stream.errorStream alongside the state. StreamError.STREAM_ERROR is informational and does not stop the stream, while StreamError.CRITICAL_STREAM_ERROR means the stream should be torn down.

lifecycleScope.launch {
    camera.stream.errorStream.collect { error ->
        showStreamError(error.description)
    }
}

Receive frames

lifecycleScope.launch {
    camera.stream.videoStream.collect { frame ->
        updatePreview(frame)
    }
}

By default the SDK decodes on the phone and VideoFrame.buffer holds YUV pixel data. Set StreamConfiguration(compressVideo = true) to receive compressed HEVC buffers instead; then check frame.isCompressed and frame.isCodecConfig and feed the frames to your own decoder. Keep frame handling off the main thread for anything heavier than a buffer copy.

In-stream photo capture

This path captures while video streaming remains active. For standalone high-quality capture with resolution, quality, transfer progress, and its own lifecycle, use the camera-capture skill.

capturePhoto() only succeeds while the stream is active, and returns a PhotoData sealed type — branch on the variant instead of reading a single data property.

import com.meta.wearable.dat.camera.types.PhotoData

lifecycleScope.launch {
    camera.stream.capturePhoto()
        .onSuccess { photoData ->
            when (photoData) {
                is PhotoData.Bitmap -> savePhoto(photoData.bitmap)
                is PhotoData.HEIC -> saveHeic(photoData.data) // ByteBuffer of HEIC bytes
            }
        }
        .onFailure { error, _ ->
            showCaptureError(error.description)
        }
}

Only one capture can be in flight at a time; a second concurrent call fails with CaptureError.CaptureInProgress.

Clean up

Stop the camera when you no longer need camera data, then stop the parent session if the device interaction is finished. Stopping the camera cascades to its stream, and stopping the session cascades to every attached capability.

camera.stop()
session.stop()

Camera.stop() and Stream.stop() invalidate the instance — they cannot be restarted. Call session.removeCamera() to detach the capability so a later session.addCamera(...) on the same session can succeed.

Links

DAT SDK Conventions (Android)

Quick Reference

TaskCommand
Build app./gradlew assembleDebug
Run tests./gradlew test
Install app./gradlew installDebug
Lint app./gradlew lint

Architecture

The SDK is organized into four public modules:

  • mwdat-core: Registration, permissions, devices, and session creation
  • mwdat-camera: Camera capability, video frames, and photo capture
  • mwdat-display: Display capability, display UI components, icons, images, buttons, and video
  • mwdat-mockdevice: MockDeviceKit for testing without hardware

Initialization and session setup

DeviceSession.start() is fire-and-forget and returns Unit; capabilities can only be attached once the session reports DeviceSessionState.STARTED.

Wearables.initialize(context)

var camera: Camera? = null

Wearables.createSession(AutoDeviceSelector()).fold(
    onSuccess = { session ->
        lifecycleScope.launch {
            session.state.collect { state ->
                if (state == DeviceSessionState.STARTED && camera == null) {
                    session.addCamera(StreamConfiguration()).fold(
                        onSuccess = { addedCamera ->
                            camera = addedCamera
                            addedCamera.stream.start().onFailure { error, _ ->
                                showError(error.description)
                            }
                        },
                        onFailure = { error, _ -> showError(error.description) },
                    )
                }
            }
        }
        lifecycleScope.launch {
            session.errors.collect { error -> showError(error.description) }
        }
        session.start()
    },
    onFailure = { error, _ -> showError(error.description) },
)

Kotlin patterns

  • Use DatResult<T, E> for typed success and failure handling
  • Prefer fold, onSuccess, and the two-parameter onFailure { error, cause -> } overload, which hands you the typed DatError with a description
  • getOrElse { } receives a raw Throwable, not a typed DatError, so it cannot read error.description
  • Observe state with StateFlow and Flow
  • Create a DeviceSession first, then attach capabilities such as Camera or Display after it reaches STARTED
  • Keep frame handling off the main thread when doing heavier processing

Error handling

lifecycleScope.launch {
    Wearables.checkPermissionStatus(Permission.CAMERA)
        .onSuccess { status -> /* handle status */ }
        .onFailure { error, _ -> /* handle error */ }
}

Avoid getOrThrow() in user-facing samples. Surface typed errors from DatResult instead.

Naming conventions

TypePurposeExample
DeviceSessionDevice connection lifecycleWearables.createSession(...)
CameraCamera capability on a sessionsession.addCamera(...)
StreamVideo stream from a cameracamera.stream
DisplayDisplay capability on a sessionsession.addDisplay(...)
*SelectorDevice targetingAutoDeviceSelector, SpecificDeviceSelector
*ErrorTyped failure surfaceDeviceSessionError, StreamError, CaptureError

Key types

  • Wearables — SDK entry point
  • DeviceSession — lifecycle for an interaction with a linked device
  • Camera — camera capability attached to a session
  • Stream — video stream accessed through camera.stream
  • Display — display capability attached to a session
  • StreamConfiguration — video quality, frame rate, and compression configuration
  • MockDeviceKit — simulated device environment for testing

Live docs search

If your editor supports remote MCP servers, connect https://mcp.developer.meta.com/wearables and use search_dat_docs for current DAT setup, session lifecycle, camera streaming, MockDeviceKit, permissions, and exact API symbols. This public docs server does not require authentication; do not configure tokens, OAuth, or custom authorization headers for it.

See the dat-docs-mcp skill for client setup and troubleshooting. Use llms.txt when your tool only supports static reference context.

Testing with MockDeviceKit

val mockDeviceKit = MockDeviceKit.getInstance(context)
mockDeviceKit.enable()
mockDeviceKit.pairGlasses(GlassesModel.RAYBAN_META).fold(
    onSuccess = { device -> onMockDevicePaired(device) },
    onFailure = { error, _ -> showError(error.description) },
)

Use MockDeviceKit to drive registration, device availability, streaming media, and permission scenarios without physical hardware.

Common pitfalls

  • Do not call SDK APIs before Wearables.initialize(context)
  • Do not call addCamera(...) or addDisplay() immediately after session.start(); wait for DeviceSessionState.STARTED
  • Do not assume a session implies streaming or display access; capabilities are attached separately
  • Do not ignore DatResult failures from createSession, addCamera, addDisplay, stream.start(), or capturePhoto; session.start() returns Unit, so watch session.errors for its failures
  • Do not reuse terminally stopped sessions, cameras, or streams

Links

DAT Docs MCP (Android)

Use this skill whenever a user asks about the DAT MCP server, live docs search, current DAT documentation, API lookup, code examples, MCP setup, or MCP Inspector.

This is the documentation counterpart to live-debugging-mcp: use this skill to look up published DAT documentation, and use live-debugging-mcp for observed runtime behavior in a running app.

Endpoint

  • MCP endpoint: https://mcp.developer.meta.com/wearables
  • Transport: HTTP (Streamable HTTP)
  • Authentication: none. Do not configure tokens, OAuth, or custom Authorization headers for this server.

Connect to the MCP host above directly. Do not point an MCP client at the Wearables developer website URL; that is a docs site, not the MCP endpoint.

Tools

  • search_dat_docs — semantic search over DAT guides, API reference, and code examples.

The server may add more DAT MCP tools over time, so list the tools after connecting instead of assuming this is the only one.

Repo config vs. MCP

  • Use the repo-local AI config (these skills, .cursor/rules/, .github/copilot-instructions.md, AGENTS.md) for coding patterns, setup steps, and project conventions.
  • Use the MCP endpoint for live DAT documentation and exact current API symbols, especially when a symbol may have changed since this repo config was written.

Setup: Muse Code, Claude Code, and Codex

Installing the mwdat-android plugin registers the hosted DAT docs MCP server in Muse Code, Claude Code, and Codex. No separate MCP configuration is required.

Setup: Cursor

Add an HTTP MCP server with:

  • Name: wearables-dat
  • Type/transport: HTTP
  • URL: https://mcp.developer.meta.com/wearables

For JSON-backed Cursor settings:

{
  "mcpServers": {
    "wearables-dat": {
      "type": "http",
      "url": "https://mcp.developer.meta.com/wearables"
    }
  }
}

Setup: MCP Inspector

npx @modelcontextprotocol/inspector
  1. Set transport to Streamable HTTP, or HTTP if that is the label in the installed Inspector.
  2. Set URL to https://mcp.developer.meta.com/wearables.
  3. Set connection type to Direct.
  4. Initialize, then list tools and confirm search_dat_docs appears.
  5. Run a query such as camera streaming.

Example prompts

  • Search the Wearables DAT docs for camera streaming setup on Android.
  • How do I stream camera on Ray-Ban Meta glasses?
  • How do I initialize the Wearables SDK and access a connected device?
  • How do I test DAT integrations without physical glasses?
  • Bluetooth connection lifecycle events

Troubleshooting

  • Server does not connect: re-enter https://mcp.developer.meta.com/wearables and reconnect.
  • No tools appear: reconnect and initialize the session before listing tools.
  • The client asks for auth: remove any stale custom auth headers or tokens for this server and reconnect. This server does not require authentication.
  • Results are too broad: include the platform, module, or exact API name in the query, for example Android StreamConfiguration frameRate instead of frame rate.

Static fallback

When a tool cannot use remote MCP servers at all, point it at the llms.txt index instead and follow its links: https://wearables.developer.meta.com/llms.txt.

Debugging (Android)

Diagnose common setup, session, and stream issues in DAT SDK integrations.

If local DAT Inspector MCP tools are available, use the live-debugging-mcp skill before changing app code. Prefer get_dat_readiness, get_companion_boundary_diagnosis, and get_device_path to separate app bugs from Meta AI app/device boundary issues using read-only DAT evidence.

Quick diagnosis

No eligible device or session won't start?
|
+-- Did you call Wearables.initialize(context)? -> Must happen before SDK usage
|
+-- Did registration complete? -> Observe Wearables.registrationState
|
+-- Is Developer Mode enabled? -> Enable it in the Meta AI app for dev builds
|
+-- Does Wearables.devices contain a linked device? -> Check Bluetooth and range
|
+-- Did createSession() or addCamera() return a DatResult failure? -> Surface the typed error

Developer Mode

Developer Mode must be enabled for local development builds that use mwdat_application_id = 0 and mwdat_client_token = 0.

Symptoms when Developer Mode is disabled

  • Registration flow completes but the device never becomes eligible
  • Permission requests do not succeed for development builds
  • Wearables.createSession(...) fails with no eligible device

Watch for

  • Developer Mode may reset after app or firmware updates
  • Developer Mode is configured per linked device
  • Production builds use a real APPLICATION_ID, CLIENT_TOKEN, and release-channel gating instead

Session and stream issues

Session never reaches STARTED

  • Verify Wearables.registrationState
  • Check that Wearables.devices contains a compatible linked device
  • Ensure the glasses are powered on, unfolded, and in range

Stream never reaches STREAMING

  • Call session.addCamera(...) only after session.state reports STARTED; session.start() returns Unit, so watch session.errors for start failures
  • Check camera permission status through Wearables.checkPermissionStatus(...)
  • Make sure camera.stream.start() returned success

Photo capture fails

  • capturePhoto() only succeeds while the stream is actively streaming
  • Surface the returned CaptureError instead of discarding the DatResult

Version compatibility

Ensure compatible versions of the SDK, Meta AI app, and glasses firmware. See version dependencies for the current compatibility matrix.

Logging

private const val TAG = "DATWearables"

camera.stream.start()
    .onFailure { error, _ -> Log.e(TAG, "Failed to start stream: ${error.description}") }

Prefer logging typed DatResult failures and observed state transitions over generic exceptions.

Live MCP evidence

When the developer has a local DAT debug server connected to the agent:

  • Start with get_sdk_state and get_dat_readiness
  • Use get_companion_boundary_diagnosis for permission, app identity, device capability, device-selection, deeplink-return, and DAM/DWA-visible blockers
  • Use get_device_path to trace app -> DAT registration -> Meta AI app permissions -> device selection/link -> session -> stream
  • Ask the developer to reproduce the issue, then use wait_for_events with the narrowest category or source that matches the flow
  • Use export_diagnostic_bundle for redacted support handoff

Do not treat this as companion app introspection. It is diagnosis from app-visible DAT debug events.

Checklist

  • Wearables.initialize(context) ran before SDK usage
  • Developer Mode enabled for development builds
  • APPLICATION_ID and CLIENT_TOKEN match the build mode
  • Registration completed before session creation
  • Bluetooth permission granted
  • Camera permission granted through Meta AI
  • Session and stream DatResult failures are surfaced in logs or UI

Links

Display Access (Android)

Use mwdat-display to render content on Meta Ray-Ban Display glasses.

Use this skill with getting-started and permissions-registration when creating a full app. A Display app still needs SDK initialization, app registration, Android permissions, and DAT manifest metadata before it can create a session.

Add the Display dependency

In libs.versions.toml:

[libraries]
mwdat-display = { group = "com.meta.wearable", name = "mwdat-display", version.ref = "mwdat" }

In app/build.gradle.kts:

dependencies {
    implementation(libs.mwdat.core)
    implementation(libs.mwdat.display)
}

Configure the app for Display

Display apps need the same core DAT setup as other apps:

<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.INTERNET" />

<application ...>
    <meta-data
        android:name="com.meta.wearable.mwdat.APPLICATION_ID"
        android:value="${mwdat_application_id}" />
    <meta-data
        android:name="com.meta.wearable.mwdat.CLIENT_TOKEN"
        android:value="${mwdat_client_token}" />
</application>

Set mwdat_application_id and mwdat_client_token from Gradle manifest placeholders or local.properties, as in the DisplayAccess sample. Developer Mode builds can use the developer registration flow, but production builds need real Wearables Developer Center credentials.

Request the runtime permissions before initializing DAT, then call Wearables.initialize(context) once and start observing SDK state. Start registration with Wearables.startRegistration(activity), collect Wearables.registrationState, collect Wearables.registrationErrorStream, and wait for RegistrationState.REGISTERED before creating a display session. Use Wearables.startUnregistration(activity) when the user disconnects the app.

Select a display-capable device

Display content only works on connected, compatible devices whose type supports display. Use the public device filter when you want automatic selection:

import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector

val selector = AutoDeviceSelector(filter = { device -> device.isDisplayCapable() })
val sessionResult = Wearables.createSession(selector)

Use SpecificDeviceSelector(selectedDeviceId) instead when your UI lets the user pick a specific DeviceIdentifier from Wearables.devices.

For device picker UI, mirror the DisplayAccess sample: collect Wearables.devices, start a metadata collection for each device ID from Wearables.devicesMetadata[id], remove metadata for devices that disappear, and show device name, device.deviceType.description, device.linkState, device.compatibility, and device.isDisplayCapable(). Keep selection disabled unless the device is LinkState.CONNECTED and display-capable, and surface DeviceCompatibility.DEVICE_UPDATE_REQUIRED with an openFirmwareUpdate action.

import com.meta.wearable.dat.core.types.Device
import com.meta.wearable.dat.core.types.DeviceIdentifier
import com.meta.wearable.dat.core.types.LinkState
import kotlinx.coroutines.Job
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.update

private val metadataJobs = mutableMapOf<DeviceIdentifier, Job>()
private val devicesMetadata = MutableStateFlow<Map<DeviceIdentifier, Device>>(emptyMap())

lifecycleScope.launch {
    Wearables.devices.collect { deviceIds ->
        (metadataJobs.keys - deviceIds).forEach { removedId ->
            metadataJobs.remove(removedId)?.cancel()
            devicesMetadata.update { current -> current - removedId }
        }

        deviceIds.forEach { id ->
            metadataJobs.getOrPut(id) {
                launch {
                    Wearables.devicesMetadata[id]?.collect { device ->
                        devicesMetadata.update { current -> current + (id to device) }
                        updateDeviceRow(
                            id = id,
                            name = device.name,
                            type = device.deviceType.description,
                            isConnected = device.linkState == LinkState.CONNECTED,
                            isDisplayCapable = device.isDisplayCapable(),
                            compatibility = device.compatibility,
                        )
                    }
                }
            }
        }
    }
}

If Wearables.createSession(...) fails with DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED, show a DAT glasses app update action that calls Wearables.openDATGlassesAppUpdate(activity).

Attach Display after the session starts

Add the Display capability only after the DeviceSession reaches DeviceSessionState.STARTED. Keep the sample app pattern of showing a "preparing" state after session start and enabling user content only once the Display capability reports DisplayState.STARTED.

import androidx.lifecycle.lifecycleScope
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.types.DeviceIdentifier
import com.meta.wearable.dat.core.types.DeviceSessionError
import com.meta.wearable.dat.core.types.RegistrationState
import com.meta.wearable.dat.core.session.DeviceSession
import com.meta.wearable.dat.core.session.DeviceSessionState
import com.meta.wearable.dat.core.selectors.SpecificDeviceSelector
import com.meta.wearable.dat.display.Display
import com.meta.wearable.dat.display.addDisplay
import com.meta.wearable.dat.display.removeDisplay
import com.meta.wearable.dat.display.types.DisplayState
import kotlinx.coroutines.launch

private var display: Display? = null

fun startDisplaySession(selectedDeviceId: DeviceIdentifier) {
    if (Wearables.registrationState.value != RegistrationState.REGISTERED) {
        showError("Register with Meta AI before starting Display")
        return
    }

    Wearables.createSession(SpecificDeviceSelector(selectedDeviceId))
        .fold(
            onSuccess = { session ->
                lifecycleScope.launch {
                    session.errors.collect { error ->
                        if (error == DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED) {
                            showDatAppUpdateAction()
                        }
                        showError(error.description)
                    }
                }
                lifecycleScope.launch {
                    session.state.collect { state ->
                        if (state == DeviceSessionState.STARTED && display == null) {
                            attachDisplay(session)
                        }
                    }
                }
                session.start()
            },
            onFailure = { error, _ ->
                if (error == DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED) {
                    showDatAppUpdateAction()
                }
                showError(error.description)
            },
        )
}

private fun attachDisplay(session: DeviceSession) {
    session.addDisplay()
        .fold(
            onSuccess = { newDisplay ->
                display = newDisplay
                lifecycleScope.launch {
                    newDisplay.state.collect { state ->
                        if (state == DisplayState.STARTED) {
                            setTryItEnabled(true)
                        } else {
                            setTryItEnabled(false)
                        }
                    }
                }
            },
            onFailure = { error, _ ->
                showError(error.description)
            },
        )
}

fun stopDisplaySession(session: DeviceSession) {
    session.removeDisplay().onFailure { error, _ -> showError(error.description) }
    session.stop()
    display = null
}

Send display UI

Build exactly one root view per sendContent call. Each call replaces the previous content on the glasses. Use a root flexBox { ... } for UI, or a root video(player = player) for video; do not put video(...) inside a flexBox.

Button and clickable flexBox callbacks are routed back to the phone app. Keep callbacks fast and delegate to your ViewModel to send the next content screen, as the DisplayAccess sample does for list items, Back, Start, Previous, Next, Done, and Watch video buttons.

import com.meta.wearable.dat.display.views.ButtonStyle
import com.meta.wearable.dat.display.views.FlexBoxBackground
import com.meta.wearable.dat.display.views.IconName
import com.meta.wearable.dat.display.views.TextColor
import com.meta.wearable.dat.display.views.TextStyle

private suspend fun sendStatusCard(display: Display) {
    display.sendContent {
        flexBox(
            gap = 12,
            padding = 24,
            background = FlexBoxBackground.CARD,
            onClick = { showDetailState() },
        ) {
            text("Bike ride", style = TextStyle.HEADING)
            text("Turn right in 200 ft", style = TextStyle.BODY, color = TextColor.SECONDARY)
            button(
                label = "Done",
                style = ButtonStyle.PRIMARY,
                iconName = IconName.CHECKMARK,
                onClick = { showDoneState() },
            )
        }
    }.onFailure { error, _ ->
        showError(error.description)
    }
}

Use images and built-in icons

Use HTTPS image URLs for image content, and use IconName enum values for built-in icons. Do not invent string icon names.

import com.meta.wearable.dat.display.views.CornerRadius
import com.meta.wearable.dat.display.views.IconStyle
import com.meta.wearable.dat.display.views.ImageSize

display.sendContent {
    flexBox(gap = 8, padding = 24) {
        image(
            uri = "https://example.com/thumbnail.png",
            sizePreset = ImageSize.FILL,
            cornerRadius = CornerRadius.MEDIUM,
        )
        icon(name = IconName.GEAR, style = IconStyle.FILLED)
        text("Device settings", style = TextStyle.BODY)
    }
}

Send video

For URL-based video, create a VideoPlayer, send it as the root content, then call play() after the send succeeds. Collect both player.state and player.error; on VideoPlayerState.ENDED, cancel the video observer and send the next display screen. Use an HTTP or HTTPS VideoSource.Url, VideoCodec.MP4 for MP4 assets, and a video sized within the public player limits.

import com.meta.wearable.dat.display.types.VideoCodec
import com.meta.wearable.dat.display.types.VideoPlayerState
import com.meta.wearable.dat.display.types.VideoSource
import com.meta.wearable.dat.display.views.VideoPlayer
import kotlinx.coroutines.Job

private var videoStateJob: Job? = null
private var videoErrorJob: Job? = null

lifecycleScope.launch {
    val currentDisplay = display ?: return@launch
    val player = VideoPlayer(
        source = VideoSource.Url("https://example.com/tutorial.mp4"),
        codec = VideoCodec.MP4,
    )

    videoStateJob?.cancel()
    videoErrorJob?.cancel()

    videoStateJob = launch {
        player.state.collect { state ->
            if (state == VideoPlayerState.ENDED) {
                videoStateJob?.cancel()
                videoStateJob = null
                sendStatusCard(currentDisplay)
            }
        }
    }
    videoErrorJob = launch {
        player.error.collect { error ->
            if (error != null) {
                showError(error.description)
            }
        }
    }

    currentDisplay.sendContent { video(player = player) }
        .onSuccess { player.play() }
        .onFailure { error, _ -> showError(error.description) }
}

Display rules

  • Initialize DAT and complete registration before creating the session.
  • Observe Wearables.registrationErrorStream, session.errors, Display.sendContent failures, and VideoPlayer.error; DeviceSession.start() returns Unit and reports async failures through session.errors.
  • Use SpecificDeviceSelector for a user-picked device and AutoDeviceSelector(filter = { it.isDisplayCapable() }) only when automatic selection is acceptable.
  • Wait for DeviceSessionState.STARTED before calling session.addDisplay().
  • Wait for DisplayState.STARTED before enabling or sending user-triggered content.
  • If device metadata reports DeviceCompatibility.DEVICE_UPDATE_REQUIRED, offer Wearables.openFirmwareUpdate(activity).
  • If session creation or session.errors reports DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED, offer Wearables.openDATGlassesAppUpdate(activity).
  • Keep display callbacks fast; move app state changes back onto your UI layer as needed.
  • Cancel state/error collection jobs, close or replace active VideoPlayer instances, detach with session.removeDisplay(), and then stop the session when the display experience ends.

Sample app

Use the Display Access sample app for a complete flow: registration, device selection, display attachment, interactive content, and video.

Links

Getting Started with DAT SDK (Android)

Set up the Meta Wearables Device Access Toolkit in an Android app.

Prerequisites

  • Android Studio Flamingo or newer
  • Kotlin, Android Gradle Plugin and JDK versions matching the CameraAccess sample or newer (see its gradle/libs.versions.toml and app/build.gradle.kts)
  • Android 10+ test device with the Meta AI app installed
  • Supported Meta glasses or MockDeviceKit for local testing
  • Developer Mode enabled in the Meta AI app for development builds

Step 1: Declare dependencies

The SDK is published to Maven Central under the com.meta.wearable group, so no access token or extra repository definition is required. The mavenCentral() repository in your settings.gradle.kts is all you need.

In libs.versions.toml:

[versions]
mwdat = "1.1.0"

[libraries]
mwdat-core = { group = "com.meta.wearable", name = "mwdat-core", version.ref = "mwdat" }
mwdat-camera = { group = "com.meta.wearable", name = "mwdat-camera", version.ref = "mwdat" }
mwdat-display = { group = "com.meta.wearable", name = "mwdat-display", version.ref = "mwdat" }
mwdat-mockdevice = { group = "com.meta.wearable", name = "mwdat-mockdevice", version.ref = "mwdat" }

In app/build.gradle.kts:

android {
    defaultConfig {
        manifestPlaceholders["mwdat_application_id"] = "0"
        manifestPlaceholders["mwdat_client_token"] = "0"
    }
}

dependencies {
    implementation(libs.mwdat.core)
    implementation(libs.mwdat.camera)
    implementation(libs.mwdat.display)
    implementation(libs.mwdat.mockdevice)
}

Step 2: Configure AndroidManifest.xml

<manifest ...>
    <uses-permission android:name="android.permission.BLUETOOTH" />
    <uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
    <uses-permission android:name="android.permission.INTERNET" />

    <application ...>
        <meta-data
            android:name="com.meta.wearable.mwdat.APPLICATION_ID"
            android:value="${mwdat_application_id}" />
        <meta-data
            android:name="com.meta.wearable.mwdat.CLIENT_TOKEN"
            android:value="${mwdat_client_token}" />

        <activity android:name=".MainActivity" ...>
            <intent-filter>
                <action android:name="android.intent.action.VIEW" />
                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />
                <data android:scheme="myexampleapp" />
            </intent-filter>
        </activity>
    </application>
</manifest>

APPLICATION_ID and CLIENT_TOKEN are used for app attestation and can be found in the Wearables Developer Center. In Developer Mode, attestation is not used, so the manifest placeholders can both be 0. For production, replace both placeholders with the credentials for your Wearables Developer Center app. Replace myexampleapp with your app's URL scheme.

Step 3: Initialize the SDK

import android.util.Log
import com.meta.wearable.dat.core.Wearables

class MyApplication : Application() {
    override fun onCreate() {
        super.onCreate()
        Wearables.initialize(this)
            .onFailure { error, _ ->
                Log.e("DATWearables", "Failed to initialize DAT: ${error.description}")
            }
    }
}

Step 4: Register and create a session

Registration must complete before a session can start. Observe Wearables.registrationState and wait for RegistrationState.REGISTERED.

import android.app.Activity
import androidx.lifecycle.lifecycleScope
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector
import com.meta.wearable.dat.core.session.DeviceSession
import com.meta.wearable.dat.core.session.DeviceSessionState
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.launch

fun connect(activity: Activity) {
    Wearables.startRegistration(activity)
}

fun startSession() {
    Wearables.createSession(AutoDeviceSelector()).fold(
        onSuccess = { session ->
            lifecycleScope.launch {
                session.errors.collect { error -> showError(error.description) }
            }
            lifecycleScope.launch {
                session.state.first { state -> state == DeviceSessionState.STARTED }
                addCameraStreaming(session)
            }
            // Fire-and-forget: returns Unit and reports failures through session.errors.
            session.start()
        },
        onFailure = { error, _ -> showError(error.description) },
    )
}

Observe registration and available devices:

lifecycleScope.launch {
    Wearables.registrationState.collect { state ->
        // Update registration UI
    }
}

lifecycleScope.launch {
    Wearables.devices.collect { devices ->
        // Update the device list
    }
}

Step 5: Add camera streaming

Call this only once the session reports DeviceSessionState.STARTED; adding a capability earlier fails with DeviceSessionError.SESSION_IDLE.

import com.meta.wearable.dat.camera.addCamera
import com.meta.wearable.dat.camera.types.StreamConfiguration
import com.meta.wearable.dat.camera.types.VideoQuality

fun addCameraStreaming(session: DeviceSession) {
    session.addCamera(
        StreamConfiguration(videoQuality = VideoQuality.MEDIUM, frameRate = 24),
    ).fold(
        onSuccess = { camera ->
            camera.stream.start().onFailure { error, _ -> showError(error.description) }
        },
        onFailure = { error, _ -> showError(error.description) },
    )
}

Camera access also needs the DAT camera permission; see the permissions-registration skill.

Next steps

  • camera-streaming — Stream capability, video frames, photo capture
  • mockdevice-testing — Test without hardware
  • session-lifecycle — Handle session and stream state changes
  • permissions-registration — Registration and permission flows
  • display-access — Render content on Meta Ray-Ban Display glasses
  • Full Android API reference

Inputs (Android)

Use the experimental Inputs capability to receive semantic interactions from glasses. The app must have Inputs enabled for it in Wearables Developer Center. Inputs does not have a Permission value to request at runtime.

Attach after the session starts

Call addInputs only after DeviceSessionState.STARTED. Adding the capability starts it automatically; there is no separate public start call.

session.addInputs(
    InputsConfiguration(consumeBack = true),
).fold(
    onSuccess = { inputs ->
        inputEventsJob = lifecycleScope.launch {
            inputs.events.collect { event ->
                when (event) {
                    is InputEvent.Nav -> moveFocus(event.direction)
                    is InputEvent.Select -> activateFocusedItem()
                    is InputEvent.Back -> dismissCurrentView()
                    is InputEvent.Button -> handleButton(event.button)
                    is InputEvent.Capture -> handleCaptureButton(event.pressType)
                    is InputEvent.Drag -> updateDrag(
                        event.action,
                        event.x,
                        event.y,
                        event.dx,
                        event.dy,
                    )
                }
            }
        }
    },
    onFailure = { error, _ -> showError(error.description) },
)

Retain the event job while Inputs is attached. Collect state when the UI needs activation state and errors for activation, connection, and communication failures.

The default configuration enables every known source except UNKNOWN and consumes Back. Set consumeBack to false when the system should handle Back.

Each event includes its source and device timestamp in milliseconds. Keep the when exhaustive so new behavior is intentional.

Errors and cleanup

InputsError.PERMISSION_DENIED means the capability was not approved for the app; it is not fixed with requestPermission. Other errors cover activation, connection, and communication failures.

Cancel the collection jobs, then call session.removeInputs() and handle its DatResult before adding another Inputs capability.

Test with MockDeviceKit

After the app has attached Inputs, inject events through the paired mock glasses with calls such as glasses.services.input.navDown(). The service also exposes Select, Back, Button, Capture, and Drag injection. Calls made before Inputs is active, or from a source excluded by the configuration, are dropped.

Availability

Inputs is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

Live DAT Debugging with MCP (Android)

Use this skill when a developer is debugging an Android DAT app and has a local DAT Inspector MCP server or DAT debug server available to the agent. This is the live-runtime counterpart to dat-docs-mcp: use docs search for API lookup, and use this skill for observed app/device behavior.

Ground rules

  • Stay read-only unless the user explicitly asks for an app code change.
  • Do not mutate Meta AI app, device, account, permission, registration, or app state.
  • Treat companion-app/device results as boundary diagnosis from app-visible DAT events, not as direct access to companion app internals.
  • Use live DAT evidence before guessing: registration, permissions, device selection, link state, session state, stream state, typed errors, and SDK logs.
  • If the MCP tools are not configured, ask the developer to enable their local DAT debug setup and fall back to logs plus search_dat_docs.

Normal agent loop

  1. Discover and connect to the running app debug server:
    • discover_debug_servers
    • connect_to_debug_server
  2. Establish the baseline with all three calls before diagnosing deeper:
    • get_connection_status
    • get_sdk_state
    • get_dat_readiness Do not skip get_connection_status; it separates MCP/debug-server connectivity from SDK, registration, permission, session, and stream state.
  3. If the issue looks like Meta AI app or device boundary behavior, inspect:
    • get_companion_boundary_diagnosis
    • get_device_path
    • get_permissions
    • get_device_properties
  4. Ask the developer to reproduce the failing Android flow while connected.
  5. Wait for narrow evidence instead of polling everything:
    • wait_for_events with category or source
    • get_errors
    • get_event_digest
  6. For handoff, collect a redacted bundle:
    • export_diagnostic_bundle

Evidence map

  • Initialization: app logs around Wearables.initialize(context)
  • Registration: Wearables.registrationState and wearables.registrationState
  • Device availability: Wearables.devices plus debug events from wearables.devices, device.compatibility, and device.properties
  • Device link: device.linkState
  • Permissions: Wearables.checkPermissionStatus(...), device.permission, check_permissions, and get_permissions
  • Session: createSession(...), session.start(), deviceSession.state, session.state, and session errors
  • Stream: session.addCamera(...), camera.stream.start(), camera.stream.state, frame counters, and stream errors
  • DAM/DWA-visible transport: DAM/DWA error values

Diagnosis patterns

  • Registration blocked: verify Android manifest metadata, app credentials or Developer Mode setup, and callback/deeplink handling.
  • No eligible device: inspect Wearables.devices, device compatibility, and device.linkState before changing selector code.
  • Permission failure: use get_permissions and get_companion_boundary_diagnosis; do not assume Android runtime permission is the same as DAT permission through the Meta AI app/device boundary.
  • Session fails before stream creation: inspect session errors and selected device compatibility before changing camera code.
  • Stream fails after session start: inspect stream state, camera permission, and recent errors before changing frame processing.

Output expectations

When summarizing, name the evidence plane, the blocking state, and the next developer action. Example:

DAT readiness is blocked at permissions: camera is denied in app-visible DAT
events. Grant the DAT camera permission in Meta AI, return to the app, then
retry session.start() and camera.stream.start() while the MCP connection waits for
permission and stream events.

MockDevice MCP (Android)

Use this skill when you want to verify DAT app behavior without physical glasses by controlling a simulated device over the MockDevice MCP server. The MCP is a client of the app's in-app HTTP test server; it drives device state, and you observe the result with the existing Android screenshot / UI / logcat tools.

No third-party dependencies — the server runs on the Python 3.8+ standard library, launched with python3. The MCP server is mcp/mockdevice/mockdevice_mcp_server.py.

Prerequisite: the app must start its MockDevice test server

The test server starts when the app calls startTestServer — typically a debug build launched with the mwdat_ui_testing intent extra, which also puts MockDeviceKit into mock mode, as the DAT sample apps (e.g. CameraAccess) do. You are driving a simulated device, not real glasses.

Launch (or relaunch) the app that way with adb, e.g.:

adb shell am start -n <package>/<launcher-activity> --ez mwdat_ui_testing true

If mock_connect reports no server, relaunch with that extra and try again.

Normal agent loop

  1. Make sure the app is running its MockDevice test server (sample apps: launch with the mwdat_ui_testing extra, see above).
  2. mock_connect(bundle_id="<your app package>", platform="android") — auto-detects the attached device, forwards a local port to the app's fixed test-server loopback port over adb, and verifies health. Pass udid (the device serial) if more than one device/emulator is attached.
  3. Drive device state, e.g.:
    • mock_pair_device(), then mock_power_on() and mock_don() — pairs Ray-Ban Meta, powers it on, and wears it. A newly paired device is off, folded, and not worn.
    • mock_set_camera_feed(resource_name="plant", ext="mp4") — the app must bundle the feed as an asset, for example app/src/debug/assets/plant.mp4.
    • mock_power_off / mock_power_on, mock_doff / mock_don, mock_fold / mock_unfold, mock_captouch_tap.
  4. Observe the effect with the Android screenshot / UI / logcat tools and mock_device_state.

device_id is optional on device commands — it is auto-resolved when exactly one device is paired.

Tools

The server advertises every tool with its arguments and a description of how to verify its effect, so read them from your tool list. Each tool name starts with mock_.

Notes

  • Camera feeds name an asset bundled in the app, by name and extension — not an arbitrary file path, and not a res/raw resource. The app's test server opens <resource_name>.<ext> with AssetManager, so a file outside assets/ is reported as resource not found. The same applies to captured images and motion feeds.
  • mock_connect refuses an app whose DAT SDK serves a different MockDevice route contract than this MCP calls, and says whether to update the app or the plugin. Relay that to the developer rather than retrying.
  • mock_connect also reports the routes the attached app advertises. A tool added in a later SDK release than the app uses reports that it is unavailable instead of failing opaquely.
  • Apart from mock_display_content, which reads the display back, these tools drive device state only. To assert other outcomes (a stream started, a photo was captured), read the app with the Android screenshot / UI / logcat tools.
  • Display tools need a display-capable device: pair with mock_pair_device(model="metaRayBanDisplay"), not the default rayBanMeta. mock_display_content returns the display content the app sent, as a node tree — the semantic content, not a rendered image. It is a debug payload rather than a stable API, so read the document you get back instead of assuming field names. For pixels, host the SDK's own preview view and screenshot it; that renders the same content on the phone and is what a developer looks at. Click identifiers are content-derived and change on every redraw — take a fresh one from the tree each time. A mock_display_click ok means the tap was delivered, not that it matched an element, so confirm by re-reading and checking the revision moved.

MockDevice Testing (Android)

Use MockDeviceKit to test DAT SDK integrations without physical Meta glasses.

MockDeviceKit simulates Meta glasses behavior for development and testing. It provides:

  • MockDeviceKit — Entry point for creating simulated devices
  • MockGlasses — Simulated Ray-Ban Meta glasses
  • MockCameraKit — Simulated camera with configurable video feed and photo capture

Setup

Add mwdat-mockdevice to your Gradle dependencies:

dependencies {
    implementation(libs.mwdat.mockdevice)
}

Creating a mock device

import com.meta.wearable.dat.mockdevice.MockDeviceKit
import com.meta.wearable.dat.mockdevice.api.GlassesModel
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitConfig

val mockDeviceKit = MockDeviceKit.getInstance(context)

// Attach fake registration and connectivity (auto-initializes Wearables if needed).
// By default, Wearables.registrationState transitions to Registered.
mockDeviceKit.enable()

// Or start in unregistered state to test registration flows:
// mockDeviceKit.enable(MockDeviceKitConfig(initiallyRegistered = false))

val device = mockDeviceKit.pairGlasses(GlassesModel.RAYBAN_META).getOrThrow()

You can check mockDeviceKit.isEnabled to query whether the mock environment is active.

Simulating device states

// Simulate glasses lifecycle
device.powerOn()
device.unfold()
device.don()    // Simulate wearing the glasses

// Later...
device.doff()   // Simulate removing
device.fold()
device.powerOff()

Configuring permissions

MockDeviceKit provides permissions to control permission behavior without the Meta AI app.

By default, RequestPermissionContract returns Granted. Use set() to control checkPermissionStatus() and setRequestResult() to control request outcomes.

val mockDeviceKit = MockDeviceKit.getInstance(context)

// Simulate denied camera permission status
mockDeviceKit.permissions.set(Permission.CAMERA, PermissionStatus.Denied)

// Simulate denied request result (user tapping "deny")
mockDeviceKit.permissions.setRequestResult(Permission.CAMERA, PermissionStatus.Denied)

Setting up mock camera feeds

Video streaming

val camera = device.services.camera
camera.setCameraFeed(videoUri)

Photo capture

val camera = device.services.camera
camera.setCapturedImage(imageUri)

Note: Android doesn't transcode video automatically. Mock video files must be in h.265 format: the SDK requests HEVC, so MockDeviceKit rejects an h.264 file when the stream starts. Use FFmpeg to convert:

ffmpeg -i input.mp4 -c:v libx265 -tag:v hvc1 -an -vf "scale=540:960" output.mp4

On macOS, -c:v hevc_videotoolbox uses the hardware encoder instead.

Phone camera as the feed

setCameraFeed(CameraFacing) sources frames from the handset camera, so the app needs android.permission.CAMERA in the manifest and at runtime. That is not the DAT Permission.CAMERA covered by MockDeviceKitConfig(initialPermissionsGranted = true) — no real-glasses path needs the handset grant, so an existing integration will never have requested it.

Grant it before calling setCameraFeed; in instrumentation tests, pm grant <your.package> android.permission.CAMERA. Without it the mock rejects the stream start and the failure names the missing permission. setCameraFeed(Uri) has no such requirement.

Writing instrumentation tests

Create a reusable test base class:

import android.content.Context
import androidx.test.ext.junit.rules.ActivityScenarioRule
import androidx.test.platform.app.InstrumentationRegistry
import com.meta.wearable.dat.mockdevice.MockDeviceKit
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitConfig
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitInterface
import org.junit.After
import org.junit.Before
import org.junit.Rule

open class MockDeviceKitTestCase<T : Any>(
    private val activityClass: Class<T>
) {
    @get:Rule
    val scenarioRule = ActivityScenarioRule(activityClass)

    protected lateinit var mockDeviceKit: MockDeviceKitInterface
    protected lateinit var targetContext: Context
    protected open val mockDeviceKitConfig = MockDeviceKitConfig()

    @Before
    open fun setUp() {
        val instrumentation = InstrumentationRegistry.getInstrumentation()
        targetContext = instrumentation.targetContext
        mockDeviceKit = MockDeviceKit.getInstance(targetContext)
        grantRuntimePermissions()
        mockDeviceKit.enable(mockDeviceKitConfig)
    }

    @After
    open fun tearDown() {
        mockDeviceKit.disable()
    }

    private fun grantRuntimePermissions() {
        val packageName = targetContext.packageName
        val shell = InstrumentationRegistry.getInstrumentation().uiAutomation
        shell.executeShellCommand("pm grant $packageName android.permission.BLUETOOTH_CONNECT")
        shell.executeShellCommand("pm grant $packageName android.permission.CAMERA")
    }
}

To start a test in another state, override mockDeviceKitConfig in the subclass, for example with MockDeviceKitConfig(initiallyRegistered = false). Don't call enable again inside the test: once the kit is enabled, further enable calls are ignored until disable().

Using MockDeviceKit in the CameraAccess sample

The CameraAccess sample app includes a Debug menu for MockDeviceKit:

  1. Tap the Debug icon to open the MockDeviceKit menu
  2. Tap Pair RayBan Meta to create a simulated device
  3. Use PowerOn, Unfold, Don to simulate glasses states
  4. Select video/image files for mock camera feeds
  5. Start streaming to see simulated frames

Supported media formats

TypeFormats
Videoh.265 (HEVC)
ImageJPEG, PNG

Links

Motion (Android)

Motion is an experimental capability for streaming inertial sensor samples. Enable it for the app in Wearables Developer Center. It has no runtime Permission value.

Attach, collect, and start

Call addMotion only after DeviceSessionState.STARTED. The returned capability starts in STOPPED; launch collectors before calling start().

session.addMotion(
    MotionConfiguration(samplingRate = MotionSamplingRate.HZ_30),
).fold(
    onSuccess = { motion ->
        motionSampleJob = lifecycleScope.launch {
            motion.samples.collect { sample -> render(sample) }
        }
        motion.start()
    },
    onFailure = { error, _ -> showError(error.description) },
)

Retain the sample job while streaming. Collect state for lifecycle updates and errors for sensor, connection, and closure failures.

Sampling rates are 5, 10, 15, 24, 30, and 60 Hz; 10 Hz is the default.

Read samples defensively

timestampNs uses the device monotonic clock. Accelerometer values are m/s², gyroscope values are rad/s, and magnetometer values are µT. Each sensor value and orientation can be null; source identifies glasses, Neural Band, or an unknown source.

Motion normally transitions STOPPED -> STARTING -> STARTED. When its parent session pauses, it reports MotionState.PAUSED and automatically restarts when the session resumes.

Stop and remove

motion.stop() is reusable. Cancel the sample job before calling session.removeMotion() and handle its DatResult; removal is terminal.

Test with MockDeviceKit

Replay an in-memory recording with glasses.services.motion.setMotionFeed(samples, loop = true) or provide a CSV file URI. App logic should observe the SDK Motion state rather than isStreaming, which is a test assertion helper.

Availability

Motion is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

Permissions & Registration (Android)

Register your app with Meta AI, then request the device permissions it needs.

The DAT SDK separates two steps:

  1. Registration: The user connects your app to Meta AI.
  2. Device permissions: After registration, your app requests capabilities such as camera access.

Both flows depend on the Meta AI app being installed on the phone.

Start registration

Wearables.startRegistration(activity)

Observe registration state:

lifecycleScope.launch {
    Wearables.registrationState.collect { state ->
        // Update your registration UI
    }
}

To unregister:

Wearables.startUnregistration(activity)

Check permission status

checkPermissionStatus(...) is a suspend API that returns a DatResult.

lifecycleScope.launch {
    Wearables.checkPermissionStatus(Permission.CAMERA)
        .onSuccess { status ->
            if (status == PermissionStatus.Granted) {
                startStreaming()
            }
        }
        .onFailure { error, _ ->
            showPermissionError(error.description)
        }
}

Request a permission

Use Wearables.RequestPermissionContract() with the Activity Result API:

private val permissionLauncher =
    registerForActivityResult(Wearables.RequestPermissionContract()) { result ->
        result.onSuccess { status ->
            if (status == PermissionStatus.Granted) {
                startStreaming()
            }
        }.onFailure { error, _ ->
            showPermissionError(error.description)
        }
    }

fun requestCameraPermission() {
    permissionLauncher.launch(Permission.CAMERA)
}

Users can allow once or allow always through the Meta AI flow.

Developer Mode vs production

ModeRegistration behavior
Developer ModeUse mwdat_application_id = 0 and mwdat_client_token = 0 manifest placeholders for local development
ProductionUse the application ID and client token assigned in the Wearables Developer Center

For development builds, enable Developer Mode in the Meta AI app before testing registration and permissions.

Prerequisites

  • Internet connection for registration
  • Meta AI app installed on the phone
  • Callback URI scheme configured in AndroidManifest.xml
  • Bluetooth permission granted on Android

Links

Sample App Guide (Android)

Build an Android DAT app with registration, sessions, camera streaming, and photo capture.

Pair this with the CameraAccess sample.

Project setup

  1. Create an Android Studio app project.
  2. Declare the SDK dependencies in libs.versions.toml and your app's build.gradle.kts.
  3. Configure AndroidManifest.xml for registration callbacks plus APPLICATION_ID and CLIENT_TOKEN.
  4. Initialize Wearables in your Application.

Suggested app structure

app/src/main/java/com/example/myapp/
├── MyApplication.kt
├── MainActivity.kt
├── session/
│   └── SessionViewModel.kt
└── ui/
    ├── RegistrationScreen.kt
    └── CameraScreen.kt

Registration and session creation

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        lifecycleScope.launch {
            Wearables.registrationState.collect { state ->
                // Update registration UI
            }
        }
    }

    fun register() {
        Wearables.startRegistration(this)
    }
}

DeviceSession.start() is fire-and-forget. Attach the camera from the session-state collector once the session reports DeviceSessionState.STARTED; adding a capability to an IDLE session fails with DeviceSessionError.SESSION_IDLE.

class SessionViewModel : ViewModel() {
    private var session: DeviceSession? = null
    private var camera: Camera? = null

    fun startCameraSession() {
        Wearables.createSession(AutoDeviceSelector()).fold(
            onSuccess = { createdSession ->
                session = createdSession

                viewModelScope.launch {
                    createdSession.errors.collect { error -> showError(error.description) }
                }
                viewModelScope.launch {
                    createdSession.state.collect { state ->
                        when (state) {
                            DeviceSessionState.STARTED -> attachCamera(createdSession)
                            DeviceSessionState.STOPPED -> clearSession(createdSession)
                            else -> Unit
                        }
                    }
                }

                createdSession.start()
            },
            onFailure = { error, _ -> showError(error.description) },
        )
    }

    private fun attachCamera(activeSession: DeviceSession) {
        if (camera != null) return
        activeSession.addCamera(
            StreamConfiguration(videoQuality = VideoQuality.MEDIUM, frameRate = 24),
        ).fold(
            onSuccess = { addedCamera ->
                camera = addedCamera
                viewModelScope.launch {
                    addedCamera.stream.videoStream.collect { frame ->
                        // Render preview
                    }
                }
                addedCamera.stream.start().onFailure { error, _ ->
                    showError(error.description)
                }
            },
            onFailure = { error, _ -> showError(error.description) },
        )
    }

    private fun clearSession(stoppedSession: DeviceSession) {
        if (session !== stoppedSession) return
        camera = null
        session = null
    }
}

Capture photos

fun capturePhoto() {
    viewModelScope.launch {
        camera?.stream?.capturePhoto()
            ?.onSuccess { photoData ->
                when (photoData) {
                    is PhotoData.Bitmap -> savePhoto(photoData.bitmap)
                    is PhotoData.HEIC -> saveHeic(photoData.data)
                }
            }
            ?.onFailure { error, _ ->
                showCaptureError(error.description)
            }
    }
}

Shutdown

fun stopCameraSession() {
    camera?.stop()
    session?.stop()
    camera = null
    session = null
}

Testing with MockDeviceKit

Use MockDeviceKit to simulate linking glasses, permission state, and camera media without physical hardware. See the mockdevice-testing skill for setup details.

Links

Session Lifecycle (Android)

Manage session and stream state in DAT SDK integrations.

Create a DeviceSession with Wearables.createSession(...), start it, then attach capabilities such as camera streaming once the session is STARTED. Session lifecycle and capability lifecycle are related but distinct.

Session states

StateMeaningApp action
IDLESession created, not started yetCall session.start()
STARTINGConnecting to the deviceShow loading UI
STARTEDSession active and ready for capabilitiesAdd or use capabilities
PAUSEDSession temporarily suspended by pause() or the deviceKeep state, call session.resume() or wait for the device to resume
STOPPINGSession is shutting downStop user work and wait
STOPPEDTerminal; session endedRelease resources and create a new session if needed

STOPPED is terminal: a stopped session cannot be restarted, so call Wearables.createSession(...) again for the next connection.

Observe session state

start(), stop(), pause(), and resume() are all sync fire-and-forget and return Unit. Subscribe to state and errors before calling start() so no transition is missed, and treat session.errors as the failure channel for start().

Wearables.createSession(AutoDeviceSelector()).fold(
    onSuccess = { session ->
        lifecycleScope.launch {
            session.state.collect { state ->
                when (state) {
                    DeviceSessionState.STARTED -> onStarted(session)
                    DeviceSessionState.PAUSED -> onPaused()
                    DeviceSessionState.STOPPED -> onStopped()
                    else -> Unit
                }
            }
        }
        lifecycleScope.launch {
            session.errors.collect { error -> showError(error.description) }
        }
        session.start()
    },
    onFailure = { error, _ -> showError(error.description) },
)

Add each capability once from onStarted(session): a resumed session goes from PAUSED back to STARTED and keeps its capabilities, so a second session.addCamera(...) fails with DeviceSessionError.CAPABILITY_ALREADY_ADDED. Calling session.addCamera(...) while the session is still IDLE fails with DeviceSessionError.SESSION_IDLE, and calling it after STOPPED fails with DeviceSessionError.SESSION_ALREADY_STOPPED.

Stream state

Camera streaming has its own state flow after you add a camera:

STOPPED -> STARTING -> STARTED -> STREAMING -> STOPPING -> STOPPED

CLOSED is the terminal stream state, and PAUSED is reported when the device pauses streaming (for example a single cap-touch tap) and can resume on its own.

lifecycleScope.launch {
    camera.stream.state.collect { state ->
        // React to stream state changes
    }
}

Common transitions

The SDK may pause or stop a session when:

  • Another experience takes over the device
  • The user removes or folds the glasses
  • Bluetooth connectivity drops
  • The user unregisters the app or revokes needed access

Pause and resume

Call pause() on a STARTED session and resume() on a PAUSED one. Like start() and stop(), both are sync fire-and-forget: observe state for the STARTED -> PAUSED -> STARTED transitions. pause() outside STARTED and resume() outside PAUSED are no-ops. The state changes only when the device confirms, so wait for PAUSED before calling resume() (and for STARTED before calling pause() again); earlier calls are ignored. If the observed state doesn't match what your app wants, call pause() or resume() again once it settles.

pauseButton.setOnClickListener { session.pause() }   // STARTED -> PAUSED
resumeButton.setOnClickListener { session.resume() } // PAUSED -> STARTED

The device can also pause and resume the session on its own (for example a single cap-touch tap). When a session is paused:

  • The device connection may remain active
  • Attached capabilities stop doing useful work
  • Your app should not restart the session; call resume() or wait for the next observed session state

Device availability

lifecycleScope.launch {
    Wearables.devices.collect { devices ->
        // Update the list of available devices
    }
}

Use Wearables.devices and device metadata to decide when it is sensible to create a new session after a stop.

Checklist

  • Handle all DeviceSessionState values you care about
  • Observe stream state separately from session state
  • Release resources only after stop or close
  • Recreate sessions after terminal stops instead of reusing dead ones
  • Surface typed DeviceSessionError and StreamError failures

Links

Speech (Android)

Speech is an experimental on-device recognition capability. Enable it for the app in Wearables Developer Center and obtain Permission.MICROPHONE before starting it.

Permission flow

Check permission before attaching Speech. Launch Wearables.RequestPermissionContract only from a user-initiated action; it opens the Meta AI app. Android RECORD_AUDIO is not required for real-glasses recognition. It is needed only when a MockDevice test deliberately uses live phone recognition.

Attach, collect, and start

Call addSpeech after DeviceSessionState.STARTED. Launch collectors before start().

session.addSpeech().fold(
    onSuccess = { speech ->
        transcriptionJob = lifecycleScope.launch {
            speech.transcriptions.collect { result ->
                result ?: return@collect
                if (result.isFinal) commitTranscript(result.text)
                else showPartialTranscript(result.text)
            }
        }
        speech.start().onFailure { error, _ -> showError(error.description) }
    },
    onFailure = { error, _ -> showError(error.description) },
)

Retain the transcription job while Speech is attached. Collect state, locale, and errors when the app needs lifecycle, locale, or failure updates.

confidence is between 0 and 1, or -1 when unavailable. errorDetails is diagnostic text and should not be displayed directly to end users.

Stop and remove

speech.stop() is reusable; removal is terminal. Cancel all collection jobs before session.removeSpeech().

Test with MockDeviceKit

After Speech is listening, inject results with calls such as glasses.services.speech.simulateTranscription("turn left", isFinal = true, confidence = 0.9f). The service can also inject locale changes, partial text, completion, and terminal errors. The live-device source and host RECORD_AUDIO permission are only for manual MockDevice testing.

Availability

Speech is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

Voice Invocations (Android)

Voice Invocations lets Meta AI launch or foreground an app when the wearer says “Hey Meta, start {app name}.” It is a Wearables-level stream, not a DeviceSession capability, and does not require a running device session.

Configure the app

Register the package name, request Voice Invocation approval, and configure the spoken app name in Wearables Developer Center. Do not declare invocation phrases in the manifest. Voice Invocations does not require camera or microphone permission.

Detect cold and warm launches

Use the SDK validator in both entry points. Do not inspect intent extras yourself.

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    if (isVoiceInvocationsIntent(intent)) startYourExperience()
}

override fun onNewIntent(intent: Intent) {
    super.onNewIntent(intent)
    setIntent(intent)
    if (isVoiceInvocationsIntent(intent)) startYourExperience()
}

Start and retain the stream

Store one app-scoped stream. The factory starts it automatically; there is no separate public start call.

stream = Wearables.startVoiceInvocationsStream(AutoDeviceSelector())

invocationJob = appScope.launch {
    stream.invocations.collect { invocation ->
        when (invocation) {
            is LaunchApp -> {
                navigateToMainScreen()
                invocation.responseHandle.sendSuccess(actionOutput = null)
            }
        }
    }
}

Collect state and errors when the app needs lifecycle or failure updates.

Every delivered invocation must be answered exactly once with sendSuccess or sendFailure. Both are suspend functions returning whether delivery succeeded.

The stream reports STARTING, STARTED, and STOPPED. Creation and automatic startup do not throw; failures surface through errors and state.

Close and test

close() is terminal. Cancel collectors before closing the app-scoped stream when the app no longer listens.

To test without hardware, check glasses.services.voiceInvocation.hasConnectedApps before calling simulateLaunchAppAction(). Use simulateIncompleteAction() to verify malformed actions surface an error rather than an invocation.

Availability

Voice Invocations is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.

See the Voice Invocations guide.

MCP servers 2

Sponsors

Details

Marketplace
meta/meta-wearables-dat-android
Source
plugins/mwdat-android
Author
Meta Wearables
Category
Community