
mwdat-android
Git-native DAT SDK guidance for android development, covering setup, camera streaming, display access, MockDeviceKit, session lifecycle, permissions, debugging, and sample app patterns.
- version
- 1.0.1
- GitHub stars
- 406
Install
/plugin install mwdat-android@mwdat-android-marketplaceRun it inside Claude Code after adding the marketplace with /plugin marketplace add meta/meta-wearables-dat-android.
Contents
Discovered from sourceSkills 18
Audio Streaming (Android)
Audio Streaming is an experimental extension of Camera Stream, not a separate capability. Follow the camera-streaming skill for session and Camera setup, then add audio configuration and collection.
Permissions
The app needs Camera and Audio Streaming approval in Wearables Developer Center plus both Permission.CAMERA and Permission.MICROPHONE. Check first and launch permission requests only after a user action.
Configure PCM audio
Set a non-null codec in StreamConfiguration:
val configuration = StreamConfiguration(
audioCodec = AudioCodec.PCM(
sampleRate = AudioSampleRate.RATE_16000,
numberOfChannels = 1,
),
videoQuality = VideoQuality.MEDIUM,
frameRate = 24,
)
Pass this configuration to addCamera using the Camera Streaming flow. Then collect audio before starting its Stream:
audioJob = lifecycleScope.launch(Dispatchers.IO) {
camera.stream.audioStream.collect { frame ->
consumePCM(frame.buffer, frame.presentationTimeUs)
}
}
camera.stream.start().onFailure { error, _ -> showError(error.description) }
Supported sample rates are 16,000, 44,100, and 48,000 Hz. Configure downstream processing with the same rate and channel count. Use presentationTimeUs to preserve order or align audio with video. Process PCM off the main thread because slow collectors can drop frames.
Audio shares Stream state and errors. A microphone denial is surfaced as a stream permission failure; do not model a separate audio lifecycle.
Cleanup and testing limits
Cancel audio and any other Stream collectors before calling camera.stop(). Stopping Camera cascades to its Stream, and the instances cannot be restarted.
MockDeviceKit can negotiate an audio-enabled Camera stream, but it does not provide a public deterministic audio-frame injection API. Do not assume mock PCM frames will arrive.
Availability
Audio Streaming is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
Camera Capture (Android)
Camera Capture is the experimental standalone high-quality photo path exposed by camera.photo. It is different from stream.capturePhoto(), which takes a photo while video streaming remains active.
Prepare Camera and permission
The app needs Camera approval in Wearables Developer Center and Permission.CAMERA. Add Camera only after DeviceSessionState.STARTED. Stream and standalone Photo compete for camera hardware, so stop an active Stream before accessing Photo.
Observe Photo
Accessing camera.photo lazily creates and starts the standalone Photo child. Collect its flows and wait for PhotoState.STARTED before enabling capture.
val photo = camera.photo
photoStateJob = lifecycleScope.launch {
photo.state.collect { state ->
photoButtonEnabled = state == PhotoState.STARTED
}
}
photoDataJob = lifecycleScope.launch {
photo.photoStream.collect { capture ->
save(capture.imageData, capture.metadata)
}
}
progressJob = lifecycleScope.launch {
photo.transferProgressStream.collect { progress -> updateProgress(progress) }
}
errorJob = lifecycleScope.launch {
photo.errors.collect { error -> showError(error.description) }
}
After Photo reaches PhotoState.STARTED, call photo.capturePhoto(resolution = PhotoResolution.FULL, quality = PhotoQuality.HIGH) with the desired resolution and quality. Capture is asynchronous; data arrives through photoStream. Collect transferProgressStream for progress and errors for failures.
Errors and cleanup
Handle not-ready, capture, setup, disconnect, busy, service, permission, and device-health errors from errors. After delivery, stop Photo, cancel its collectors, and stop Camera.
Photo is terminal after stop(), and accessing camera.photo again returns the same stopped instance. To capture again, remove the current Camera from the session, add a new Camera, and access Photo from that new instance. Stopping Camera or the parent session also cascades cleanup.
Test with MockDeviceKit
Configure the next image with glasses.services.cameraCapture.setCapturedPhoto(imageUri) or use simulateCaptureFailure() for a one-shot failure.
Availability
Standalone Camera Capture is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
Camera Streaming (Android)
Use a DeviceSession and an attached Camera to receive frames and capture photos through camera.stream.
For PCM audio delivered with this stream, use the audio-streaming skill.
Key concepts
- DeviceSession: Device connection lifecycle created through
Wearables.createSession(...) - Camera: Camera capability attached to a session with
session.addCamera(...) - Stream: Video stream accessed through
camera.stream - StreamConfiguration: Video quality, frame rate, and compression configuration for the stream
- PhotoData: Still image captured from glasses while streaming
Create a session and attach a stream
DeviceSession.start() is fire-and-forget: it returns Unit and the connection completes in the background. A capability can only be added once the session reports DeviceSessionState.STARTED — calling addCamera(...) right after start() fails with DeviceSessionError.SESSION_IDLE. Add the camera from the session-state collector.
import com.meta.wearable.dat.camera.Camera
import com.meta.wearable.dat.camera.addCamera
import com.meta.wearable.dat.camera.types.StreamConfiguration
import com.meta.wearable.dat.camera.types.VideoQuality
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector
import com.meta.wearable.dat.core.session.DeviceSessionState
var camera: Camera? = null
Wearables.createSession(AutoDeviceSelector()).fold(
onSuccess = { session ->
lifecycleScope.launch {
session.errors.collect { error -> showError(error.description) }
}
lifecycleScope.launch {
session.state.collect { state ->
if (state == DeviceSessionState.STARTED && camera == null) {
session.addCamera(
StreamConfiguration(
videoQuality = VideoQuality.MEDIUM,
frameRate = 24,
),
).fold(
onSuccess = { addedCamera ->
camera = addedCamera
addedCamera.stream.start().onFailure { error, _ ->
showError(error.description)
}
},
onFailure = { error, _ -> showError(error.description) },
)
}
}
}
// Subscribe before start() so no initial transition is missed.
session.start()
},
onFailure = { error, _ -> showError(error.description) },
)
Check Wearables.checkPermissionStatus(Permission.CAMERA) before starting the stream; see the permissions-registration skill.
Resolution options
| Quality | Size |
|---|---|
VideoQuality.HIGH | 720 x 1280 |
VideoQuality.MEDIUM | 504 x 896 |
VideoQuality.LOW | 360 x 640 |
Frame rate options
Valid values: 2, 7, 15, 24, 30 FPS.
Lower resolution and frame rate usually produce better visual quality per frame over Bluetooth.
Observe stream state
StreamState transitions: STOPPED -> STARTING -> STARTED -> STREAMING -> STOPPING -> STOPPED, and CLOSED once the stream is terminal. PAUSED is reported when the device pauses the stream, for example on a single cap-touch tap; the stream can resume on its own from PAUSED.
lifecycleScope.launch {
camera.stream.state.collect { state ->
when (state) {
StreamState.STREAMING -> {
// Frames are flowing
}
StreamState.PAUSED -> {
// Paused by the device; wait for it to resume
}
StreamState.STOPPED -> {
// Streaming ended
}
StreamState.CLOSED -> {
// Stream fully closed
}
else -> Unit
}
}
}
Observe camera.stream.errorStream alongside the state. StreamError.STREAM_ERROR is informational and does not stop the stream, while StreamError.CRITICAL_STREAM_ERROR means the stream should be torn down.
lifecycleScope.launch {
camera.stream.errorStream.collect { error ->
showStreamError(error.description)
}
}
Receive frames
lifecycleScope.launch {
camera.stream.videoStream.collect { frame ->
updatePreview(frame)
}
}
By default the SDK decodes on the phone and VideoFrame.buffer holds YUV pixel data. Set StreamConfiguration(compressVideo = true) to receive compressed HEVC buffers instead; then check frame.isCompressed and frame.isCodecConfig and feed the frames to your own decoder. Keep frame handling off the main thread for anything heavier than a buffer copy.
In-stream photo capture
This path captures while video streaming remains active. For standalone high-quality capture with resolution, quality, transfer progress, and its own lifecycle, use the camera-capture skill.
capturePhoto() only succeeds while the stream is active, and returns a PhotoData sealed type — branch on the variant instead of reading a single data property.
import com.meta.wearable.dat.camera.types.PhotoData
lifecycleScope.launch {
camera.stream.capturePhoto()
.onSuccess { photoData ->
when (photoData) {
is PhotoData.Bitmap -> savePhoto(photoData.bitmap)
is PhotoData.HEIC -> saveHeic(photoData.data) // ByteBuffer of HEIC bytes
}
}
.onFailure { error, _ ->
showCaptureError(error.description)
}
}
Only one capture can be in flight at a time; a second concurrent call fails with CaptureError.CaptureInProgress.
Clean up
Stop the camera when you no longer need camera data, then stop the parent session if the device interaction is finished. Stopping the camera cascades to its stream, and stopping the session cascades to every attached capability.
camera.stop()
session.stop()
Camera.stop() and Stream.stop() invalidate the instance — they cannot be restarted. Call session.removeCamera() to detach the capability so a later session.addCamera(...) on the same session can succeed.
Links
DAT SDK Conventions (Android)
Quick Reference
| Task | Command |
|---|---|
| Build app | ./gradlew assembleDebug |
| Run tests | ./gradlew test |
| Install app | ./gradlew installDebug |
| Lint app | ./gradlew lint |
Architecture
The SDK is organized into four public modules:
- mwdat-core: Registration, permissions, devices, and session creation
- mwdat-camera: Camera capability, video frames, and photo capture
- mwdat-display: Display capability, display UI components, icons, images, buttons, and video
- mwdat-mockdevice: MockDeviceKit for testing without hardware
Initialization and session setup
DeviceSession.start() is fire-and-forget and returns Unit; capabilities can only be attached once the session reports DeviceSessionState.STARTED.
Wearables.initialize(context)
var camera: Camera? = null
Wearables.createSession(AutoDeviceSelector()).fold(
onSuccess = { session ->
lifecycleScope.launch {
session.state.collect { state ->
if (state == DeviceSessionState.STARTED && camera == null) {
session.addCamera(StreamConfiguration()).fold(
onSuccess = { addedCamera ->
camera = addedCamera
addedCamera.stream.start().onFailure { error, _ ->
showError(error.description)
}
},
onFailure = { error, _ -> showError(error.description) },
)
}
}
}
lifecycleScope.launch {
session.errors.collect { error -> showError(error.description) }
}
session.start()
},
onFailure = { error, _ -> showError(error.description) },
)
Kotlin patterns
- Use
DatResult<T, E>for typed success and failure handling - Prefer
fold,onSuccess, and the two-parameteronFailure { error, cause -> }overload, which hands you the typedDatErrorwith adescription getOrElse { }receives a rawThrowable, not a typedDatError, so it cannot readerror.description- Observe state with
StateFlowandFlow - Create a
DeviceSessionfirst, then attach capabilities such asCameraorDisplayafter it reachesSTARTED - Keep frame handling off the main thread when doing heavier processing
Error handling
lifecycleScope.launch {
Wearables.checkPermissionStatus(Permission.CAMERA)
.onSuccess { status -> /* handle status */ }
.onFailure { error, _ -> /* handle error */ }
}
Avoid getOrThrow() in user-facing samples. Surface typed errors from DatResult instead.
Naming conventions
| Type | Purpose | Example |
|---|---|---|
DeviceSession | Device connection lifecycle | Wearables.createSession(...) |
Camera | Camera capability on a session | session.addCamera(...) |
Stream | Video stream from a camera | camera.stream |
Display | Display capability on a session | session.addDisplay(...) |
*Selector | Device targeting | AutoDeviceSelector, SpecificDeviceSelector |
*Error | Typed failure surface | DeviceSessionError, StreamError, CaptureError |
Key types
Wearables— SDK entry pointDeviceSession— lifecycle for an interaction with a linked deviceCamera— camera capability attached to a sessionStream— video stream accessed throughcamera.streamDisplay— display capability attached to a sessionStreamConfiguration— video quality, frame rate, and compression configurationMockDeviceKit— simulated device environment for testing
Live docs search
If your editor supports remote MCP servers, connect https://mcp.developer.meta.com/wearables and use search_dat_docs for current DAT setup, session lifecycle, camera streaming, MockDeviceKit, permissions, and exact API symbols. This public docs server does not require authentication; do not configure tokens, OAuth, or custom authorization headers for it.
See the dat-docs-mcp skill for client setup and troubleshooting. Use llms.txt when your tool only supports static reference context.
Testing with MockDeviceKit
val mockDeviceKit = MockDeviceKit.getInstance(context)
mockDeviceKit.enable()
mockDeviceKit.pairGlasses(GlassesModel.RAYBAN_META).fold(
onSuccess = { device -> onMockDevicePaired(device) },
onFailure = { error, _ -> showError(error.description) },
)
Use MockDeviceKit to drive registration, device availability, streaming media, and permission scenarios without physical hardware.
Common pitfalls
- Do not call SDK APIs before
Wearables.initialize(context) - Do not call
addCamera(...)oraddDisplay()immediately aftersession.start(); wait forDeviceSessionState.STARTED - Do not assume a session implies streaming or display access; capabilities are attached separately
- Do not ignore
DatResultfailures fromcreateSession,addCamera,addDisplay,stream.start(), orcapturePhoto;session.start()returnsUnit, so watchsession.errorsfor its failures - Do not reuse terminally stopped sessions, cameras, or streams
Links
DAT Docs MCP (Android)
Use this skill whenever a user asks about the DAT MCP server, live docs search, current DAT documentation, API lookup, code examples, MCP setup, or MCP Inspector.
This is the documentation counterpart to live-debugging-mcp: use this skill to
look up published DAT documentation, and use live-debugging-mcp for observed
runtime behavior in a running app.
Endpoint
- MCP endpoint:
https://mcp.developer.meta.com/wearables - Transport: HTTP (Streamable HTTP)
- Authentication: none. Do not configure tokens, OAuth, or custom
Authorizationheaders for this server.
Connect to the MCP host above directly. Do not point an MCP client at the Wearables developer website URL; that is a docs site, not the MCP endpoint.
Tools
search_dat_docs— semantic search over DAT guides, API reference, and code examples.
The server may add more DAT MCP tools over time, so list the tools after connecting instead of assuming this is the only one.
Repo config vs. MCP
- Use the repo-local AI config (these skills,
.cursor/rules/,.github/copilot-instructions.md,AGENTS.md) for coding patterns, setup steps, and project conventions. - Use the MCP endpoint for live DAT documentation and exact current API symbols, especially when a symbol may have changed since this repo config was written.
Setup: Muse Code, Claude Code, and Codex
Installing the mwdat-android plugin registers the hosted DAT docs MCP server
in Muse Code, Claude Code, and Codex. No separate MCP configuration is required.
Setup: Cursor
Add an HTTP MCP server with:
- Name:
wearables-dat - Type/transport: HTTP
- URL:
https://mcp.developer.meta.com/wearables
For JSON-backed Cursor settings:
{
"mcpServers": {
"wearables-dat": {
"type": "http",
"url": "https://mcp.developer.meta.com/wearables"
}
}
}
Setup: MCP Inspector
npx @modelcontextprotocol/inspector
- Set transport to Streamable HTTP, or HTTP if that is the label in the installed Inspector.
- Set URL to
https://mcp.developer.meta.com/wearables. - Set connection type to Direct.
- Initialize, then list tools and confirm
search_dat_docsappears. - Run a query such as
camera streaming.
Example prompts
Search the Wearables DAT docs for camera streaming setup on Android.How do I stream camera on Ray-Ban Meta glasses?How do I initialize the Wearables SDK and access a connected device?How do I test DAT integrations without physical glasses?Bluetooth connection lifecycle events
Troubleshooting
- Server does not connect: re-enter
https://mcp.developer.meta.com/wearablesand reconnect. - No tools appear: reconnect and initialize the session before listing tools.
- The client asks for auth: remove any stale custom auth headers or tokens for this server and reconnect. This server does not require authentication.
- Results are too broad: include the platform, module, or exact API name in the
query, for example
Android StreamConfiguration frameRateinstead offrame rate.
Static fallback
When a tool cannot use remote MCP servers at all, point it at the llms.txt
index instead and follow its links: https://wearables.developer.meta.com/llms.txt.
Debugging (Android)
Diagnose common setup, session, and stream issues in DAT SDK integrations.
If local DAT Inspector MCP tools are available, use the live-debugging-mcp
skill before changing app code. Prefer get_dat_readiness,
get_companion_boundary_diagnosis, and get_device_path to separate app bugs from
Meta AI app/device boundary issues using read-only DAT evidence.
Quick diagnosis
No eligible device or session won't start?
|
+-- Did you call Wearables.initialize(context)? -> Must happen before SDK usage
|
+-- Did registration complete? -> Observe Wearables.registrationState
|
+-- Is Developer Mode enabled? -> Enable it in the Meta AI app for dev builds
|
+-- Does Wearables.devices contain a linked device? -> Check Bluetooth and range
|
+-- Did createSession() or addCamera() return a DatResult failure? -> Surface the typed error
Developer Mode
Developer Mode must be enabled for local development builds that use mwdat_application_id = 0 and mwdat_client_token = 0.
Symptoms when Developer Mode is disabled
- Registration flow completes but the device never becomes eligible
- Permission requests do not succeed for development builds
Wearables.createSession(...)fails with no eligible device
Watch for
- Developer Mode may reset after app or firmware updates
- Developer Mode is configured per linked device
- Production builds use a real
APPLICATION_ID,CLIENT_TOKEN, and release-channel gating instead
Session and stream issues
Session never reaches STARTED
- Verify
Wearables.registrationState - Check that
Wearables.devicescontains a compatible linked device - Ensure the glasses are powered on, unfolded, and in range
Stream never reaches STREAMING
- Call
session.addCamera(...)only aftersession.statereportsSTARTED;session.start()returnsUnit, so watchsession.errorsfor start failures - Check camera permission status through
Wearables.checkPermissionStatus(...) - Make sure
camera.stream.start()returned success
Photo capture fails
capturePhoto()only succeeds while the stream is actively streaming- Surface the returned
CaptureErrorinstead of discarding theDatResult
Version compatibility
Ensure compatible versions of the SDK, Meta AI app, and glasses firmware. See version dependencies for the current compatibility matrix.
Logging
private const val TAG = "DATWearables"
camera.stream.start()
.onFailure { error, _ -> Log.e(TAG, "Failed to start stream: ${error.description}") }
Prefer logging typed DatResult failures and observed state transitions over generic exceptions.
Live MCP evidence
When the developer has a local DAT debug server connected to the agent:
- Start with
get_sdk_stateandget_dat_readiness - Use
get_companion_boundary_diagnosisfor permission, app identity, device capability, device-selection, deeplink-return, and DAM/DWA-visible blockers - Use
get_device_pathto trace app -> DAT registration -> Meta AI app permissions -> device selection/link -> session -> stream - Ask the developer to reproduce the issue, then use
wait_for_eventswith the narrowest category or source that matches the flow - Use
export_diagnostic_bundlefor redacted support handoff
Do not treat this as companion app introspection. It is diagnosis from app-visible DAT debug events.
Checklist
-
Wearables.initialize(context)ran before SDK usage - Developer Mode enabled for development builds
-
APPLICATION_IDandCLIENT_TOKENmatch the build mode - Registration completed before session creation
- Bluetooth permission granted
- Camera permission granted through Meta AI
- Session and stream
DatResultfailures are surfaced in logs or UI
Links
Display Access (Android)
Use mwdat-display to render content on Meta Ray-Ban Display glasses.
Use this skill with getting-started and permissions-registration when creating a full app. A Display app still needs SDK initialization, app registration, Android permissions, and DAT manifest metadata before it can create a session.
Add the Display dependency
In libs.versions.toml:
[libraries]
mwdat-display = { group = "com.meta.wearable", name = "mwdat-display", version.ref = "mwdat" }
In app/build.gradle.kts:
dependencies {
implementation(libs.mwdat.core)
implementation(libs.mwdat.display)
}
Configure the app for Display
Display apps need the same core DAT setup as other apps:
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.INTERNET" />
<application ...>
<meta-data
android:name="com.meta.wearable.mwdat.APPLICATION_ID"
android:value="${mwdat_application_id}" />
<meta-data
android:name="com.meta.wearable.mwdat.CLIENT_TOKEN"
android:value="${mwdat_client_token}" />
</application>
Set mwdat_application_id and mwdat_client_token from Gradle manifest placeholders or local.properties, as in the DisplayAccess sample. Developer Mode builds can use the developer registration flow, but production builds need real Wearables Developer Center credentials.
Request the runtime permissions before initializing DAT, then call Wearables.initialize(context) once and start observing SDK state. Start registration with Wearables.startRegistration(activity), collect Wearables.registrationState, collect Wearables.registrationErrorStream, and wait for RegistrationState.REGISTERED before creating a display session. Use Wearables.startUnregistration(activity) when the user disconnects the app.
Select a display-capable device
Display content only works on connected, compatible devices whose type supports display. Use the public device filter when you want automatic selection:
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector
val selector = AutoDeviceSelector(filter = { device -> device.isDisplayCapable() })
val sessionResult = Wearables.createSession(selector)
Use SpecificDeviceSelector(selectedDeviceId) instead when your UI lets the user pick a specific DeviceIdentifier from Wearables.devices.
For device picker UI, mirror the DisplayAccess sample: collect Wearables.devices, start a metadata collection for each device ID from Wearables.devicesMetadata[id], remove metadata for devices that disappear, and show device name, device.deviceType.description, device.linkState, device.compatibility, and device.isDisplayCapable(). Keep selection disabled unless the device is LinkState.CONNECTED and display-capable, and surface DeviceCompatibility.DEVICE_UPDATE_REQUIRED with an openFirmwareUpdate action.
import com.meta.wearable.dat.core.types.Device
import com.meta.wearable.dat.core.types.DeviceIdentifier
import com.meta.wearable.dat.core.types.LinkState
import kotlinx.coroutines.Job
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.update
private val metadataJobs = mutableMapOf<DeviceIdentifier, Job>()
private val devicesMetadata = MutableStateFlow<Map<DeviceIdentifier, Device>>(emptyMap())
lifecycleScope.launch {
Wearables.devices.collect { deviceIds ->
(metadataJobs.keys - deviceIds).forEach { removedId ->
metadataJobs.remove(removedId)?.cancel()
devicesMetadata.update { current -> current - removedId }
}
deviceIds.forEach { id ->
metadataJobs.getOrPut(id) {
launch {
Wearables.devicesMetadata[id]?.collect { device ->
devicesMetadata.update { current -> current + (id to device) }
updateDeviceRow(
id = id,
name = device.name,
type = device.deviceType.description,
isConnected = device.linkState == LinkState.CONNECTED,
isDisplayCapable = device.isDisplayCapable(),
compatibility = device.compatibility,
)
}
}
}
}
}
}
If Wearables.createSession(...) fails with DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED, show a DAT glasses app update action that calls Wearables.openDATGlassesAppUpdate(activity).
Attach Display after the session starts
Add the Display capability only after the DeviceSession reaches DeviceSessionState.STARTED. Keep the sample app pattern of showing a "preparing" state after session start and enabling user content only once the Display capability reports DisplayState.STARTED.
import androidx.lifecycle.lifecycleScope
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.types.DeviceIdentifier
import com.meta.wearable.dat.core.types.DeviceSessionError
import com.meta.wearable.dat.core.types.RegistrationState
import com.meta.wearable.dat.core.session.DeviceSession
import com.meta.wearable.dat.core.session.DeviceSessionState
import com.meta.wearable.dat.core.selectors.SpecificDeviceSelector
import com.meta.wearable.dat.display.Display
import com.meta.wearable.dat.display.addDisplay
import com.meta.wearable.dat.display.removeDisplay
import com.meta.wearable.dat.display.types.DisplayState
import kotlinx.coroutines.launch
private var display: Display? = null
fun startDisplaySession(selectedDeviceId: DeviceIdentifier) {
if (Wearables.registrationState.value != RegistrationState.REGISTERED) {
showError("Register with Meta AI before starting Display")
return
}
Wearables.createSession(SpecificDeviceSelector(selectedDeviceId))
.fold(
onSuccess = { session ->
lifecycleScope.launch {
session.errors.collect { error ->
if (error == DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED) {
showDatAppUpdateAction()
}
showError(error.description)
}
}
lifecycleScope.launch {
session.state.collect { state ->
if (state == DeviceSessionState.STARTED && display == null) {
attachDisplay(session)
}
}
}
session.start()
},
onFailure = { error, _ ->
if (error == DeviceSessionError.DAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED) {
showDatAppUpdateAction()
}
showError(error.description)
},
)
}
private fun attachDisplay(session: DeviceSession) {
session.addDisplay()
.fold(
onSuccess = { newDisplay ->
display = newDisplay
lifecycleScope.launch {
newDisplay.state.collect { state ->
if (state == DisplayState.STARTED) {
setTryItEnabled(true)
} else {
setTryItEnabled(false)
}
}
}
},
onFailure = { error, _ ->
showError(error.description)
},
)
}
fun stopDisplaySession(session: DeviceSession) {
session.removeDisplay().onFailure { error, _ -> showError(error.description) }
session.stop()
display = null
}
Send display UI
Build exactly one root view per sendContent call. Each call replaces the previous content on the glasses. Use a root flexBox { ... } for UI, or a root video(player = player) for video; do not put video(...) inside a flexBox.
Button and clickable flexBox callbacks are routed back to the phone app. Keep callbacks fast and delegate to your ViewModel to send the next content screen, as the DisplayAccess sample does for list items, Back, Start, Previous, Next, Done, and Watch video buttons.
import com.meta.wearable.dat.display.views.ButtonStyle
import com.meta.wearable.dat.display.views.FlexBoxBackground
import com.meta.wearable.dat.display.views.IconName
import com.meta.wearable.dat.display.views.TextColor
import com.meta.wearable.dat.display.views.TextStyle
private suspend fun sendStatusCard(display: Display) {
display.sendContent {
flexBox(
gap = 12,
padding = 24,
background = FlexBoxBackground.CARD,
onClick = { showDetailState() },
) {
text("Bike ride", style = TextStyle.HEADING)
text("Turn right in 200 ft", style = TextStyle.BODY, color = TextColor.SECONDARY)
button(
label = "Done",
style = ButtonStyle.PRIMARY,
iconName = IconName.CHECKMARK,
onClick = { showDoneState() },
)
}
}.onFailure { error, _ ->
showError(error.description)
}
}
Use images and built-in icons
Use HTTPS image URLs for image content, and use IconName enum values for built-in icons. Do not invent string icon names.
import com.meta.wearable.dat.display.views.CornerRadius
import com.meta.wearable.dat.display.views.IconStyle
import com.meta.wearable.dat.display.views.ImageSize
display.sendContent {
flexBox(gap = 8, padding = 24) {
image(
uri = "https://example.com/thumbnail.png",
sizePreset = ImageSize.FILL,
cornerRadius = CornerRadius.MEDIUM,
)
icon(name = IconName.GEAR, style = IconStyle.FILLED)
text("Device settings", style = TextStyle.BODY)
}
}
Send video
For URL-based video, create a VideoPlayer, send it as the root content, then call play() after the send succeeds. Collect both player.state and player.error; on VideoPlayerState.ENDED, cancel the video observer and send the next display screen. Use an HTTP or HTTPS VideoSource.Url, VideoCodec.MP4 for MP4 assets, and a video sized within the public player limits.
import com.meta.wearable.dat.display.types.VideoCodec
import com.meta.wearable.dat.display.types.VideoPlayerState
import com.meta.wearable.dat.display.types.VideoSource
import com.meta.wearable.dat.display.views.VideoPlayer
import kotlinx.coroutines.Job
private var videoStateJob: Job? = null
private var videoErrorJob: Job? = null
lifecycleScope.launch {
val currentDisplay = display ?: return@launch
val player = VideoPlayer(
source = VideoSource.Url("https://example.com/tutorial.mp4"),
codec = VideoCodec.MP4,
)
videoStateJob?.cancel()
videoErrorJob?.cancel()
videoStateJob = launch {
player.state.collect { state ->
if (state == VideoPlayerState.ENDED) {
videoStateJob?.cancel()
videoStateJob = null
sendStatusCard(currentDisplay)
}
}
}
videoErrorJob = launch {
player.error.collect { error ->
if (error != null) {
showError(error.description)
}
}
}
currentDisplay.sendContent { video(player = player) }
.onSuccess { player.play() }
.onFailure { error, _ -> showError(error.description) }
}
Display rules
- Initialize DAT and complete registration before creating the session.
- Observe
Wearables.registrationErrorStream,session.errors,Display.sendContentfailures, andVideoPlayer.error;DeviceSession.start()returnsUnitand reports async failures throughsession.errors. - Use
SpecificDeviceSelectorfor a user-picked device andAutoDeviceSelector(filter = { it.isDisplayCapable() })only when automatic selection is acceptable. - Wait for
DeviceSessionState.STARTEDbefore callingsession.addDisplay(). - Wait for
DisplayState.STARTEDbefore enabling or sending user-triggered content. - If device metadata reports
DeviceCompatibility.DEVICE_UPDATE_REQUIRED, offerWearables.openFirmwareUpdate(activity). - If session creation or
session.errorsreportsDAT_APP_ON_THE_GLASSES_UPDATE_REQUIRED, offerWearables.openDATGlassesAppUpdate(activity). - Keep display callbacks fast; move app state changes back onto your UI layer as needed.
- Cancel state/error collection jobs, close or replace active
VideoPlayerinstances, detach withsession.removeDisplay(), and then stop the session when the display experience ends.
Sample app
Use the Display Access sample app for a complete flow: registration, device selection, display attachment, interactive content, and video.
Links
Getting Started with DAT SDK (Android)
Set up the Meta Wearables Device Access Toolkit in an Android app.
Prerequisites
- Android Studio Flamingo or newer
- Kotlin, Android Gradle Plugin and JDK versions matching the CameraAccess sample or newer (see its
gradle/libs.versions.tomlandapp/build.gradle.kts) - Android 10+ test device with the Meta AI app installed
- Supported Meta glasses or MockDeviceKit for local testing
- Developer Mode enabled in the Meta AI app for development builds
Step 1: Declare dependencies
The SDK is published to Maven Central under the com.meta.wearable group, so no access token or extra repository definition is required. The mavenCentral() repository in your settings.gradle.kts is all you need.
In libs.versions.toml:
[versions]
mwdat = "1.1.0"
[libraries]
mwdat-core = { group = "com.meta.wearable", name = "mwdat-core", version.ref = "mwdat" }
mwdat-camera = { group = "com.meta.wearable", name = "mwdat-camera", version.ref = "mwdat" }
mwdat-display = { group = "com.meta.wearable", name = "mwdat-display", version.ref = "mwdat" }
mwdat-mockdevice = { group = "com.meta.wearable", name = "mwdat-mockdevice", version.ref = "mwdat" }
In app/build.gradle.kts:
android {
defaultConfig {
manifestPlaceholders["mwdat_application_id"] = "0"
manifestPlaceholders["mwdat_client_token"] = "0"
}
}
dependencies {
implementation(libs.mwdat.core)
implementation(libs.mwdat.camera)
implementation(libs.mwdat.display)
implementation(libs.mwdat.mockdevice)
}
Step 2: Configure AndroidManifest.xml
<manifest ...>
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.INTERNET" />
<application ...>
<meta-data
android:name="com.meta.wearable.mwdat.APPLICATION_ID"
android:value="${mwdat_application_id}" />
<meta-data
android:name="com.meta.wearable.mwdat.CLIENT_TOKEN"
android:value="${mwdat_client_token}" />
<activity android:name=".MainActivity" ...>
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="myexampleapp" />
</intent-filter>
</activity>
</application>
</manifest>
APPLICATION_ID and CLIENT_TOKEN are used for app attestation and can be found in the Wearables Developer Center. In Developer Mode, attestation is not used, so the manifest placeholders can both be 0. For production, replace both placeholders with the credentials for your Wearables Developer Center app. Replace myexampleapp with your app's URL scheme.
Step 3: Initialize the SDK
import android.util.Log
import com.meta.wearable.dat.core.Wearables
class MyApplication : Application() {
override fun onCreate() {
super.onCreate()
Wearables.initialize(this)
.onFailure { error, _ ->
Log.e("DATWearables", "Failed to initialize DAT: ${error.description}")
}
}
}
Step 4: Register and create a session
Registration must complete before a session can start. Observe Wearables.registrationState and wait for RegistrationState.REGISTERED.
import android.app.Activity
import androidx.lifecycle.lifecycleScope
import com.meta.wearable.dat.core.Wearables
import com.meta.wearable.dat.core.selectors.AutoDeviceSelector
import com.meta.wearable.dat.core.session.DeviceSession
import com.meta.wearable.dat.core.session.DeviceSessionState
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.launch
fun connect(activity: Activity) {
Wearables.startRegistration(activity)
}
fun startSession() {
Wearables.createSession(AutoDeviceSelector()).fold(
onSuccess = { session ->
lifecycleScope.launch {
session.errors.collect { error -> showError(error.description) }
}
lifecycleScope.launch {
session.state.first { state -> state == DeviceSessionState.STARTED }
addCameraStreaming(session)
}
// Fire-and-forget: returns Unit and reports failures through session.errors.
session.start()
},
onFailure = { error, _ -> showError(error.description) },
)
}
Observe registration and available devices:
lifecycleScope.launch {
Wearables.registrationState.collect { state ->
// Update registration UI
}
}
lifecycleScope.launch {
Wearables.devices.collect { devices ->
// Update the device list
}
}
Step 5: Add camera streaming
Call this only once the session reports DeviceSessionState.STARTED; adding a capability earlier fails with DeviceSessionError.SESSION_IDLE.
import com.meta.wearable.dat.camera.addCamera
import com.meta.wearable.dat.camera.types.StreamConfiguration
import com.meta.wearable.dat.camera.types.VideoQuality
fun addCameraStreaming(session: DeviceSession) {
session.addCamera(
StreamConfiguration(videoQuality = VideoQuality.MEDIUM, frameRate = 24),
).fold(
onSuccess = { camera ->
camera.stream.start().onFailure { error, _ -> showError(error.description) }
},
onFailure = { error, _ -> showError(error.description) },
)
}
Camera access also needs the DAT camera permission; see the permissions-registration skill.
Next steps
camera-streaming— Stream capability, video frames, photo capturemockdevice-testing— Test without hardwaresession-lifecycle— Handle session and stream state changespermissions-registration— Registration and permission flowsdisplay-access— Render content on Meta Ray-Ban Display glasses- Full Android API reference
Inputs (Android)
Use the experimental Inputs capability to receive semantic interactions from glasses. The app must have Inputs enabled for it in Wearables Developer Center. Inputs does not have a Permission value to request at runtime.
Attach after the session starts
Call addInputs only after DeviceSessionState.STARTED. Adding the capability starts it automatically; there is no separate public start call.
session.addInputs(
InputsConfiguration(consumeBack = true),
).fold(
onSuccess = { inputs ->
inputEventsJob = lifecycleScope.launch {
inputs.events.collect { event ->
when (event) {
is InputEvent.Nav -> moveFocus(event.direction)
is InputEvent.Select -> activateFocusedItem()
is InputEvent.Back -> dismissCurrentView()
is InputEvent.Button -> handleButton(event.button)
is InputEvent.Capture -> handleCaptureButton(event.pressType)
is InputEvent.Drag -> updateDrag(
event.action,
event.x,
event.y,
event.dx,
event.dy,
)
}
}
}
},
onFailure = { error, _ -> showError(error.description) },
)
Retain the event job while Inputs is attached. Collect state when the UI needs activation state and errors for activation, connection, and communication failures.
The default configuration enables every known source except UNKNOWN and consumes Back. Set consumeBack to false when the system should handle Back.
Each event includes its source and device timestamp in milliseconds. Keep the when exhaustive so new behavior is intentional.
Errors and cleanup
InputsError.PERMISSION_DENIED means the capability was not approved for the app; it is not fixed with requestPermission. Other errors cover activation, connection, and communication failures.
Cancel the collection jobs, then call session.removeInputs() and handle its DatResult before adding another Inputs capability.
Test with MockDeviceKit
After the app has attached Inputs, inject events through the paired mock glasses with calls such as glasses.services.input.navDown(). The service also exposes Select, Back, Button, Capture, and Drag injection. Calls made before Inputs is active, or from a source excluded by the configuration, are dropped.
Availability
Inputs is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
Live DAT Debugging with MCP (Android)
Use this skill when a developer is debugging an Android DAT app and has a local
DAT Inspector MCP server or DAT debug server available to the agent. This is
the live-runtime counterpart to dat-docs-mcp: use docs search for API lookup,
and use this skill for observed app/device behavior.
Ground rules
- Stay read-only unless the user explicitly asks for an app code change.
- Do not mutate Meta AI app, device, account, permission, registration, or app state.
- Treat companion-app/device results as boundary diagnosis from app-visible DAT events, not as direct access to companion app internals.
- Use live DAT evidence before guessing: registration, permissions, device selection, link state, session state, stream state, typed errors, and SDK logs.
- If the MCP tools are not configured, ask the developer to enable their local
DAT debug setup and fall back to logs plus
search_dat_docs.
Normal agent loop
- Discover and connect to the running app debug server:
discover_debug_serversconnect_to_debug_server
- Establish the baseline with all three calls before diagnosing deeper:
get_connection_statusget_sdk_stateget_dat_readinessDo not skipget_connection_status; it separates MCP/debug-server connectivity from SDK, registration, permission, session, and stream state.
- If the issue looks like Meta AI app or device boundary behavior, inspect:
get_companion_boundary_diagnosisget_device_pathget_permissionsget_device_properties
- Ask the developer to reproduce the failing Android flow while connected.
- Wait for narrow evidence instead of polling everything:
wait_for_eventswithcategoryorsourceget_errorsget_event_digest
- For handoff, collect a redacted bundle:
export_diagnostic_bundle
Evidence map
- Initialization: app logs around
Wearables.initialize(context) - Registration:
Wearables.registrationStateandwearables.registrationState - Device availability:
Wearables.devicesplus debug events fromwearables.devices,device.compatibility, anddevice.properties - Device link:
device.linkState - Permissions:
Wearables.checkPermissionStatus(...),device.permission,check_permissions, andget_permissions - Session:
createSession(...),session.start(),deviceSession.state,session.state, and session errors - Stream:
session.addCamera(...),camera.stream.start(),camera.stream.state, frame counters, and stream errors - DAM/DWA-visible transport: DAM/DWA error values
Diagnosis patterns
- Registration blocked: verify Android manifest metadata, app credentials or Developer Mode setup, and callback/deeplink handling.
- No eligible device: inspect
Wearables.devices, device compatibility, anddevice.linkStatebefore changing selector code. - Permission failure: use
get_permissionsandget_companion_boundary_diagnosis; do not assume Android runtime permission is the same as DAT permission through the Meta AI app/device boundary. - Session fails before stream creation: inspect session errors and selected device compatibility before changing camera code.
- Stream fails after session start: inspect stream state, camera permission, and recent errors before changing frame processing.
Output expectations
When summarizing, name the evidence plane, the blocking state, and the next developer action. Example:
DAT readiness is blocked at permissions: camera is denied in app-visible DAT
events. Grant the DAT camera permission in Meta AI, return to the app, then
retry session.start() and camera.stream.start() while the MCP connection waits for
permission and stream events.
MockDevice MCP (Android)
Use this skill when you want to verify DAT app behavior without physical glasses by controlling a simulated device over the MockDevice MCP server. The MCP is a client of the app's in-app HTTP test server; it drives device state, and you observe the result with the existing Android screenshot / UI / logcat tools.
No third-party dependencies — the server runs on the Python 3.8+ standard
library, launched with python3. The MCP server is
mcp/mockdevice/mockdevice_mcp_server.py.
Prerequisite: the app must start its MockDevice test server
The test server starts when the app calls startTestServer — typically a debug build
launched with the mwdat_ui_testing intent extra, which also puts MockDeviceKit into
mock mode, as the DAT sample apps (e.g. CameraAccess) do. You are driving a simulated
device, not real glasses.
Launch (or relaunch) the app that way with adb, e.g.:
adb shell am start -n <package>/<launcher-activity> --ez mwdat_ui_testing true
If mock_connect reports no server, relaunch with that extra and try again.
Normal agent loop
- Make sure the app is running its MockDevice test server (sample apps: launch with the
mwdat_ui_testingextra, see above). mock_connect(bundle_id="<your app package>", platform="android")— auto-detects the attached device, forwards a local port to the app's fixed test-server loopback port overadb, and verifies health. Passudid(the device serial) if more than one device/emulator is attached.- Drive device state, e.g.:
mock_pair_device(), thenmock_power_on()andmock_don()— pairs Ray-Ban Meta, powers it on, and wears it. A newly paired device is off, folded, and not worn.mock_set_camera_feed(resource_name="plant", ext="mp4")— the app must bundle the feed as an asset, for exampleapp/src/debug/assets/plant.mp4.mock_power_off/mock_power_on,mock_doff/mock_don,mock_fold/mock_unfold,mock_captouch_tap.
- Observe the effect with the Android screenshot / UI / logcat tools and
mock_device_state.
device_id is optional on device commands — it is auto-resolved when exactly one
device is paired.
Tools
The server advertises every tool with its arguments and a description of how to verify
its effect, so read them from your tool list. Each tool name starts with mock_.
Notes
- Camera feeds name an asset bundled in the app, by name and extension — not an
arbitrary file path, and not a
res/rawresource. The app's test server opens<resource_name>.<ext>withAssetManager, so a file outsideassets/is reported asresource not found. The same applies to captured images and motion feeds. mock_connectrefuses an app whose DAT SDK serves a different MockDevice route contract than this MCP calls, and says whether to update the app or the plugin. Relay that to the developer rather than retrying.mock_connectalso reports the routes the attached app advertises. A tool added in a later SDK release than the app uses reports that it is unavailable instead of failing opaquely.- Apart from
mock_display_content, which reads the display back, these tools drive device state only. To assert other outcomes (a stream started, a photo was captured), read the app with the Android screenshot / UI / logcat tools. - Display tools need a display-capable device: pair with
mock_pair_device(model="metaRayBanDisplay"), not the defaultrayBanMeta.mock_display_contentreturns the display content the app sent, as a node tree — the semantic content, not a rendered image. It is a debug payload rather than a stable API, so read the document you get back instead of assuming field names. For pixels, host the SDK's own preview view and screenshot it; that renders the same content on the phone and is what a developer looks at. Click identifiers are content-derived and change on every redraw — take a fresh one from the tree each time. Amock_display_clickok means the tap was delivered, not that it matched an element, so confirm by re-reading and checking the revision moved.
MockDevice Testing (Android)
Use MockDeviceKit to test DAT SDK integrations without physical Meta glasses.
MockDeviceKit simulates Meta glasses behavior for development and testing. It provides:
MockDeviceKit— Entry point for creating simulated devicesMockGlasses— Simulated Ray-Ban Meta glassesMockCameraKit— Simulated camera with configurable video feed and photo capture
Setup
Add mwdat-mockdevice to your Gradle dependencies:
dependencies {
implementation(libs.mwdat.mockdevice)
}
Creating a mock device
import com.meta.wearable.dat.mockdevice.MockDeviceKit
import com.meta.wearable.dat.mockdevice.api.GlassesModel
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitConfig
val mockDeviceKit = MockDeviceKit.getInstance(context)
// Attach fake registration and connectivity (auto-initializes Wearables if needed).
// By default, Wearables.registrationState transitions to Registered.
mockDeviceKit.enable()
// Or start in unregistered state to test registration flows:
// mockDeviceKit.enable(MockDeviceKitConfig(initiallyRegistered = false))
val device = mockDeviceKit.pairGlasses(GlassesModel.RAYBAN_META).getOrThrow()
You can check mockDeviceKit.isEnabled to query whether the mock environment is active.
Simulating device states
// Simulate glasses lifecycle
device.powerOn()
device.unfold()
device.don() // Simulate wearing the glasses
// Later...
device.doff() // Simulate removing
device.fold()
device.powerOff()
Configuring permissions
MockDeviceKit provides permissions to control permission behavior without the Meta AI app.
By default, RequestPermissionContract returns Granted. Use set() to control checkPermissionStatus() and setRequestResult() to control request outcomes.
val mockDeviceKit = MockDeviceKit.getInstance(context)
// Simulate denied camera permission status
mockDeviceKit.permissions.set(Permission.CAMERA, PermissionStatus.Denied)
// Simulate denied request result (user tapping "deny")
mockDeviceKit.permissions.setRequestResult(Permission.CAMERA, PermissionStatus.Denied)
Setting up mock camera feeds
Video streaming
val camera = device.services.camera
camera.setCameraFeed(videoUri)
Photo capture
val camera = device.services.camera
camera.setCapturedImage(imageUri)
Note: Android doesn't transcode video automatically. Mock video files must be in h.265 format: the SDK requests HEVC, so MockDeviceKit rejects an h.264 file when the stream starts. Use FFmpeg to convert:
ffmpeg -i input.mp4 -c:v libx265 -tag:v hvc1 -an -vf "scale=540:960" output.mp4
On macOS, -c:v hevc_videotoolbox uses the hardware encoder instead.
Phone camera as the feed
setCameraFeed(CameraFacing) sources frames from the handset camera, so the app needs android.permission.CAMERA in the manifest and at runtime. That is not the DAT Permission.CAMERA covered by MockDeviceKitConfig(initialPermissionsGranted = true) — no real-glasses path needs the handset grant, so an existing integration will never have requested it.
Grant it before calling setCameraFeed; in instrumentation tests, pm grant <your.package> android.permission.CAMERA. Without it the mock rejects the stream start and the failure names the missing permission. setCameraFeed(Uri) has no such requirement.
Writing instrumentation tests
Create a reusable test base class:
import android.content.Context
import androidx.test.ext.junit.rules.ActivityScenarioRule
import androidx.test.platform.app.InstrumentationRegistry
import com.meta.wearable.dat.mockdevice.MockDeviceKit
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitConfig
import com.meta.wearable.dat.mockdevice.api.MockDeviceKitInterface
import org.junit.After
import org.junit.Before
import org.junit.Rule
open class MockDeviceKitTestCase<T : Any>(
private val activityClass: Class<T>
) {
@get:Rule
val scenarioRule = ActivityScenarioRule(activityClass)
protected lateinit var mockDeviceKit: MockDeviceKitInterface
protected lateinit var targetContext: Context
protected open val mockDeviceKitConfig = MockDeviceKitConfig()
@Before
open fun setUp() {
val instrumentation = InstrumentationRegistry.getInstrumentation()
targetContext = instrumentation.targetContext
mockDeviceKit = MockDeviceKit.getInstance(targetContext)
grantRuntimePermissions()
mockDeviceKit.enable(mockDeviceKitConfig)
}
@After
open fun tearDown() {
mockDeviceKit.disable()
}
private fun grantRuntimePermissions() {
val packageName = targetContext.packageName
val shell = InstrumentationRegistry.getInstrumentation().uiAutomation
shell.executeShellCommand("pm grant $packageName android.permission.BLUETOOTH_CONNECT")
shell.executeShellCommand("pm grant $packageName android.permission.CAMERA")
}
}
To start a test in another state, override mockDeviceKitConfig in the subclass, for example with MockDeviceKitConfig(initiallyRegistered = false). Don't call enable again inside the test: once the kit is enabled, further enable calls are ignored until disable().
Using MockDeviceKit in the CameraAccess sample
The CameraAccess sample app includes a Debug menu for MockDeviceKit:
- Tap the Debug icon to open the MockDeviceKit menu
- Tap Pair RayBan Meta to create a simulated device
- Use PowerOn, Unfold, Don to simulate glasses states
- Select video/image files for mock camera feeds
- Start streaming to see simulated frames
Supported media formats
| Type | Formats |
|---|---|
| Video | h.265 (HEVC) |
| Image | JPEG, PNG |
Links
Motion (Android)
Motion is an experimental capability for streaming inertial sensor samples. Enable it for the app in Wearables Developer Center. It has no runtime Permission value.
Attach, collect, and start
Call addMotion only after DeviceSessionState.STARTED. The returned capability starts in STOPPED; launch collectors before calling start().
session.addMotion(
MotionConfiguration(samplingRate = MotionSamplingRate.HZ_30),
).fold(
onSuccess = { motion ->
motionSampleJob = lifecycleScope.launch {
motion.samples.collect { sample -> render(sample) }
}
motion.start()
},
onFailure = { error, _ -> showError(error.description) },
)
Retain the sample job while streaming. Collect state for lifecycle updates and errors for sensor, connection, and closure failures.
Sampling rates are 5, 10, 15, 24, 30, and 60 Hz; 10 Hz is the default.
Read samples defensively
timestampNs uses the device monotonic clock. Accelerometer values are m/s², gyroscope values are rad/s, and magnetometer values are µT. Each sensor value and orientation can be null; source identifies glasses, Neural Band, or an unknown source.
Motion normally transitions STOPPED -> STARTING -> STARTED. When its parent session pauses, it reports MotionState.PAUSED and automatically restarts when the session resumes.
Stop and remove
motion.stop() is reusable. Cancel the sample job before calling session.removeMotion() and handle its DatResult; removal is terminal.
Test with MockDeviceKit
Replay an in-memory recording with glasses.services.motion.setMotionFeed(samples, loop = true) or provide a CSV file URI. App logic should observe the SDK Motion state rather than isStreaming, which is a test assertion helper.
Availability
Motion is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
Permissions & Registration (Android)
Register your app with Meta AI, then request the device permissions it needs.
The DAT SDK separates two steps:
- Registration: The user connects your app to Meta AI.
- Device permissions: After registration, your app requests capabilities such as camera access.
Both flows depend on the Meta AI app being installed on the phone.
Start registration
Wearables.startRegistration(activity)
Observe registration state:
lifecycleScope.launch {
Wearables.registrationState.collect { state ->
// Update your registration UI
}
}
To unregister:
Wearables.startUnregistration(activity)
Check permission status
checkPermissionStatus(...) is a suspend API that returns a DatResult.
lifecycleScope.launch {
Wearables.checkPermissionStatus(Permission.CAMERA)
.onSuccess { status ->
if (status == PermissionStatus.Granted) {
startStreaming()
}
}
.onFailure { error, _ ->
showPermissionError(error.description)
}
}
Request a permission
Use Wearables.RequestPermissionContract() with the Activity Result API:
private val permissionLauncher =
registerForActivityResult(Wearables.RequestPermissionContract()) { result ->
result.onSuccess { status ->
if (status == PermissionStatus.Granted) {
startStreaming()
}
}.onFailure { error, _ ->
showPermissionError(error.description)
}
}
fun requestCameraPermission() {
permissionLauncher.launch(Permission.CAMERA)
}
Users can allow once or allow always through the Meta AI flow.
Developer Mode vs production
| Mode | Registration behavior |
|---|---|
| Developer Mode | Use mwdat_application_id = 0 and mwdat_client_token = 0 manifest placeholders for local development |
| Production | Use the application ID and client token assigned in the Wearables Developer Center |
For development builds, enable Developer Mode in the Meta AI app before testing registration and permissions.
Prerequisites
- Internet connection for registration
- Meta AI app installed on the phone
- Callback URI scheme configured in
AndroidManifest.xml - Bluetooth permission granted on Android
Links
Sample App Guide (Android)
Build an Android DAT app with registration, sessions, camera streaming, and photo capture.
Pair this with the CameraAccess sample.
Project setup
- Create an Android Studio app project.
- Declare the SDK dependencies in
libs.versions.tomland your app'sbuild.gradle.kts. - Configure
AndroidManifest.xmlfor registration callbacks plusAPPLICATION_IDandCLIENT_TOKEN. - Initialize
Wearablesin yourApplication.
Suggested app structure
app/src/main/java/com/example/myapp/
├── MyApplication.kt
├── MainActivity.kt
├── session/
│ └── SessionViewModel.kt
└── ui/
├── RegistrationScreen.kt
└── CameraScreen.kt
Registration and session creation
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
lifecycleScope.launch {
Wearables.registrationState.collect { state ->
// Update registration UI
}
}
}
fun register() {
Wearables.startRegistration(this)
}
}
DeviceSession.start() is fire-and-forget. Attach the camera from the session-state collector once the session reports DeviceSessionState.STARTED; adding a capability to an IDLE session fails with DeviceSessionError.SESSION_IDLE.
class SessionViewModel : ViewModel() {
private var session: DeviceSession? = null
private var camera: Camera? = null
fun startCameraSession() {
Wearables.createSession(AutoDeviceSelector()).fold(
onSuccess = { createdSession ->
session = createdSession
viewModelScope.launch {
createdSession.errors.collect { error -> showError(error.description) }
}
viewModelScope.launch {
createdSession.state.collect { state ->
when (state) {
DeviceSessionState.STARTED -> attachCamera(createdSession)
DeviceSessionState.STOPPED -> clearSession(createdSession)
else -> Unit
}
}
}
createdSession.start()
},
onFailure = { error, _ -> showError(error.description) },
)
}
private fun attachCamera(activeSession: DeviceSession) {
if (camera != null) return
activeSession.addCamera(
StreamConfiguration(videoQuality = VideoQuality.MEDIUM, frameRate = 24),
).fold(
onSuccess = { addedCamera ->
camera = addedCamera
viewModelScope.launch {
addedCamera.stream.videoStream.collect { frame ->
// Render preview
}
}
addedCamera.stream.start().onFailure { error, _ ->
showError(error.description)
}
},
onFailure = { error, _ -> showError(error.description) },
)
}
private fun clearSession(stoppedSession: DeviceSession) {
if (session !== stoppedSession) return
camera = null
session = null
}
}
Capture photos
fun capturePhoto() {
viewModelScope.launch {
camera?.stream?.capturePhoto()
?.onSuccess { photoData ->
when (photoData) {
is PhotoData.Bitmap -> savePhoto(photoData.bitmap)
is PhotoData.HEIC -> saveHeic(photoData.data)
}
}
?.onFailure { error, _ ->
showCaptureError(error.description)
}
}
}
Shutdown
fun stopCameraSession() {
camera?.stop()
session?.stop()
camera = null
session = null
}
Testing with MockDeviceKit
Use MockDeviceKit to simulate linking glasses, permission state, and camera media without physical hardware. See the mockdevice-testing skill for setup details.
Links
Session Lifecycle (Android)
Manage session and stream state in DAT SDK integrations.
Create a DeviceSession with Wearables.createSession(...), start it, then attach capabilities such as camera streaming once the session is STARTED. Session lifecycle and capability lifecycle are related but distinct.
Session states
| State | Meaning | App action |
|---|---|---|
IDLE | Session created, not started yet | Call session.start() |
STARTING | Connecting to the device | Show loading UI |
STARTED | Session active and ready for capabilities | Add or use capabilities |
PAUSED | Session temporarily suspended by pause() or the device | Keep state, call session.resume() or wait for the device to resume |
STOPPING | Session is shutting down | Stop user work and wait |
STOPPED | Terminal; session ended | Release resources and create a new session if needed |
STOPPED is terminal: a stopped session cannot be restarted, so call Wearables.createSession(...) again for the next connection.
Observe session state
start(), stop(), pause(), and resume() are all sync fire-and-forget and return Unit. Subscribe to state and errors before calling start() so no transition is missed, and treat session.errors as the failure channel for start().
Wearables.createSession(AutoDeviceSelector()).fold(
onSuccess = { session ->
lifecycleScope.launch {
session.state.collect { state ->
when (state) {
DeviceSessionState.STARTED -> onStarted(session)
DeviceSessionState.PAUSED -> onPaused()
DeviceSessionState.STOPPED -> onStopped()
else -> Unit
}
}
}
lifecycleScope.launch {
session.errors.collect { error -> showError(error.description) }
}
session.start()
},
onFailure = { error, _ -> showError(error.description) },
)
Add each capability once from onStarted(session): a resumed session goes from PAUSED back to STARTED and keeps its capabilities, so a second session.addCamera(...) fails with DeviceSessionError.CAPABILITY_ALREADY_ADDED. Calling session.addCamera(...) while the session is still IDLE fails with DeviceSessionError.SESSION_IDLE, and calling it after STOPPED fails with DeviceSessionError.SESSION_ALREADY_STOPPED.
Stream state
Camera streaming has its own state flow after you add a camera:
STOPPED -> STARTING -> STARTED -> STREAMING -> STOPPING -> STOPPED
CLOSED is the terminal stream state, and PAUSED is reported when the device pauses streaming (for example a single cap-touch tap) and can resume on its own.
lifecycleScope.launch {
camera.stream.state.collect { state ->
// React to stream state changes
}
}
Common transitions
The SDK may pause or stop a session when:
- Another experience takes over the device
- The user removes or folds the glasses
- Bluetooth connectivity drops
- The user unregisters the app or revokes needed access
Pause and resume
Call pause() on a STARTED session and resume() on a PAUSED one. Like start() and stop(), both are sync fire-and-forget: observe state for the STARTED -> PAUSED -> STARTED transitions. pause() outside STARTED and resume() outside PAUSED are no-ops. The state changes only when the device confirms, so wait for PAUSED before calling resume() (and for STARTED before calling pause() again); earlier calls are ignored. If the observed state doesn't match what your app wants, call pause() or resume() again once it settles.
pauseButton.setOnClickListener { session.pause() } // STARTED -> PAUSED
resumeButton.setOnClickListener { session.resume() } // PAUSED -> STARTED
The device can also pause and resume the session on its own (for example a single cap-touch tap). When a session is paused:
- The device connection may remain active
- Attached capabilities stop doing useful work
- Your app should not restart the session; call
resume()or wait for the next observed session state
Device availability
lifecycleScope.launch {
Wearables.devices.collect { devices ->
// Update the list of available devices
}
}
Use Wearables.devices and device metadata to decide when it is sensible to create a new session after a stop.
Checklist
- Handle all
DeviceSessionStatevalues you care about - Observe stream state separately from session state
- Release resources only after stop or close
- Recreate sessions after terminal stops instead of reusing dead ones
- Surface typed
DeviceSessionErrorandStreamErrorfailures
Links
Speech (Android)
Speech is an experimental on-device recognition capability. Enable it for the app in Wearables Developer Center and obtain Permission.MICROPHONE before starting it.
Permission flow
Check permission before attaching Speech. Launch Wearables.RequestPermissionContract only from a user-initiated action; it opens the Meta AI app. Android RECORD_AUDIO is not required for real-glasses recognition. It is needed only when a MockDevice test deliberately uses live phone recognition.
Attach, collect, and start
Call addSpeech after DeviceSessionState.STARTED. Launch collectors before start().
session.addSpeech().fold(
onSuccess = { speech ->
transcriptionJob = lifecycleScope.launch {
speech.transcriptions.collect { result ->
result ?: return@collect
if (result.isFinal) commitTranscript(result.text)
else showPartialTranscript(result.text)
}
}
speech.start().onFailure { error, _ -> showError(error.description) }
},
onFailure = { error, _ -> showError(error.description) },
)
Retain the transcription job while Speech is attached. Collect state, locale, and errors when the app needs lifecycle, locale, or failure updates.
confidence is between 0 and 1, or -1 when unavailable. errorDetails is diagnostic text and should not be displayed directly to end users.
Stop and remove
speech.stop() is reusable; removal is terminal. Cancel all collection jobs before session.removeSpeech().
Test with MockDeviceKit
After Speech is listening, inject results with calls such as glasses.services.speech.simulateTranscription("turn left", isFinal = true, confidence = 0.9f). The service can also inject locale changes, partial text, completion, and terminal errors. The live-device source and host RECORD_AUDIO permission are only for manual MockDevice testing.
Availability
Speech is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
Voice Invocations (Android)
Voice Invocations lets Meta AI launch or foreground an app when the wearer says “Hey Meta, start {app name}.” It is a Wearables-level stream, not a DeviceSession capability, and does not require a running device session.
Configure the app
Register the package name, request Voice Invocation approval, and configure the spoken app name in Wearables Developer Center. Do not declare invocation phrases in the manifest. Voice Invocations does not require camera or microphone permission.
Detect cold and warm launches
Use the SDK validator in both entry points. Do not inspect intent extras yourself.
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
if (isVoiceInvocationsIntent(intent)) startYourExperience()
}
override fun onNewIntent(intent: Intent) {
super.onNewIntent(intent)
setIntent(intent)
if (isVoiceInvocationsIntent(intent)) startYourExperience()
}
Start and retain the stream
Store one app-scoped stream. The factory starts it automatically; there is no separate public start call.
stream = Wearables.startVoiceInvocationsStream(AutoDeviceSelector())
invocationJob = appScope.launch {
stream.invocations.collect { invocation ->
when (invocation) {
is LaunchApp -> {
navigateToMainScreen()
invocation.responseHandle.sendSuccess(actionOutput = null)
}
}
}
}
Collect state and errors when the app needs lifecycle or failure updates.
Every delivered invocation must be answered exactly once with sendSuccess or sendFailure. Both are suspend functions returning whether delivery succeeded.
The stream reports STARTING, STARTED, and STOPPED. Creation and automatic startup do not throw; failures surface through errors and state.
Close and test
close() is terminal. Cancel collectors before closing the app-scoped stream when the app no longer listens.
To test without hardware, check glasses.services.voiceInvocation.hasConnectedApps before calling simulateLaunchAppAction(). Use simulateIncompleteAction() to verify malformed actions surface an error rather than an invocation.
Availability
Voice Invocations is experimental. Apps can use it for development and beta testing, but cannot publish it to production release channels yet.
See the Voice Invocations guide.
MCP servers 2
Sponsors
Details
- Marketplace
- meta/meta-wearables-dat-android
- Source
- plugins/mwdat-android
- Author
- Meta Wearables
- Category
- Community