lars/splat-indoor-navigation
1
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity

Decide: how the user's pose origin gets fixed, and re-fixed as it drifts #15

New Issue
Open
opened 2026-08-11 15:52:11 +00:00 by lars · 1 comment
lars commented 2026-08-11 15:52:11 +00:00
Owner
Copy Link
Copy Source

Question

Surfaced by #2: there is no mobile splat relocalizer, so nothing hands the app an absolute pose for free. ARKit VIO tracks relative motion accurately (published indoor drift ~0.5% of distance travelled) but must be told where it started, and drift accumulates from there. This makes origin acquisition and re-anchoring a first-class product decision rather than an implementation detail — and it has to be settled before the bake-off can be designed.

Settle:

  • Initial fix. How does a session learn where the user is? Candidates: user taps a recognizable landmark on a deck map ("I'm at the aft elevator lobby"), scans a QR/marker placed at known points, a BLE zone fix, or a hosted VPS relocalization.
  • Re-anchoring. VIO drift means the fix decays. Is re-anchoring periodic (pass a known marker/beacon), opportunistic, or user-triggered when the render visibly disagrees with reality?
  • What the passenger will tolerate. A cruise passenger who must scan a QR code to start navigating may simply not bother. Weigh this against accuracy honestly — it may be the deciding constraint.
  • Physical infrastructure implied. Markers are cheap, passive, and need no batteries; beacons need maintenance. The project goal is minimal added hardware, and printed markers may satisfy it better than BLE.
  • Consequence for the pose contract (#7): whether pose carries a "time since last fix" or drift-estimate field so the renderer can degrade as confidence decays.

Note the interaction with mirrored ships: a wrong initial fix in a mirrored layout is not slightly wrong, it is confidently wrong on the other side of the vessel. Whatever is chosen must fail loudly rather than quietly.

Part of the wayfinder map #1.

## Question Surfaced by #2: there is no mobile splat relocalizer, so nothing hands the app an absolute pose for free. ARKit VIO tracks *relative* motion accurately (published indoor drift ~0.5% of distance travelled) but must be **told where it started**, and drift accumulates from there. This makes origin acquisition and re-anchoring a first-class product decision rather than an implementation detail — and it has to be settled before the bake-off can be designed. Settle: - **Initial fix.** How does a session learn where the user is? Candidates: user taps a recognizable landmark on a deck map ("I'm at the aft elevator lobby"), scans a QR/marker placed at known points, a BLE zone fix, or a hosted VPS relocalization. - **Re-anchoring.** VIO drift means the fix decays. Is re-anchoring periodic (pass a known marker/beacon), opportunistic, or user-triggered when the render visibly disagrees with reality? - **What the passenger will tolerate.** A cruise passenger who must scan a QR code to start navigating may simply not bother. Weigh this against accuracy honestly — it may be the deciding constraint. - **Physical infrastructure implied.** Markers are cheap, passive, and need no batteries; beacons need maintenance. The project goal is minimal added hardware, and printed markers may satisfy it better than BLE. - **Consequence for the pose contract (#7):** whether pose carries a "time since last fix" or drift-estimate field so the renderer can degrade as confidence decays. Note the interaction with mirrored ships: a wrong initial fix in a mirrored layout is not slightly wrong, it is confidently wrong on the other side of the vessel. Whatever is chosen must fail loudly rather than quietly. --- Part of the wayfinder map #1.
lars added this to the Wayfinder: RN prototyping spec milestone 2026-08-11 15:52:11 +00:00
lars added the wayfinder:grilling label 2026-08-11 15:52:11 +00:00
lars commented 2026-08-11 16:02:01 +00:00
Author
Owner
Copy Link
Copy Source

Largely pre-answered — every absolute-heading option is disqualified, so markers it is

A late research pass on ARKit VIO and RN bindings lands almost squarely on this ticket. Read before working it.

The drift number is a range, not a value

Best primary source (Kim et al., Sensors 22(24):9873, iPhone 12 Pro Max, loop closure explicitly disabled — our exact regime):

sequence length final drift % of distance
long U-corridor, poor textures, 5 pure rotations 145 m 0.79 m 0.54 %
open hallway 84 m 0.14 m 0.17 %
stairs B2→5F 114 m 0.19 m 0.17 %

But an iPhone X measurement over a 19.1 m corridor gave ~1.5 m, ~7.9 %. So plan for 0.5 %–8 %, device and scene dependent. Consequence: an absolute re-fix every ~20–30 m keeps you under 1 m even at the pessimistic end. That is a concrete deployment density for whatever the fix mechanism turns out to be.

Absolute heading: all three candidates are ruled out

  • ARGeoTrackingConfiguration — Apple, verbatim: "Geotracking occurs exclusively outdoors." Needs GPS, internet, and street-level imagery. Dead.
  • ARCore Geospatial — indoors it depends entirely on VPS, which is built from Street View. None inside a ship. Dead.
  • .gravityAndHeading — works, but it is compass-based, Apple publishes no accuracy figure and no runtime quality signal, and indoor ferrous heading error runs 60°+ raw. A bad fix at session start becomes a permanent, invisible yaw offset; 30° over 40 m is ~20 m lateral. Dead as a primary.

So: worldAlignment: .gravity plus a marker-derived heading. Poll CLHeading.headingAccuracy only as a diagnostic (negative means "strong interference from local magnetic fields" — on a ship, expect that constantly). This settles the "which origin mechanism" question in favour of fiducial markers on the evidence, not on preference: markers are passive, need no batteries, cost pennies, give position and heading in one observation, and fit the project's minimal-added-hardware goal better than beacons.

And the marker path is already available in JS

@reactvision/react-viro 2.57.5 (published 2026-07-27, repo active 2026-08-10, 1819★) exposes with zero Swift: onCameraTransformUpdate (position/rotation/forward/up, ≤60Hz), onTrackingUpdated (mapping 1:1 onto ARKit's TrackingState and reasons incl. EXCESSIVE_MOTION / INSUFFICIENT_FEATURES), worldAlignment as a prop, ViroARImageMarker — exactly the fiducial relocalization this ticket needs — and occlusionMode: "depthBased". react-native-arkit is dead (repo gone). WebXR is definitively dead on iOS (caniuse n through 26.5), and 8th Wall's hosted platform retired 2026-02-28.

What Viro cannot do: no ARWorldMap persistence, no relocalization control, no raw frames, no depth to JS. Also legacy-arch views on the interop layer. And note ARKit's ARSession owns the camera exclusively — you get VIO+pose or arbitrary frames, never both, so any neural localizer must live in Swift inside the ARSession, not in JS via ExecuTorch/TFLite.

ARWorldMap is not the re-anchoring answer

Apple documents no size limit, no multi-room or multi-deck support, no lighting or viewpoint tolerance — only that "the device must be returned to a position and orientation near where it was." Failed relocalization stays .relocalizing indefinitely and Apple's advice is to offer a reset. ARCore Cloud Anchors are worse for a ship: "best used for room-scale", internet required, and explicitly "avoid reflective surfaces" plus "sufficient lighting" — polished metal and mood lighting is the whole environment.

What is left to actually decide here

  1. Marker type and placement — printed image targets vs QR, size, how many, and whether they can be visually unobtrusive enough for a passenger ship's interior design team to accept. This is now the crux.
  2. The passenger interaction — does the user knowingly point at a marker, or is it opportunistic (markers near lifts and stair landings, picked up in passing)?
  3. Fallback when no marker has been seen — the app has some pose from VIO the whole time; decide what it does when the last fix is 100 m old.
  4. Mirrored-ship failure detection — still the unsolved part. A marker fix is unambiguous, but between fixes a mirrored layout offers no correction.
## Largely pre-answered — every absolute-heading option is disqualified, so markers it is A late research pass on ARKit VIO and RN bindings lands almost squarely on this ticket. Read before working it. ### The drift number is a range, not a value Best primary source (Kim et al., *Sensors* 22(24):9873, iPhone 12 Pro Max, **loop closure explicitly disabled** — our exact regime): | sequence | length | final drift | % of distance | |---|---|---|---| | long U-corridor, poor textures, 5 pure rotations | 145 m | **0.79 m** | 0.54 % | | open hallway | 84 m | 0.14 m | 0.17 % | | stairs B2→5F | 114 m | 0.19 m | 0.17 % | But an iPhone X measurement over a 19.1 m corridor gave **~1.5 m, ~7.9 %**. So plan for **0.5 %–8 %**, device and scene dependent. **Consequence: an absolute re-fix every ~20–30 m keeps you under 1 m even at the pessimistic end.** That is a concrete deployment density for whatever the fix mechanism turns out to be. ### Absolute heading: all three candidates are ruled out - **`ARGeoTrackingConfiguration`** — Apple, verbatim: *"Geotracking occurs exclusively outdoors."* Needs GPS, internet, and street-level imagery. Dead. - **ARCore Geospatial** — indoors it depends entirely on VPS, which is built from **Street View**. None inside a ship. Dead. - **`.gravityAndHeading`** — works, but it is compass-based, Apple publishes **no accuracy figure and no runtime quality signal**, and indoor ferrous heading error runs **60°+** raw. A bad fix at session start becomes a permanent, invisible yaw offset; 30° over 40 m is ~20 m lateral. Dead as a primary. **So: `worldAlignment: .gravity` plus a marker-derived heading.** Poll `CLHeading.headingAccuracy` only as a diagnostic (negative means *"strong interference from local magnetic fields"* — on a ship, expect that constantly). This settles the "which origin mechanism" question in favour of **fiducial markers** on the evidence, not on preference: markers are passive, need no batteries, cost pennies, give position *and* heading in one observation, and fit the project's minimal-added-hardware goal better than beacons. ### And the marker path is already available in JS **`@reactvision/react-viro` 2.57.5** (published 2026-07-27, repo active 2026-08-10, 1819★) exposes with **zero Swift**: `onCameraTransformUpdate` (position/rotation/forward/up, ≤60Hz), `onTrackingUpdated` (mapping 1:1 onto ARKit's `TrackingState` and reasons incl. `EXCESSIVE_MOTION` / `INSUFFICIENT_FEATURES`), `worldAlignment` as a prop, **`ViroARImageMarker`** — exactly the fiducial relocalization this ticket needs — and `occlusionMode: "depthBased"`. `react-native-arkit` is dead (repo gone). WebXR is definitively dead on iOS (caniuse `n` through 26.5), and 8th Wall's hosted platform retired 2026-02-28. **What Viro cannot do:** no `ARWorldMap` persistence, no relocalization control, no raw frames, no depth to JS. Also legacy-arch views on the interop layer. And note **ARKit's `ARSession` owns the camera exclusively** — you get VIO+pose *or* arbitrary frames, never both, so any neural localizer must live in Swift inside the ARSession, not in JS via ExecuTorch/TFLite. ### ARWorldMap is not the re-anchoring answer Apple documents **no size limit, no multi-room or multi-deck support, no lighting or viewpoint tolerance** — only that *"the device must be returned to a position and orientation near where it was."* Failed relocalization stays `.relocalizing` **indefinitely** and Apple's advice is to offer a reset. ARCore Cloud Anchors are worse for a ship: *"best used for room-scale"*, internet required, and explicitly *"avoid reflective surfaces"* plus *"sufficient lighting"* — polished metal and mood lighting is the whole environment. ### What is left to actually decide here 1. **Marker type and placement** — printed image targets vs QR, size, how many, and whether they can be visually unobtrusive enough for a passenger ship's interior design team to accept. This is now the crux. 2. **The passenger interaction** — does the user knowingly point at a marker, or is it opportunistic (markers near lifts and stair landings, picked up in passing)? 3. **Fallback when no marker has been seen** — the app has *some* pose from VIO the whole time; decide what it does when the last fix is 100 m old. 4. **Mirrored-ship failure detection** — still the unsolved part. A marker fix is unambiguous, but between fixes a mirrored layout offers no correction.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
wayfinder:grilling
HITL ticket: conversation to reach a decision
 wayfinder:map
The wayfinder map — index of the effort
 wayfinder:prototype
HITL ticket: make something cheap to react to
 wayfinder:research
AFK ticket: find a fact a decision waits on
 wayfinder:task
Manual work that unblocks a decision
No labels wayfinder:grilling
Milestone
No items
No Milestone Wayfinder: RN prototyping spec
Projects
Clear projects
No projects
Assignees
Clear assignees
Dennis lars
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Blocks
#10 Decide: bake-off design — comparands, ground truth, decision rule
lars/splat-indoor-navigation
Reference: lars/splat-indoor-navigation#15