lars/splat-indoor-navigation
1
0
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects Releases Wiki Activity

Write the Session 1 spec (pose to splat rendering) #14

New Issue
Open
opened 2026-08-11 15:41:40 +00:00 by lars · 2 comments
lars commented 2026-08-11 15:41:40 +00:00
Owner
Copy Link
Copy Source

Question

Write the Session 1 spec as markdown in the repo, executable cold by two devs.

Must contain: hypothesis, the chosen viewer and splat format with versions pinned, the RN <-> WebView pose channel design, the coordinate-alignment method (and how alignment residual is measured on a held-out point), the rendered-vs-real validation protocol, the fake-navigation walkthrough, gates with named fallbacks, deliverables, and how the presentation decision from the prototype ticket shapes the UI.

Cites the pose contract rather than restating it. Closing this ticket means the spec is written and handed off — not that the session has run.

Part of the wayfinder map #1.

## Question Write the Session 1 spec as markdown in the repo, executable cold by two devs. Must contain: hypothesis, the chosen viewer and splat format with versions pinned, the RN <-> WebView pose channel design, the coordinate-alignment method (and how alignment residual is measured on a held-out point), the rendered-vs-real validation protocol, the fake-navigation walkthrough, gates with named fallbacks, deliverables, and how the presentation decision from the prototype ticket shapes the UI. Cites the pose contract rather than restating it. Closing this ticket means the spec is written and handed off — not that the session has run. --- Part of the wayfinder map #1.
lars added this to the Wayfinder: RN prototyping spec milestone 2026-08-11 15:41:40 +00:00
lars added the wayfinder:task label 2026-08-11 15:41:40 +00:00
lars added a new dependency 2026-08-11 15:42:14 +00:00
#3 Research: WebGL2 splat viewer performance inside iOS WKWebView
lars added a new dependency 2026-08-11 15:42:14 +00:00
#6 Research: native / RN gaussian splat rendering without a WebView
lars added a new dependency 2026-08-11 15:42:14 +00:00
#8 Capture the office-room splat and verify it loads
lars added a new dependency 2026-08-11 15:42:14 +00:00
#11 Decide: success gates and the measurement protocol
lars added a new dependency 2026-08-11 15:42:14 +00:00
#12 Prototype: does a rendered-splat route view read as navigation?
lars added a new dependency 2026-08-11 15:42:14 +00:00
#7 Decide: the pose contract — shape, units, site frame
lars commented 2026-08-11 15:54:11 +00:00
Author
Owner
Copy Link
Copy Source

Inputs from #3 for this spec.

  • Viewer + format: Spark (@sparkjsdev/spark 2.1.0, MIT, WebGL2-only), assets as SOG or SPZ. .RAD LoD streaming exists for anything beyond room scale.
  • Footgun to write into the spec: @mkkellogg/GaussianSplats3D defaults to SharedArrayBuffer, which requires COOP/COEP headers you cannot set on a bundled WebView page. Either disable that path or don't pick that viewer.
  • WebGPU is unusable in WKWebView below iOS 26 (confirmed first-party), so pin WebGL2. WebGPU stays the later upgrade path.
  • Keep the pose interpolation loop inside the WebView and push only sparse pose updates across the bridge — don't drive the camera per-frame from RN.
  • Budget: ~1.5M gaussians, ≤300–400MB in the WebContent process, watching the load-time transient.
  • Both perf gates are unvalidated (see #11) — the spec should have the devs measure the bridge latency in hour one rather than assume it passes.
**Inputs from #3 for this spec.** - **Viewer + format:** Spark (`@sparkjsdev/spark` 2.1.0, MIT, WebGL2-only), assets as SOG or SPZ. `.RAD` LoD streaming exists for anything beyond room scale. - **Footgun to write into the spec:** `@mkkellogg/GaussianSplats3D` defaults to `SharedArrayBuffer`, which requires COOP/COEP headers you **cannot set on a bundled WebView page**. Either disable that path or don't pick that viewer. - **WebGPU is unusable** in WKWebView below iOS 26 (confirmed first-party), so pin WebGL2. WebGPU stays the later upgrade path. - **Keep the pose interpolation loop inside the WebView** and push only sparse pose updates across the bridge — don't drive the camera per-frame from RN. - **Budget:** ~1.5M gaussians, ≤300–400MB in the WebContent process, watching the load-time transient. - Both perf gates are unvalidated (see #11) — the spec should have the devs measure the bridge latency in hour one rather than assume it passes.
lars added a new dependency 2026-08-11 15:55:43 +00:00
#16 Measure the RN → WebView pose-bridge latency
lars commented 2026-08-11 16:00:28 +00:00
Author
Owner
Copy Link
Copy Source

Architecture inputs, corrected and expanded (see #3 addendum 3, #6 addendum 2).

  • The bridge is not the bottleneck on iOS (~0.2–1.5ms one-way). Budget is dominated by ARKit pose age (10–25ms), two forced 60Hz quantizations (rAF + out-of-process CA commit), and scanout. Total ≈45–100ms best case.
  • Spec in-WebView forward prediction from day one. WKWebView exposes no late-warp hook, so extrapolating ~25ms ahead from pose + velocity is the only latency compensation. Not a stretch goal.
  • rAF is capped at 60Hz — don't design for 120.
  • Pose goes native→web only. Use postMessage for rare control events; its Fabric path is run-loop-gated, uncoalesced, and unbounded.
  • Consider a native module owning both ARSession and WKWebView and injecting from session(_:didUpdate:) with completionHandler: nil — Mozilla's WebXR Viewer's design. #16 will say whether the delta justifies it.
  • Inject a resident window.__setPose once; pass 7 float literals, never JSON.stringify per frame; never route pose through React state.
  • Version pins: react-native-webview npm latest is 14.0.1 (16.0.0 is next, legacy arch removed); requires RN 0.76+ and iOS 15.1+.
  • Assets: .ply / compressed .ply / .splat / SOG — and note Spark cannot run outside a WebView (Vite-only worker import, mandatory Rust WASM, no Hermes WebAssembly), which is itself part of why the WebView is the right host.

Live alternative worth one paragraph in the spec's "why not native" section: Babylon React Native 2.0.5 genuinely renders splats natively on iOS and Android via a C++ sortSplats. It loses on Expo support (disclaimed), RN version lag (0.81.6 vs 0.87), a legacy view under new-arch interop, and Hermes format gaps (SPZ v4 rejected, SOG crashes). Record it as measured-and-rejected, not unconsidered.

**Architecture inputs, corrected and expanded (see #3 addendum 3, #6 addendum 2).** - **The bridge is not the bottleneck on iOS** (~0.2–1.5ms one-way). Budget is dominated by **ARKit pose age (10–25ms)**, two forced 60Hz quantizations (rAF + out-of-process CA commit), and scanout. Total ≈45–100ms best case. - **Spec in-WebView forward prediction from day one.** WKWebView exposes no late-warp hook, so extrapolating ~25ms ahead from pose + velocity is the *only* latency compensation. Not a stretch goal. - **rAF is capped at 60Hz** — don't design for 120. - **Pose goes native→web only.** Use `postMessage` for rare control events; its Fabric path is run-loop-gated, uncoalesced, and unbounded. - **Consider a native module owning both `ARSession` and `WKWebView`** and injecting from `session(_:didUpdate:)` with `completionHandler: nil` — Mozilla's WebXR Viewer's design. #16 will say whether the delta justifies it. - Inject a resident `window.__setPose` once; pass 7 float literals, never `JSON.stringify` per frame; never route pose through React state. - **Version pins:** `react-native-webview` npm `latest` is 14.0.1 (16.0.0 is `next`, legacy arch removed); requires RN 0.76+ and iOS 15.1+. - **Assets: `.ply` / compressed `.ply` / `.splat` / SOG** — and note Spark cannot run outside a WebView (Vite-only worker import, mandatory Rust WASM, no Hermes WebAssembly), which is itself part of why the WebView is the right host. **Live alternative worth one paragraph in the spec's "why not native" section:** Babylon React Native 2.0.5 genuinely renders splats natively on iOS *and* Android via a C++ `sortSplats`. It loses on Expo support (disclaimed), RN version lag (0.81.6 vs 0.87), a legacy view under new-arch interop, and Hermes format gaps (SPZ v4 rejected, SOG crashes). Record it as measured-and-rejected, not unconsidered.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
wayfinder:grilling
HITL ticket: conversation to reach a decision
 wayfinder:map
The wayfinder map — index of the effort
 wayfinder:prototype
HITL ticket: make something cheap to react to
 wayfinder:research
AFK ticket: find a fact a decision waits on
 wayfinder:task
Manual work that unblocks a decision
No labels wayfinder:task
Milestone
No items
No Milestone Wayfinder: RN prototyping spec
Projects
Clear projects
No projects
Assignees
Clear assignees
Dennis lars
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Depends on
#3 Research: WebGL2 splat viewer performance inside iOS WKWebView
lars/splat-indoor-navigation
#6 Research: native / RN gaussian splat rendering without a WebView
lars/splat-indoor-navigation
#7 Decide: the pose contract — shape, units, site frame
lars/splat-indoor-navigation
#8 Capture the office-room splat and verify it loads
lars/splat-indoor-navigation
#11 Decide: success gates and the measurement protocol
lars/splat-indoor-navigation
#12 Prototype: does a rendered-splat route view read as navigation?
lars/splat-indoor-navigation
#16 Measure the RN → WebView pose-bridge latency
lars/splat-indoor-navigation
#17 Decide: does orientation bypass the bridge via DeviceOrientationEvent in the WebView?
lars/splat-indoor-navigation
Reference: lars/splat-indoor-navigation#14