Research: native / RN gaussian splat rendering without a WebView #6

New Issue

2026-08-11T15:41:38Z

lars commented

2026-08-11 15:41:38 +00:00

Question

Is there a 2026 option for rendering a gaussian splat in React Native without a WebView, and would it change the architecture?

Answer specifically:

Existing RN packages or native SDKs (Metal/Vulkan splat renderers, react-native-filament, Babylon RN, Unity/Unreal as a view, Three.js on react-native-wgpu) — maturity, licence, whether they actually read splat formats.
Effort to bridge a native splat renderer versus the WebView route.
Whether any of them accept an externally supplied camera pose cleanly.
Verdict: does this change the recommendation for Session 1, or is it confirmed as a later optimization only?

Part of the wayfinder map #1.

## Question Is there a 2026 option for rendering a gaussian splat in React Native **without** a WebView, and would it change the architecture? Answer specifically: - Existing RN packages or native SDKs (Metal/Vulkan splat renderers, `react-native-filament`, Babylon RN, Unity/Unreal as a view, Three.js on `react-native-wgpu`) — maturity, licence, whether they actually read splat formats. - Effort to bridge a native splat renderer versus the WebView route. - Whether any of them accept an externally supplied camera pose cleanly. - Verdict: does this change the recommendation for Session 1, or is it confirmed as a later optimization only? --- Part of the wayfinder map #1.

lars added this to the Wayfinder: RN prototyping spec milestone 2026-08-11 15:41:38 +00:00

lars added the wayfinder:research label 2026-08-11 15:41:38 +00:00

lars added a new dependency 2026-08-11 15:42:14 +00:00

#14 Write the Session 1 spec (pose to splat rendering)

lars commented

2026-08-11 15:55:01 +00:00

Answer

Short version: yes, a WebView-free native path exists on iOS and it is genuinely good — but only on iOS, and only if you write the React Native bridge yourself. There is no cross-platform, off-the-shelf RN gaussian-splat component in 2026. The WebView route stands for Session 1.

All dates below are as of 2026-08-11. Every version/date claim is from npm registry metadata, the GitHub API, or the project's own docs/source.

1. The "dedicated RN gaussian splat package" question — settled

react-native-gsplat is the only npm package that claims to be this. It is empty.

Single version 0.0.1, published 2026-04-19, MIT, author joncodeofficial, no README, no repository field.
I downloaded the tarball: unpackedSize = 301 bytes, and the archive contains exactly one file, package/package.json. main points at index.js, which does not exist. No dependencies, no native code.
Source: https://registry.npmjs.org/react-native-gsplat and its dist tarball.

A GitHub repository search for react-native gaussian splat returns no library — only one unrelated hackathon app (Simon-xP/CxC-2026, 0 stars). Conclusion: no dedicated RN splat package exists. Treat any search hit for react-native-gsplat as a name-squat.

2. Package-by-package

react-native-filament (Margelo) — healthy, but cannot render splats

npm 1.11.0, published 2026-05-27. MIT. GitHub margelo/react-native-filament: 1,387 stars, last push 2026-05-27, not archived, 49 open issues. Actively maintained.
Formats: .glb only. Its own Getting Started guide: "Currently only loading .glb files is supported. You can convert any other 3D models (gltf, obj, FBX, etc) to .glb..." No splat, no point cloud.
Upstream Filament itself does not support splats. In google/filament discussion #8033 (opened 2024-08-06), maintainer pixelflinger answered (2024-08-11): "It can be implemented using our material system. It's not supported natively." Later comments in the same thread — the most recent dated 2025-10-29 — identify the real blocker: "Filament doesn't support compute shaders so there's no (easy) way to depth-sort the splats based on camera orientation on the GPU." Two years on, still unresolved. Open3D works around exactly this by running a separate GPU compute pipeline alongside Filament and compositing via a shared depth texture — i.e. the splat renderer is not Filament, it just sits next to it.
Camera pose: good. <Camera position={...} target={...} near far /> — declarative props you can drive per frame from JS, plus a worklet render callback (react-native-worklets-core). The gesture path (CameraManipulator.grabBegin/grabUpdate/grabEnd) is optional, not mandatory. Limitations stated: one camera per scene, perspective only.
Expo: has a config plugin; also requires react-native-worklets-core — note a likely conflict with our current react-native-worklets@0.5.1 (the Reanimated-4 package, a different library). Needs a dev build, not Expo Go.
Verdict: you would write the entire splat sort + render pipeline from scratch, on an engine that lacks compute shaders. Worst of both worlds. Rule out.

Babylon React Native — maintained, but structurally blocked

npm @babylonjs/react-native 2.0.5, published 2026-05-19. MIT. GitHub BabylonJS/BabylonReactNative: 433 stars, last push 2026-06-02, 32 open issues. Alive.
Babylon.js core does read splats — SPLATFileLoader handles .splat, .ply, and .spz; there is a documented GaussianSplattingMaterial / GaussianSplattingMesh. On paper this is the most attractive option.
But the RN binding is the problem. Babylon React Native runs Babylon Native, and per the Babylon forum thread "Babylon React Native Compute Shaders" (#37568) the RN package is still WebGL-based, so compute shaders are unavailable. Worse: Babylon's splat implementation depth-sorts in a Web Worker (per Babylon's own Gaussian Splatting docs: "sorting is handled through a web worker for asynchronous sorting"). React Native has no Web Worker API. So the sort has nowhere to run except the JS thread.
Camera pose: Babylon's FreeCamera/TargetCamera accept position + rotationQuaternion set imperatively — this part is clean.
Expo: bare RN / prebuild required; heavy native build (Babylon Native ships large prebuilt binaries).
Verdict: no evidence anyone has rendered a splat through Babylon React Native. Requires validation before it can be believed. Not a Session-1 bet.

react-native-webgpu (William Candillon, Dawn-backed) — the most interesting native option, but you still write the renderer

Two npm names, same repo: react-native-webgpu 0.8.2 (2026-08-06) is the live one; react-native-wgpu 0.5.17 (2026-07-08) is now explicitly "Shim that re-exports react-native-webgpu under its previous package name." Both MIT.
GitHub wcandillon/react-native-webgpu: 1,204 stars, created 2024-06-17, last push 2026-08-06, not archived. Very actively developed — but still pre-1.0 at 0.8.x.
Site: "Built for iOS, Android, macOS, visionOS, and Web", powered by Dawn. Requirements: React Native >= 0.81.0 (we are on 0.81.5 ✔), New Architecture only — "does not work on the legacy architecture"; optional react-native-worklets >= 0.7.2 for off-thread rendering (we have 0.5.1, would need a bump).
Expo: ships a config plugin ("react-native-webgpu" in plugins), works with Continuous Native Generation, requires npx expo prebuild --clean → development build, not Expo Go. Their template "works with React Three Fiber on iOS, Android, and web."
This is the one path that gives you real compute shaders on both platforms, which is exactly what a modern splat renderer wants (GPU radix sort by depth per frame, instanced indirect draw). Three.js has been WebGPU-production-ready via three/webgpu since r171.
But: it does not read splat formats. There is no maintained three.js-WebGPU splat renderer you can drop in. @sparkjsdev/spark is explicitly WebGL2 (see §4), so it does not run on this. You would port or write a WGSL splat pipeline yourself.
Verdict: the right long-term native answer if we ever leave the WebView. Not a Session-1 answer.

expo-gl / expo-three — expo-gl fine, expo-three dead, and the combination can't host a splat viewer

expo-gl is healthy and in our SDK: 16.0.10 (SDK 54 line) published 2026-01-27; SDK 57 line at 57.0.2 (2026-07-15). MIT. Included in Expo Go.
WebGL2 is genuinely there. Package description: "Provides GLView that acts as OpenGL ES render target and gives GL context object implementing WebGL 2.0 specification." Docs: "It resembles a WebGL2RenderingContext in the WebGL 2 spec", with the caveat "Some older Android devices may not support WebGL2 features... use gl instanceof WebGL2RenderingContext." So instanced arrays / float textures / integer textures are available in principle.
However, 20 WebGL2 methods are documented as unimplemented, including the entire sync family (fenceSync, clientWaitSync, waitSync, getSyncParameter, isSync, deleteSync), getBufferSubData, renderbufferStorageMultisample, getInternalformatParameter, getActiveUniformBlockParameter, and all compressedTexImage*. Also: "The pixels argument of texImage2D() must be null, an ArrayBuffer with pixel data, or an object of the form { localUri }" — no ImageBitmap, no DOM sources.
The killer is threading, not GL. Splat viewers depth-sort on a Web Worker. RN has none, and expo-gl's own docs say of its worklet escape hatch: "Third-party libraries like Pixi.js or Three.js won't work inside the worklet, you can only use functions that have 'worklet' added at the start." So a 1–5M-splat CPU sort would run on the RN JS thread, alongside our positioning logic and UI. That is not viable.
expo-three last published 2024-07-28 (8.0.0). Two years stale — treat as abandoned.
Verdict: rule out. WebGL2 is present but the runtime around it (no workers, no DOM, no sync objects) is not a browser, and every existing splat library assumes a browser.

react-native-gpu — does not exist

No such package on npm. Anything referring to it means react-native-webgpu / react-native-wgpu. Nothing to evaluate.

Unity / Unreal as a hosted view — possible, wildly disproportionate

@azesmway/react-native-unity 1.1.1, published 2026-08-07, MIT — active, and currently the only maintained option. (The old react-native-unity-view last shipped 1.3.3 on 2019-01-07 — dead for 7½ years.)
The splat side would be aras-p/UnityGaussianSplatting: MIT, 3,379 stars, but last push 2025-10-17 (~10 months stale) and self-described as "Toy Gaussian Splatting visualization in Unity."
Cost: Unity-as-a-Library into an Expo app means bare RN, a Unity licence, +tens of MB of app size, two build toolchains, and a Unity↔RN message bridge you'd be sending pose over anyway.
Verdict: rule out for a prototype.

3. MetalSplatter and bridging a native Metal/Vulkan renderer

MetalSplatter is real, current, well-shaped, and it is the strongest native option — on iOS only. I read its source directly.

github.com/scier/MetalSplatter — "Render Gaussian Splats using Metal on Apple platforms (iOS/iPhone/iPad, macOS, and visionOS)". MIT. 708 stars, created 2023-12-31, last push 2026-05-28, repo updated 2026-08-08. Not archived. Actively maintained.
Not visionOS-only. Package.swift declares iOS 18, macOS 15, visionOS 2, Swift 6 language mode. Note the iOS 18 floor.
It is a reusable library, not just a sample app. SPM products: MetalSplatter (core renderer), SplatIO, PLYIO, SampleBoxRenderer, plus a SplatConverter CLI.
Formats, from SplatIO/Sources/SplatFileFormat.swift: .ply, .splat, .spz. (There is a SPZSceneReader/SPZSceneWriter and an SPZ compression dependency.)
It already has its own async sorter — MetalSplatter/Sources/SplatSorter.swift, with onSortStart / onSortComplete(TimeInterval) / afterNextSort hooks and a sortTimeout. You do not write the sort.
Externally supplied camera pose: this is the best answer of anything surveyed — it is the only way to drive it. From SplatRenderer.swift:

public struct ViewportDescriptor {
    public var viewport: MTLViewport
    public var projectionMatrix: simd_float4x4
    public var viewMatrix: simd_float4x4
    public var screenSize: SIMD2<Int>
}

public func render(viewports: [ViewportDescriptor],
                   colorTexture: MTLTexture,
                   colorStoreAction: MTLStoreAction,
                   depthTexture: MTLTexture?,
                   rasterizationRateMap: MTLRasterizationRateMap?,
                   renderTargetArrayLength: Int,
                   accessTimeout: TimeInterval = 0.1,
                   sortTimeout: TimeInterval = 0.1,
                   to commandBuffer: MTLCommandBuffer) throws -> Bool

You hand it a view matrix, a projection matrix, and a target texture. There are no built-in gesture/orbit controls to fight. For "we drive the camera from a positioning system every frame," this is exactly the shape we want. It also renders into a caller-supplied MTLTexture, so hosting it in an MTKView inside a native RN view is straightforward.

Caveats in its own README: "The documentation is entirely missing", and it must be built in Release — Debug is "more than an order of magnitude slower."
No RN bridge exists. A GitHub search for a MetalSplatter React Native bridge returns nothing.

Android has no equivalent. This is the decisive asymmetry. What exists is desktop-oriented:

nvpro-samples/vk_gaussian_splatting — Apache-2.0, 481 stars, last push 2026-06-04. NVIDIA desktop Vulkan sample, not an Android library.
jaesung-cs/vkgs — MIT, 159 stars, last push 2026-07-17. Vulkan viewer + Python binding; desktop.
MouseChannel/MCGS — Vulkan compute-shader splat renderer claiming to run anywhere, but 37 stars, last push 2025-02-05, no licence file. Unusable.
No Android/Kotlin GLES or Vulkan splat library with a documented embedding API turned up at all.

Cross-platform escape hatches, so you don't write it twice:

ArthurBrussee/brush — Rust + wgpu, Apache-2.0, 4,907 stars, last push 2026-08-09. Genuinely cross-platform and genuinely current. But it is a reconstruction app/trainer, not an embeddable render library, and embedding Rust+wgpu into an RN native module is its own multi-week project.
react-native-webgpu (Dawn) is the cleaner shared-code answer: one WGSL pipeline for iOS + Android. But you write the renderer (§2).
Filament is not an escape hatch — no compute shaders (§2).
Apple ships no first-party gaussian-splat renderer as of 2026 (nothing in RealityKit/ARKit/USDZ); MetalSplatter exists precisely because there is no Apple API for it.

Effort estimate for the native bridge (my estimate, not sourced — treat as an order of magnitude):

Work	Estimate
Expo Module scaffold + native view	0.5 d
Getting an SPM-only package (MetalSplatter) into an Expo/CocoaPods module — vendoring or podspec wrapper	1–2 d
`MTKView` host, render loop, `CAMetalLayer` lifecycle	2–3 d
Pose prop (view+projection matrix) settable per frame from JS; ideally via a worklet/shadow-node path so it doesn't cross the async bridge 60×/s	1–2 d
Splat asset loading (bundle / filesystem / remote), progress + error surfaces	1–2 d
Perf + memory tuning on a real ship-sized scene, Release-only build config, undocumented-API archaeology	2–4 d
iOS subtotal	~8–13 engineer-days (1.5–2.5 weeks)
Android from scratch (GLES3/Vulkan splat renderer + depth sort + loaders, no library to reuse)	+3–6 weeks
or WebGPU route instead: write one WGSL splat pipeline on `react-native-webgpu` for both platforms	~2–4 weeks, on a 0.8.x dependency

Versus the WebView route: ~2–4 days to a working, pose-driven splat view on both platforms, with zero native code. That is a 5–15× difference for Session 1, and the native route buys nothing we need yet.

4. The WebView route, re-checked (so the comparison is fair)

react-native-webview — MIT, very active: latest = 14.0.1 (2026-06-20), with 15.0.0 (2026-07-05) and 16.0.0 (2026-07-11) on the next tag. 347 versions. It is bundled in Expo Go, so Session 1 needs no dev build at all.
WebGL2 in the platform webviews is a solved problem: WKWebView has had WebGL2 on by default since iOS 15 (2021); Android System WebView is Chromium.
@sparkjsdev/spark is the standout viewer choice. 2.1.0 published 2026-05-18, MIT, peer dep three >= 0.180.0, only runtime dep fflate. Its own README: "Portable: Works across almost all devices, targeting 98%+ WebGL2 support", "Renders fast even on low-powered mobile devices", "Render multiple splat objects together with correct sorting", and format support for .PLY (including compressed PLY), .SPZ, .SPLAT, .KSPLAT, .SOG. Being WebGL2 (not WebGPU) is a feature here — it works in today's webviews.
Alternatives, dated: playcanvas 2.21.3 (2026-07-29, MIT, active, has gsplat + SOG); @mkkellogg/gaussian-splats-3d 0.4.7 (2025-01-25 — 19 months stale); gsplat 1.2.9 (2025-07-12 — ~13 months stale); @zappar/three-gaussian-splat 0.1.0 (2023-12-19 — abandoned). three itself is 0.185.1 (2026-07-01).
The one real WebView risk is the pose channel, and it is manageable. postMessage is implemented on top of injectJavaScript on all platforms, and the RN-WebView community measures a round trip (web → RN → state → back) at "tens of milliseconds at minimum". But we do not need a round trip: we need a one-way, fire-and-forget injectJavaScript of ~7 floats at 30–60 Hz, with the WebView's own requestAnimationFrame doing the interpolation. Mitigations to design in from day one: send pose at 20–30 Hz and interpolate/extrapolate in-page rather than pushing every frame; keep the payload a fixed-width numeric string, not JSON; and read the gyro/orientation directly inside the WebView via DeviceOrientation/DeviceMotion so only the low-rate position fix has to cross the bridge. This should be measured in Session 1 — it is the single number that would justify revisiting the native route.

5. Expo dev client vs bare RN — summary

Option	Expo Go	Expo dev build + config plugin	Bare RN required
react-native-webview + Spark	Yes (bundled in Expo Go)	yes	no
expo-gl	yes	yes	no
react-native-webgpu	no	yes (own config plugin, CNG, `expo prebuild`; New Arch only, RN ≥ 0.81)	no
react-native-filament	no	yes (config plugin)	no
Babylon React Native	no	unlikely / undocumented	effectively yes
MetalSplatter bridge	no	yes — an Expo Module with a config plugin is the right shape	no
Unity as a view	no	no	yes

Nothing here forces bare React Native except Unity and (in practice) Babylon RN. A MetalSplatter bridge can absolutely live as an Expo Module in a dev client. So "we'd have to abandon Expo" is not a valid argument against the native route — the valid argument is Android, and cost.

Verdict

The WebView route stands for Session 1. Do not change the plan. Ship react-native-webview + a WebGL2 splat viewer in a local HTML bundle, with @sparkjsdev/spark (MIT, 2026-05-18, WebGL2, reads .ply/.spz/.splat/.ksplat/.sog, explicitly mobile-tuned) as the first choice and PlayCanvas as the fallback. It works in Expo Go today with zero native code, on both platforms, in ~2–4 days.

Three reasons the native option loses right now, despite being real:

It is iOS-only. MetalSplatter is excellent and current, and its render(viewports:) API is a near-perfect fit for externally-driven camera pose — but Android has no counterpart to bridge. You would be building one platform now and a from-scratch Vulkan/GLES renderer later, or rewriting both on react-native-webgpu.
Nothing off-the-shelf reads splats. Filament (glb only, no compute shaders), expo-gl (no workers, no DOM), react-native-webgpu (no splat loader) all require writing the sort+render pipeline. Babylon RN is the only one whose engine can read splats, and its RN binding is WebGL-only with no Web Worker for the sort — unproven, likely broken.
Effort is 5–15× the WebView route for a prototype whose open questions are about positioning accuracy and route comprehension, not about splat frame rate.

Confirmed as a later optimization, not a Session-1 change. The native path to keep warm is react-native-webgpu + a WGSL splat pipeline (one renderer, both platforms, real compute shaders, Expo config plugin, RN ≥ 0.81 ✔) — not MetalSplatter, precisely because MetalSplatter can never cover Android.

The trigger that would change this: if Session 1 measurement shows the RN→WebView pose channel cannot sustain smooth 30 Hz camera updates, or WebView splat rendering thermally throttles on target hardware within a typical walk. Measure both in Session 1.

Open / unresolved

Has anyone actually rendered a gaussian splat through Babylon React Native? Babylon.js core has SPLATFileLoader + GaussianSplattingMesh, but the RN binding is WebGL-only and the sort needs a Web Worker RN doesn't have. No working report found either way. A 1-day spike would close this — and if it works, it is the cheapest native option by far.
Measured pose-channel throughput/latency for injectJavaScript at 30–60 Hz on target devices. I found only a qualitative "tens of milliseconds" figure for a full round trip; the one-way cost is unmeasured. This is the load-bearing unknown for the whole plan.
Splat frame rate and thermals inside a WebView on target hardware. A Babylon forum thread reports splats performing notably worse on iOS than Android; unquantified, and not tested for our scene sizes.
Whether react-native-worklets-core (needed by react-native-filament) conflicts with our react-native-worklets@0.5.1. Moot if Filament is ruled out, but it affects any future worklet-based renderer.
react-native-webgpu wants react-native-worklets >= 0.7.2 for off-thread rendering; we are on 0.5.1. Unknown whether the bump is disruptive alongside Reanimated 4.
Commercial native SDKs (Luma AI, Polycam, Niantic/Scaniverse, 8th Wall) were not surveyed — the web-search budget ran out mid-ticket. If any ships a documented on-device iOS+Android splat rendering SDK with acceptable licensing, that is the one thing that could beat both routes. Worth one follow-up hour.
Scene-size ceiling in a WebView — how many splats before iOS WKWebView memory limits bite. Spark claims mobile-friendliness but publishes no budget.
Whether MetalSplatter's iOS 18 minimum is acceptable for the eventual passenger-device fleet.

_{Sources: npm registry metadata for each package; GitHub API repo metadata; scier/MetalSplatter Package.swift, Sources/MetalSplatter/SplatRenderer.swift, Sources/SplatIO/SplatFileFormat.swift (main branch, fetched 2026-08-11); google/filament discussion #8033; Babylon.js Gaussian Splatting docs, SPLATFileLoader typedoc, and forum thread #37568; expo-gl SDK 57 docs + npm package description; react-native-filament Getting Started and camera guides; react-native-webgpu docs (installation, expo); @sparkjsdev/spark README v2.1.0. No repo files were modified in producing this, per instruction.}

## Answer **Short version: yes, a WebView-free native path exists on iOS and it is genuinely good — but only on iOS, and only if you write the React Native bridge yourself. There is no cross-platform, off-the-shelf RN gaussian-splat component in 2026. The WebView route stands for Session 1.** All dates below are as of **2026-08-11**. Every version/date claim is from npm registry metadata, the GitHub API, or the project's own docs/source. --- ### 1. The "dedicated RN gaussian splat package" question — settled `react-native-gsplat` is the only npm package that claims to be this. It is **empty**. - Single version `0.0.1`, published **2026-04-19**, MIT, author `joncodeofficial`, no README, no `repository` field. - I downloaded the tarball: `unpackedSize` = **301 bytes**, and the archive contains **exactly one file, `package/package.json`**. `main` points at `index.js`, which does not exist. No dependencies, no native code. - Source: `https://registry.npmjs.org/react-native-gsplat` and its dist tarball. A GitHub repository search for `react-native gaussian splat` returns no library — only one unrelated hackathon app (`Simon-xP/CxC-2026`, 0 stars). **Conclusion: no dedicated RN splat package exists. Treat any search hit for `react-native-gsplat` as a name-squat.** --- ### 2. Package-by-package #### react-native-filament (Margelo) — **healthy, but cannot render splats** - npm `1.11.0`, published **2026-05-27**. MIT. GitHub `margelo/react-native-filament`: 1,387 stars, last push **2026-05-27**, not archived, 49 open issues. Actively maintained. - **Formats: `.glb` only.** Its own Getting Started guide: *"Currently only loading `.glb` files is supported. You can convert any other 3D models (gltf, obj, FBX, etc) to .glb..."* No splat, no point cloud. - Upstream Filament itself does not support splats. In `google/filament` discussion **#8033** (opened **2024-08-06**), maintainer *pixelflinger* answered (**2024-08-11**): **"It can be implemented using our material system. It's not supported natively."** Later comments in the same thread — the most recent dated **2025-10-29** — identify the real blocker: *"Filament doesn't support compute shaders so there's no (easy) way to depth-sort the splats based on camera orientation on the GPU."* Two years on, still unresolved. Open3D works around exactly this by running a **separate** GPU compute pipeline alongside Filament and compositing via a shared depth texture — i.e. the splat renderer is not Filament, it just sits next to it. - **Camera pose: good.** `<Camera position={...} target={...} near far />` — declarative props you can drive per frame from JS, plus a worklet render callback (`react-native-worklets-core`). The gesture path (`CameraManipulator.grabBegin/grabUpdate/grabEnd`) is optional, not mandatory. Limitations stated: one camera per scene, perspective only. - Expo: has a config plugin; also requires `react-native-worklets-core` — **note a likely conflict with our current `react-native-worklets@0.5.1`** (the Reanimated-4 package, a different library). Needs a dev build, not Expo Go. - **Verdict: you would write the entire splat sort + render pipeline from scratch, on an engine that lacks compute shaders. Worst of both worlds. Rule out.** #### Babylon React Native — **maintained, but structurally blocked** - npm `@babylonjs/react-native` `2.0.5`, published **2026-05-19**. MIT. GitHub `BabylonJS/BabylonReactNative`: 433 stars, last push **2026-06-02**, 32 open issues. Alive. - Babylon.js **core does** read splats — `SPLATFileLoader` handles `.splat`, `.ply`, and `.spz`; there is a documented `GaussianSplattingMaterial` / `GaussianSplattingMesh`. On paper this is the most attractive option. - **But the RN binding is the problem.** Babylon React Native runs Babylon **Native**, and per the Babylon forum thread *"Babylon React Native Compute Shaders"* (#37568) the RN package **is still WebGL-based, so compute shaders are unavailable**. Worse: Babylon's splat implementation depth-sorts **in a Web Worker** (per Babylon's own Gaussian Splatting docs: *"sorting is handled through a web worker for asynchronous sorting"*). **React Native has no Web Worker API.** So the sort has nowhere to run except the JS thread. - Camera pose: Babylon's `FreeCamera`/`TargetCamera` accept `position` + `rotationQuaternion` set imperatively — this part is clean. - Expo: bare RN / prebuild required; heavy native build (Babylon Native ships large prebuilt binaries). - **Verdict: no evidence anyone has rendered a splat through Babylon React Native. Requires validation before it can be believed. Not a Session-1 bet.** #### react-native-webgpu (William Candillon, Dawn-backed) — **the most interesting native option, but you still write the renderer** - Two npm names, same repo: `react-native-webgpu` `0.8.2` (**2026-08-06**) is the live one; `react-native-wgpu` `0.5.17` (**2026-07-08**) is now explicitly *"Shim that re-exports react-native-webgpu under its previous package name."* Both MIT. - GitHub `wcandillon/react-native-webgpu`: 1,204 stars, created **2024-06-17**, last push **2026-08-06**, not archived. Very actively developed — but **still pre-1.0 at 0.8.x**. - Site: *"Built for iOS, Android, macOS, visionOS, and Web"*, powered by **Dawn**. Requirements: **React Native >= 0.81.0** (we are on 0.81.5 ✔), **New Architecture only** — *"does not work on the legacy architecture"*; optional `react-native-worklets >= 0.7.2` for off-thread rendering (we have 0.5.1, would need a bump). - Expo: ships a **config plugin** (`"react-native-webgpu"` in `plugins`), works with Continuous Native Generation, requires `npx expo prebuild --clean` → **development build, not Expo Go**. Their template *"works with React Three Fiber on iOS, Android, and web."* - **This is the one path that gives you real compute shaders on both platforms**, which is exactly what a modern splat renderer wants (GPU radix sort by depth per frame, instanced indirect draw). Three.js has been WebGPU-production-ready via `three/webgpu` since r171. - **But: it does not read splat formats.** There is no maintained three.js-WebGPU splat renderer you can drop in. `@sparkjsdev/spark` is explicitly WebGL2 (see §4), so it does **not** run on this. You would port or write a WGSL splat pipeline yourself. - **Verdict: the right long-term native answer if we ever leave the WebView. Not a Session-1 answer.** #### expo-gl / expo-three — **expo-gl fine, expo-three dead, and the combination can't host a splat viewer** - `expo-gl` is healthy and in our SDK: `16.0.10` (SDK 54 line) published **2026-01-27**; SDK 57 line at `57.0.2` (**2026-07-15**). MIT. Included in Expo Go. - **WebGL2 is genuinely there.** Package description: *"Provides GLView that acts as OpenGL ES render target and gives GL context object implementing WebGL 2.0 specification."* Docs: *"It resembles a WebGL2RenderingContext in the WebGL 2 spec"*, with the caveat *"Some older Android devices may not support WebGL2 features... use `gl instanceof WebGL2RenderingContext`."* So instanced arrays / float textures / integer textures are available in principle. - **However, 20 WebGL2 methods are documented as unimplemented**, including the entire sync family (`fenceSync`, `clientWaitSync`, `waitSync`, `getSyncParameter`, `isSync`, `deleteSync`), `getBufferSubData`, `renderbufferStorageMultisample`, `getInternalformatParameter`, `getActiveUniformBlockParameter`, and all `compressedTexImage*`. Also: *"The `pixels` argument of `texImage2D()` must be `null`, an `ArrayBuffer` with pixel data, or an object of the form `{ localUri }`"* — no `ImageBitmap`, no DOM sources. - **The killer is threading, not GL.** Splat viewers depth-sort on a Web Worker. RN has none, and expo-gl's own docs say of its worklet escape hatch: *"Third-party libraries like Pixi.js or Three.js won't work inside the worklet, you can only use functions that have `'worklet'` added at the start."* So a 1–5M-splat CPU sort would run on the RN JS thread, alongside our positioning logic and UI. That is not viable. - `expo-three` last published **2024-07-28** (`8.0.0`). **Two years stale — treat as abandoned.** - **Verdict: rule out. WebGL2 is present but the runtime around it (no workers, no DOM, no sync objects) is not a browser, and every existing splat library assumes a browser.** #### react-native-gpu — **does not exist** No such package on npm. Anything referring to it means `react-native-webgpu` / `react-native-wgpu`. Nothing to evaluate. #### Unity / Unreal as a hosted view — **possible, wildly disproportionate** - `@azesmway/react-native-unity` `1.1.1`, published **2026-08-07**, MIT — active, and currently the only maintained option. (The old `react-native-unity-view` last shipped `1.3.3` on **2019-01-07** — dead for 7½ years.) - The splat side would be `aras-p/UnityGaussianSplatting`: MIT, 3,379 stars, but **last push 2025-10-17 (~10 months stale)** and self-described as *"Toy Gaussian Splatting visualization in Unity."* - Cost: Unity-as-a-Library into an Expo app means bare RN, a Unity licence, +tens of MB of app size, two build toolchains, and a Unity↔RN message bridge you'd be sending pose over anyway. - **Verdict: rule out for a prototype.** --- ### 3. MetalSplatter and bridging a native Metal/Vulkan renderer **MetalSplatter is real, current, well-shaped, and it is the strongest native option — on iOS only.** I read its source directly. - `github.com/scier/MetalSplatter` — *"Render Gaussian Splats using Metal on Apple platforms (iOS/iPhone/iPad, macOS, and visionOS)"*. **MIT.** 708 stars, created **2023-12-31**, last push **2026-05-28**, repo updated **2026-08-08**. Not archived. **Actively maintained.** - **Not visionOS-only.** `Package.swift` declares **iOS 18, macOS 15, visionOS 2**, Swift 6 language mode. Note the **iOS 18 floor**. - **It is a reusable library, not just a sample app.** SPM products: `MetalSplatter` (core renderer), `SplatIO`, `PLYIO`, `SampleBoxRenderer`, plus a `SplatConverter` CLI. - **Formats, from `SplatIO/Sources/SplatFileFormat.swift`:** `.ply`, `.splat`, `.spz`. (There is a `SPZSceneReader`/`SPZSceneWriter` and an SPZ compression dependency.) - **It already has its own async sorter** — `MetalSplatter/Sources/SplatSorter.swift`, with `onSortStart` / `onSortComplete(TimeInterval)` / `afterNextSort` hooks and a `sortTimeout`. You do not write the sort. - **Externally supplied camera pose: this is the best answer of anything surveyed — it is the *only* way to drive it.** From `SplatRenderer.swift`: ```swift public struct ViewportDescriptor { public var viewport: MTLViewport public var projectionMatrix: simd_float4x4 public var viewMatrix: simd_float4x4 public var screenSize: SIMD2<Int> } public func render(viewports: [ViewportDescriptor], colorTexture: MTLTexture, colorStoreAction: MTLStoreAction, depthTexture: MTLTexture?, rasterizationRateMap: MTLRasterizationRateMap?, renderTargetArrayLength: Int, accessTimeout: TimeInterval = 0.1, sortTimeout: TimeInterval = 0.1, to commandBuffer: MTLCommandBuffer) throws -> Bool ``` You hand it a view matrix, a projection matrix, and a target texture. There are **no** built-in gesture/orbit controls to fight. For "we drive the camera from a positioning system every frame," this is exactly the shape we want. It also renders into a caller-supplied `MTLTexture`, so hosting it in an `MTKView` inside a native RN view is straightforward. - Caveats in its own README: **"The documentation is entirely missing"**, and it must be built in **Release** — Debug is *"more than an order of magnitude slower."* - **No RN bridge exists.** A GitHub search for a MetalSplatter React Native bridge returns nothing. **Android has no equivalent.** This is the decisive asymmetry. What exists is desktop-oriented: - `nvpro-samples/vk_gaussian_splatting` — Apache-2.0, 481 stars, last push **2026-06-04**. NVIDIA desktop Vulkan sample, not an Android library. - `jaesung-cs/vkgs` — MIT, 159 stars, last push **2026-07-17**. Vulkan viewer + Python binding; desktop. - `MouseChannel/MCGS` — Vulkan compute-shader splat renderer claiming to run anywhere, but **37 stars, last push 2025-02-05, no licence file**. Unusable. - No Android/Kotlin GLES or Vulkan splat *library* with a documented embedding API turned up at all. **Cross-platform escape hatches, so you don't write it twice:** - `ArthurBrussee/brush` — Rust + **wgpu**, **Apache-2.0**, 4,907 stars, last push **2026-08-09**. Genuinely cross-platform and genuinely current. But it is a reconstruction app/trainer, not an embeddable render library, and embedding Rust+wgpu into an RN native module is its own multi-week project. - **react-native-webgpu (Dawn)** is the cleaner shared-code answer: one WGSL pipeline for iOS + Android. But you write the renderer (§2). - Filament is *not* an escape hatch — no compute shaders (§2). - Apple ships **no** first-party gaussian-splat renderer as of 2026 (nothing in RealityKit/ARKit/USDZ); MetalSplatter exists precisely because there is no Apple API for it. **Effort estimate for the native bridge** (my estimate, not sourced — treat as an order of magnitude): | Work | Estimate | |---|---| | Expo Module scaffold + native view | 0.5 d | | Getting an **SPM-only** package (MetalSplatter) into an Expo/CocoaPods module — vendoring or podspec wrapper | 1–2 d | | `MTKView` host, render loop, `CAMetalLayer` lifecycle | 2–3 d | | Pose prop (view+projection matrix) settable per frame from JS; ideally via a worklet/shadow-node path so it doesn't cross the async bridge 60×/s | 1–2 d | | Splat asset loading (bundle / filesystem / remote), progress + error surfaces | 1–2 d | | Perf + memory tuning on a real ship-sized scene, Release-only build config, undocumented-API archaeology | 2–4 d | | **iOS subtotal** | **~8–13 engineer-days (1.5–2.5 weeks)** | | **Android from scratch** (GLES3/Vulkan splat renderer + depth sort + loaders, no library to reuse) | **+3–6 weeks** | | *or* WebGPU route instead: write one WGSL splat pipeline on `react-native-webgpu` for both platforms | **~2–4 weeks**, on a 0.8.x dependency | **Versus the WebView route: ~2–4 days to a working, pose-driven splat view on both platforms, with zero native code.** That is a 5–15× difference for Session 1, and the native route buys nothing we need yet. --- ### 4. The WebView route, re-checked (so the comparison is fair) - `react-native-webview` — MIT, very active: `latest` = `14.0.1` (**2026-06-20**), with `15.0.0` (**2026-07-05**) and `16.0.0` (**2026-07-11**) on the `next` tag. 347 versions. **It is bundled in Expo Go**, so Session 1 needs no dev build at all. - WebGL2 in the platform webviews is a solved problem: WKWebView has had WebGL2 on by default since iOS 15 (2021); Android System WebView is Chromium. - **`@sparkjsdev/spark` is the standout viewer choice.** `2.1.0` published **2026-05-18**, **MIT**, peer dep `three >= 0.180.0`, only runtime dep `fflate`. Its own README: *"Portable: Works across almost all devices, targeting 98%+ **WebGL2** support"*, *"Renders fast even on low-powered mobile devices"*, *"Render multiple splat objects together with correct sorting"*, and format support for **`.PLY` (including compressed PLY), `.SPZ`, `.SPLAT`, `.KSPLAT`, `.SOG`**. Being WebGL2 (not WebGPU) is a **feature** here — it works in today's webviews. - Alternatives, dated: `playcanvas` `2.21.3` (**2026-07-29**, MIT, active, has gsplat + SOG); `@mkkellogg/gaussian-splats-3d` `0.4.7` (**2025-01-25** — 19 months stale); `gsplat` `1.2.9` (**2025-07-12** — ~13 months stale); `@zappar/three-gaussian-splat` `0.1.0` (**2023-12-19** — abandoned). `three` itself is `0.185.1` (**2026-07-01**). - **The one real WebView risk is the pose channel, and it is manageable.** `postMessage` is implemented on top of `injectJavaScript` on all platforms, and the RN-WebView community measures a *round trip* (web → RN → state → back) at *"tens of milliseconds at minimum"*. But we do not need a round trip: we need a **one-way, fire-and-forget** `injectJavaScript` of ~7 floats at 30–60 Hz, with the WebView's own `requestAnimationFrame` doing the interpolation. Mitigations to design in from day one: send pose at 20–30 Hz and interpolate/extrapolate in-page rather than pushing every frame; keep the payload a fixed-width numeric string, not JSON; and read the **gyro/orientation directly inside the WebView** via DeviceOrientation/DeviceMotion so only the low-rate *position* fix has to cross the bridge. This should be measured in Session 1 — it is the single number that would justify revisiting the native route. --- ### 5. Expo dev client vs bare RN — summary | Option | Expo Go | Expo dev build + config plugin | Bare RN required | |---|---|---|---| | react-native-webview + Spark | **Yes** (bundled in Expo Go) | yes | no | | expo-gl | yes | yes | no | | react-native-webgpu | no | **yes** (own config plugin, CNG, `expo prebuild`; New Arch only, RN ≥ 0.81) | no | | react-native-filament | no | yes (config plugin) | no | | Babylon React Native | no | unlikely / undocumented | effectively yes | | MetalSplatter bridge | no | **yes** — an Expo Module with a config plugin is the right shape | no | | Unity as a view | no | no | yes | **Nothing here forces bare React Native except Unity and (in practice) Babylon RN.** A MetalSplatter bridge can absolutely live as an Expo Module in a dev client. So "we'd have to abandon Expo" is *not* a valid argument against the native route — the valid argument is Android, and cost. --- ### Verdict **The WebView route stands for Session 1. Do not change the plan.** Ship `react-native-webview` + a WebGL2 splat viewer in a local HTML bundle, with `@sparkjsdev/spark` (MIT, 2026-05-18, WebGL2, reads .ply/.spz/.splat/.ksplat/.sog, explicitly mobile-tuned) as the first choice and PlayCanvas as the fallback. It works in **Expo Go today with zero native code**, on both platforms, in ~2–4 days. Three reasons the native option loses *right now*, despite being real: 1. **It is iOS-only.** MetalSplatter is excellent and current, and its `render(viewports:)` API is a near-perfect fit for externally-driven camera pose — but **Android has no counterpart to bridge.** You would be building one platform now and a from-scratch Vulkan/GLES renderer later, or rewriting both on `react-native-webgpu`. 2. **Nothing off-the-shelf reads splats.** Filament (glb only, no compute shaders), expo-gl (no workers, no DOM), react-native-webgpu (no splat loader) all require writing the sort+render pipeline. Babylon RN is the only one whose engine *can* read splats, and its RN binding is WebGL-only with no Web Worker for the sort — unproven, likely broken. 3. **Effort is 5–15× the WebView route** for a prototype whose open questions are about **positioning accuracy and route comprehension**, not about splat frame rate. **Confirmed as a later optimization, not a Session-1 change.** The native path to keep warm is **react-native-webgpu + a WGSL splat pipeline** (one renderer, both platforms, real compute shaders, Expo config plugin, RN ≥ 0.81 ✔) — *not* MetalSplatter, precisely because MetalSplatter can never cover Android. **The trigger that would change this:** if Session 1 measurement shows the RN→WebView pose channel cannot sustain smooth 30 Hz camera updates, or WebView splat rendering thermally throttles on target hardware within a typical walk. Measure both in Session 1. ### Open / unresolved - **Has anyone actually rendered a gaussian splat through Babylon React Native?** Babylon.js core has `SPLATFileLoader` + `GaussianSplattingMesh`, but the RN binding is WebGL-only and the sort needs a Web Worker RN doesn't have. No working report found either way. A 1-day spike would close this — and if it works, it is the cheapest native option by far. - **Measured pose-channel throughput/latency** for `injectJavaScript` at 30–60 Hz on target devices. I found only a qualitative *"tens of milliseconds"* figure for a full round trip; the one-way cost is unmeasured. **This is the load-bearing unknown for the whole plan.** - **Splat frame rate and thermals inside a WebView** on target hardware. A Babylon forum thread reports splats performing notably worse on iOS than Android; unquantified, and not tested for our scene sizes. - **Whether `react-native-worklets-core` (needed by react-native-filament) conflicts with our `react-native-worklets@0.5.1`.** Moot if Filament is ruled out, but it affects any future worklet-based renderer. - **`react-native-webgpu` wants `react-native-worklets >= 0.7.2`** for off-thread rendering; we are on 0.5.1. Unknown whether the bump is disruptive alongside Reanimated 4. - **Commercial native SDKs (Luma AI, Polycam, Niantic/Scaniverse, 8th Wall) were not surveyed** — the web-search budget ran out mid-ticket. If any ships a documented on-device iOS+Android splat rendering SDK with acceptable licensing, that is the one thing that could beat both routes. Worth one follow-up hour. - **Scene-size ceiling in a WebView** — how many splats before iOS WKWebView memory limits bite. Spark claims mobile-friendliness but publishes no budget. - Whether MetalSplatter's **iOS 18 minimum** is acceptable for the eventual passenger-device fleet. <sub>Sources: npm registry metadata for each package; GitHub API repo metadata; `scier/MetalSplatter` `Package.swift`, `Sources/MetalSplatter/SplatRenderer.swift`, `Sources/SplatIO/SplatFileFormat.swift` (main branch, fetched 2026-08-11); `google/filament` discussion #8033; Babylon.js Gaussian Splatting docs, `SPLATFileLoader` typedoc, and forum thread #37568; expo-gl SDK 57 docs + npm package description; react-native-filament Getting Started and camera guides; react-native-webgpu docs (installation, expo); `@sparkjsdev/spark` README v2.1.0. No repo files were modified in producing this, per instruction.</sub>

lars closed this issue

2026-08-11 15:55:01 +00:00

lars referenced this issue

2026-08-11 15:55:43 +00:00

Measure the RN → WebView pose-bridge latency #16

lars referenced this issue

2026-08-11 15:55:44 +00:00

Wayfinder: React Native ship-navigation prototyping spec #1

lars commented

2026-08-11 15:57:00 +00:00

Addendum — Apple ships first-party splat rendering on iOS 27

A late-reporting sub-agent pass landed after the answer above was posted. It does not change the verdict (WebView + Spark for Session 1), but it changes the long-term native picture materially and adds sourcing the original pass lacked.

Apple now has first-party Gaussian splat rendering in RealityKit — GaussianSplatComponent and GaussianSplatResource, availability iOS 27 / iPadOS 27 / macOS 27 / visionOS 27, all marked beta as of 2026-08-11 (WWDC26 session 279). The API takes raw buffers for position/scale/rotation/opacity/spherical-harmonics and assumes no file format, so a parser is still needed — MetalSplatter's MIT-licensed PLYIO + SplatIO targets fill exactly that gap. So the pragmatic future iOS stack is SplatIO for parsing + RealityKit for rendering, at roughly 5–8 days for an RN wrapper versus 10–14 for the MetalSplatter/MTKView route.

Also relevant:

Khronos published a KHR_gaussian_splatting glTF 2.0 extension release candidate (Feb 2026), backed by NVIDIA, Google, Adobe, Cesium. Splats are becoming a standard asset type, not a bespoke format zoo.
Apple added a Particle Fields USD primitive (with NVIDIA/Adobe/Pixar via AOUSD) so splats live natively in USD.
Android remains empty. A GitHub search for Android gaussian splatting tops out at ~37 stars. Filament closed its 3DGS request unimplemented (#8016); bgfx's (#3171) is still open. Writing a Vulkan splat renderer from scratch is 20–40 engineer-days of specialist graphics work.
react-native-webgpu is the corrected package name (react-native-wgpu is deprecated) — MIT, v0.8.2, 2026-08-06, Dawn-backed, iOS + Android + visionOS. Still the one write-once native path.
Unity is the worst option: UnityGaussianSplatting's own README says mobile support is inconsistent and iOS "do[es] not function properly", the author stopped significant development, and the INRIA training licence is non-commercial.
Design warning for whichever native path is taken: do not set a 4×4 camera-pose prop from JS per frame. Even on Fabric that fights the commit pipeline at 60–120Hz. Use a JSI/TurboModule imperative setter writing to a lock-free slot, or Reanimated worklets on the UI thread.
RN New Architecture is mandatory since 0.82 (2025-10-08) — Fabric is the only option; newArchEnabled=false is ignored.
Expo is sufficient throughout: a config plugin plus the Expo Modules API produces a Fabric-compatible native view. The likely biter is raising deploymentTarget (iOS 18 for MetalSplatter, iOS 27 for RealityKit).

Revisit trigger for the native decision: iOS 27 going GA (~Sept 2026). At that point the iOS native path costs days rather than weeks — but Android still has nothing, so a single WebGPU renderer stays the only write-once answer.

## Addendum — Apple ships first-party splat rendering on iOS 27 A late-reporting sub-agent pass landed after the answer above was posted. It does **not** change the verdict (WebView + Spark for Session 1), but it changes the long-term native picture materially and adds sourcing the original pass lacked. **Apple now has first-party Gaussian splat rendering in RealityKit** — `GaussianSplatComponent` and `GaussianSplatResource`, availability **iOS 27 / iPadOS 27 / macOS 27 / visionOS 27, all marked beta** as of 2026-08-11 (WWDC26 session 279). The API takes **raw buffers** for position/scale/rotation/opacity/spherical-harmonics and assumes **no file format**, so a parser is still needed — MetalSplatter's MIT-licensed `PLYIO` + `SplatIO` targets fill exactly that gap. So the pragmatic future iOS stack is *SplatIO for parsing + RealityKit for rendering*, at roughly **5–8 days** for an RN wrapper versus 10–14 for the MetalSplatter/`MTKView` route. Also relevant: - **Khronos published a `KHR_gaussian_splatting` glTF 2.0 extension release candidate** (Feb 2026), backed by NVIDIA, Google, Adobe, Cesium. Splats are becoming a standard asset type, not a bespoke format zoo. - Apple added a **Particle Fields** USD primitive (with NVIDIA/Adobe/Pixar via AOUSD) so splats live natively in USD. - **Android remains empty.** A GitHub search for Android gaussian splatting tops out at ~37 stars. Filament closed its 3DGS request unimplemented ([#8016](https://github.com/google/filament/issues/8016)); bgfx's ([#3171](https://github.com/bkaradzic/bgfx/issues/3171)) is still open. Writing a Vulkan splat renderer from scratch is **20–40 engineer-days** of specialist graphics work. - **`react-native-webgpu`** is the corrected package name (`react-native-wgpu` is deprecated) — MIT, **v0.8.2, 2026-08-06**, Dawn-backed, iOS + Android + visionOS. Still the one write-once native path. - **Unity is the worst option**: `UnityGaussianSplatting`'s own README says mobile support is inconsistent and iOS "do[es] not function properly", the author stopped significant development, and the INRIA training licence is non-commercial. - **Design warning for whichever native path is taken:** do not set a 4×4 camera-pose prop from JS per frame. Even on Fabric that fights the commit pipeline at 60–120Hz. Use a JSI/TurboModule imperative setter writing to a lock-free slot, or Reanimated worklets on the UI thread. - **RN New Architecture is mandatory since 0.82** (2025-10-08) — Fabric is the only option; `newArchEnabled=false` is ignored. - Expo is sufficient throughout: a config plugin plus the Expo Modules API produces a Fabric-compatible native view. The likely biter is raising `deploymentTarget` (iOS 18 for MetalSplatter, iOS 27 for RealityKit). **Revisit trigger for the native decision:** iOS 27 going GA (~Sept 2026). At that point the iOS native path costs days rather than weeks — but Android still has nothing, so a single WebGPU renderer stays the only write-once answer.

lars commented

2026-08-11 16:00:27 +00:00

Addendum 2 — correction: Babylon React Native IS a turnkey native splat path, cross-platform

The answer above says nothing cross-platform reads splats natively and that Babylon RN's binding is WebGL-only. That is wrong. A later pass verified the opposite in source.

Babylon.js core has a mature splat implementation — splatFileLoader registers .splat, .ply, compressed .ply, .spz, .sog; gaussianSplattingMeshBase.pure.ts is 175KB, plus a streaming/paging stack.
Babylon Native ships a native C++ depth sorter specifically so splats work without Web Workers — sortSplats() in Plugins/NativeOptimizations/Source/NativeOptimizations.cpp, added by commit "Native GS" (#1576) on 2026-01-14. The source branches on IsNative and skips worker instantiation entirely.
It ships in @babylonjs/react-native 2.0.5 (2026-05-19): the package pins BabylonNative commit ce2edf0 dated 2026-04-14, after the Native GS commit, and all three platform CMakeLists.txt link NativeOptimizations.
Babylon's own visual-regression suite passes on Native for compressed-PLY-SH, PLY-SH reload, bake transforms, depth, depth alpha-blended, SPZ SH, viewports, shadows, node material, and loading.
External pose is idiomatic: a FreeCamera with attachControl never called, setting position/rotationQuaternion in onBeforeRenderObservable.

The real costs, which are why the WebView verdict still stands for Session 1:

No Expo support, explicitly disclaimed in the README — bare RN, manual Xcode project reference, JAVA_HOME/NDK/Ninja/CMake. Against react-native-webview + Spark running in a dev client in days.
RN 0.81.6+ while current is 0.87; the view is a legacy requireNativeComponent with no codegenConfig, running through new-arch interop. Tracking issue #595 has been open since 2023-08-03, and Babylon's own Playground test app has newArchEnabled commented out.
Format gotchas in Hermes: no WASM and no DecompressionStream, so SPZ v4/NGSP is rejected by the native fallback loader and SOG crashes on Native. Use .ply / compressed .ply / .splat. .ksplat unsupported (convert with @playcanvas/splat-transform).

Also corrected: Spark cannot run in RN at all — import BundledWorker from "./worker?worker&inline" is Vite-only and spark_rs_bg.wasm is mandatory; Metro can't resolve either and Hermes has no WebAssembly. Spark's own RN request (#381, opened 2026-06-24) is unanswered. So Spark requires the WebView — it is not a candidate for a native path, which strengthens the WebView choice rather than weakening it.

Attribution fix: react-native-webgpu is wcandillon's (William Candillon, 347 commits), not Software Mansion's — SWM contributes and separately owns TypeGPU. Current 0.8.2 (2026-08-06), 28 commits in 30 days, ships its own Expo config plugin and an official template (create-expo-app -e with-webgpu). Still zero splat code — choosing it means writing the loader and GPU radix sort.

Two non-existent packages to not plan around: react-native-gpu (npm 404) and three-gaussian-splat (npm 404). react-native-filament is a dead end for splats — no VertexBuffer API from JS and materials must be offline-compiled .filamat. expo-three is abandoned (last release 2024-07-28).

## Addendum 2 — **correction: Babylon React Native IS a turnkey native splat path, cross-platform** The answer above says nothing cross-platform reads splats natively and that Babylon RN's binding is WebGL-only. **That is wrong.** A later pass verified the opposite in source. - **Babylon.js core has a mature splat implementation** — `splatFileLoader` registers **`.splat`, `.ply`, compressed `.ply`, `.spz`, `.sog`**; `gaussianSplattingMeshBase.pure.ts` is 175KB, plus a streaming/paging stack. - **Babylon Native ships a native C++ depth sorter specifically so splats work without Web Workers** — `sortSplats()` in `Plugins/NativeOptimizations/Source/NativeOptimizations.cpp`, added by commit "Native GS" (#1576) on **2026-01-14**. The source branches on `IsNative` and skips worker instantiation entirely. - **It ships in `@babylonjs/react-native` 2.0.5** (2026-05-19): the package pins BabylonNative commit `ce2edf0` dated 2026-04-14, after the Native GS commit, and all three platform `CMakeLists.txt` link `NativeOptimizations`. - Babylon's own visual-regression suite **passes on Native** for compressed-PLY-SH, PLY-SH reload, bake transforms, depth, depth alpha-blended, **SPZ SH**, viewports, shadows, node material, and loading. - External pose is idiomatic: a `FreeCamera` with `attachControl` never called, setting `position`/`rotationQuaternion` in `onBeforeRenderObservable`. **The real costs, which are why the WebView verdict still stands for Session 1:** - **No Expo support**, explicitly disclaimed in the README — bare RN, manual Xcode project reference, JAVA_HOME/NDK/Ninja/CMake. Against `react-native-webview` + Spark running in a dev client in days. - **RN 0.81.6+ while current is 0.87**; the view is a legacy `requireNativeComponent` with **no `codegenConfig`**, running through new-arch interop. Tracking issue #595 has been open since 2023-08-03, and Babylon's own Playground test app has `newArchEnabled` commented out. - **Format gotchas in Hermes:** no WASM and no `DecompressionStream`, so **SPZ v4/NGSP is rejected** by the native fallback loader and **SOG crashes on Native**. Use `.ply` / compressed `.ply` / `.splat`. `.ksplat` unsupported (convert with `@playcanvas/splat-transform`). **Also corrected:** **Spark cannot run in RN at all** — `import BundledWorker from "./worker?worker&inline"` is Vite-only and `spark_rs_bg.wasm` is mandatory; Metro can't resolve either and Hermes has no WebAssembly. Spark's own RN request ([#381](https://github.com/sparkjsdev/spark/issues/381), opened 2026-06-24) is unanswered. So Spark **requires** the WebView — it is not a candidate for a native path, which strengthens the WebView choice rather than weakening it. **Attribution fix:** `react-native-webgpu` is **wcandillon's** (William Candillon, 347 commits), not Software Mansion's — SWM contributes and separately owns TypeGPU. Current 0.8.2 (2026-08-06), 28 commits in 30 days, ships **its own Expo config plugin** and an official template (`create-expo-app -e with-webgpu`). Still zero splat code — choosing it means writing the loader and GPU radix sort. **Two non-existent packages to not plan around:** `react-native-gpu` (npm 404) and `three-gaussian-splat` (npm 404). `react-native-filament` is a dead end for splats — no VertexBuffer API from JS and materials must be offline-compiled `.filamat`. `expo-three` is abandoned (last release 2024-07-28).

lars referenced this issue

2026-08-11 16:00:28 +00:00

Write the Session 1 spec (pose to splat rendering) #14

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Blocks

#14 Write the Session 1 spec (pose to splat rendering)

lars/splat-indoor-navigation

Reference: lars/splat-indoor-navigation#6