Research: iPhone Pro LiDAR capture to a metric, gravity-aligned splat #5
Notifications
Due Date
No due date set.
Blocks
#8 Capture the office-room splat and verify it loads
lars/splat-indoor-navigation
Reference: lars/splat-indoor-navigation#5
Reference in New Issue
Block a user
Question
Which capture pipeline turns an iPhone Pro (LiDAR) walkthrough of an office room into a splat that is real-world scale and gravity-aligned, and in a format the chosen viewer loads?
Answer specifically:
Part of the wayfinder map #1.
Answer
Research date: 2026-08-11. Every claim below is tagged [VERIFIED] (read directly in a vendor doc / repo / source file), [VENDOR CLAIM] (stated by the vendor but unverifiable without a test), or [COMMUNITY] (forum/review/third-party). Nothing here has been tested on hardware yet - the taped-point test described at the end is what converts the vendor claims into verified ones.
1. Which tools actually produce gaussian splats, and which preserve metric scale
polyformdocs: poses follow "the ARKit gravity aligned convention", depth PNGs are 16-bit millimetres [VERIFIED]ns-process-data polycam->transforms.jsonorientation_method="up"by default (estimates up from camera vectors) [VERIFIED]splat.plywithf_dc_*/f_rest_*SH, opacity,scale_0-2,rot_0-3[VERIFIED fromexporter.py]crates/brush-dataset/src/{config.rs,formats/nerfstudio.rs,scene.rs}shows no pose scaling, no centering, no auto-orient, and it ignoresapplied_transform[VERIFIED by source].plywith SH (export_{iter}.ply). No SPZ/SOG writerGaussianSplatComponent(WWDC26)BufferResource->GaussianSplatResource1a. The nerfstudio metric-scale trap (most important single finding)
NerfstudioDataParserConfigdefaults [VERIFIED from source]:orientation_method="up",center_method="poses",auto_scale_poses=True,load_3D_points=False. The offending line is literally:i.e. the scene is normalised to fit +/-1. Worse:
ns-export gaussian-splatdoes not apply the inverse dataparser transform (unlikeExportPointCloud), so there is no post-hoc rescue - you must train unnormalised:ns-process-data record3dapplies no scaling and no unit conversion [VERIFIED fromrecord3d_utils.py] - ARKit metres pass through intotransforms.jsonunchanged. Nerfstudio's own docs state scale is "preserved from the LiDAR depth data" for record3d and "real-world scale maintained" for polycam raw [VERIFIED].2. Gravity alignment - and how to fix it if absent
ARKit's world frame with
ARWorldAlignment.gravityis already gravity-aligned: +Y opposite gravity, metres, X/Z arbitrary in the horizontal plane (.gravityAndHeadingadditionally pins -Z to north). So gravity alignment is free at capture time - the risk is a pipeline stage throwing it away, not the sensor.Consequences:
orientation_method="up". It estimates up from average camera vectors, which is strictly worse than the gravity vector ARKit already gave you. Usenoneand apply a fixed basis swap yourself.splat-transform -r <x,y,z>(Euler degrees),-t <x,y,z>,-s <factor>[VERIFIED]. SuperSplat's browser editor can do it interactively. Postshot has "Align Up Vector"; Brush 0.3.0 has a grid+arrow-key leveller.Note that rotating a splat is not just moving positions: the per-splat rotation quaternions and the spherical-harmonic coefficients must be rotated too. SPZ handles this explicitly via Wigner D-matrices on save/load [VERIFIED]. Use a real splat tool, never a naive point-cloud transform.
3. Export formats
.ply(splat PLY,f_dc_*/f_rest_*/opacity/scale_0-2/rot_0-3).splat.ksplat.ply", matching the viewer's internal layout for "the fastest loading times" [VERIFIED]util/create-ksplat.js.ply" [VERIFIED]. Now v4 (ZSTD, independent attribute streams); v1-3 still readableFormat choice does not need to block the viewer ticket. PlayCanvas's
splat-transform(open source, npm CLI, runs offline) reads PLY, compressed PLY, SOG, Streamed SOG, SPZ,.splat,.ksplat, LCC/LCC2 and writes PLY, compressed PLY, SOG, Streamed SOG, SPZ, GLB, CSV, HTML viewer, LOD, voxel, WebP [VERIFIED]. So: keep a metric, gravity-correct.plymaster and transcode to whatever the viewer ticket picks. The only lock-in risk is.ksplat, which points at one specific viewer.4. Indoor capture technique for one office room
Postshot's Capturing Guidelines is the best primary source anyone publishes [all VERIFIED, quoted]:
Failure modes to expect in an office room - note that no vendor documents these, so this is engineering judgement plus the physics of 3DGS, not a cited claim:
Practical recipe: ~4-8 minutes of walking, 3 loops, ~300-800 sampled frames.
5. Relating splat space to the physical room (the similarity transform)
The splat's own frame is whatever the trainer emitted. To bridge to a measured site frame you need a similarity transform
p_site = s*R*p_splat + t- 7 DOF (3 rotation, 3 translation, 1 uniform scale) - which is the classic Umeyama / Procrustes least-squares problem (Umeyama 1991, IEEE TPAMI).What that requires concretely:
TransformationEstimationPointToPoint(with_scaling=True), documented as producingT = [[cR, t], [0, 1]]wherewith_scaling=False"force[s] scaling to be 1" [VERIFIED]. So the same API gives you both the similarity and the rigid fit.Does metric capture remove the scale term? Effectively yes, but keep fitting it anyway. If the capture is genuinely metric,
sshould come out at 1.000 +/- a small tolerance, and you should then locks = 1and fit onlyR, t(6 DOF, rigid) for production - fitting a free scale against noisy hand-clicked points will absorb real error intosand flatter your residuals. But on the first capture, fitsfree and read it as a measurement:s = 1.03means the pipeline has a 3% scale error, and that number is the single most valuable output of the whole test. A pipeline wheresdrifts between captures is disqualified for AR navigation, because pose error grows with distance from the origin.Also: if gravity alignment holds,
Rreduces to a single yaw angle plus a fixed basis swap. Fit the fullRfirst and check that its roll/pitch are near zero - that is your empirical test of the gravity claim, and it is a much sharper test than eyeballing the render.The taped-point test this ticket needs: tape 4-6 points on the floor, measure the pairwise distances between them with a tape measure before capturing (ground truth independent of any transform), capture, then (a) measure the same distances in splat space and compare - that alone validates metric scale with zero transform fitting - and (b) fit the transform, stand at each taped point, and compare render vs reality. Report RMS residual in millimetres.
Verdict
Capture tool: Scaniverse Classic (iOS). Export format:
.plyas the metric master, transcoded withsplat-transformto whatever the viewer ticket picks.Rationale: it is the only app in the set with a vendor statement about metric scale, and it is simultaneously free, on-device/offline, watermark-free, and exports both PLY and SPZ. Polycam and KIRI both fail metric scale by construction (their splat modes don't use LiDAR at all) and Polycam additionally paywalls splat export at $300/yr. Luma is abandoned.
.plyas the master is the choice that keeps the viewer ticket genuinely open -splat-transformconverts it to.splat,.ksplat, SOG, Streamed SOG, SPZ or compressed PLY offline, so no viewer decision is foreclosed. Avoid.ksplatas a master: it points at one specific viewer.Fallback if Scaniverse's metric scale fails the taped-point test: Polycam Space-mode raw LiDAR export (Developer Mode) ->
ns-process-data polycam-> Brush. This is the rigorous path: Polycam'spolyformdocs explicitly specify the ARKit gravity-aligned convention with millimetre depth, and Brush is verified by source inspection not to touch your poses (no rescale, no re-centre, no auto-orient) - unlike nerfstudio, whose defaults silently destroy metric scale. Brush is Apache 2.0 and runs on Apple Silicon without CUDA, so it needs no new hardware. Postshot would give better quality but costs EUR 17/mo and is Windows+NVIDIA only.Capture recipe for the office room:
.ply(and SPZ as a second copy for comparison).with_scaling=True-> reads(expect ~1.000) and checkR's roll/pitch ~ 0 -> confirms gravity alignment; (c) locks=1, refit rigid, stand at each taped point and compare render vs reality; (d) report RMS residual in mm.Open / unresolved
splat-transform -r..ksplat, and only Scaniverse exports SPZ. All web-delivery formats therefore require an offlinesplat-transformstep. Fine, but it means the pipeline is never a single app.splat-transform's SH rotation correctness was not verified by source. SPZ documents Wigner D-matrix SH rotation explicitly; splat-transform's-rwas only read from its README. If gravity correction is applied via-r, verify view-dependent appearance doesn't degrade.s ~ 1.000proves little; the requirement is thatsbe reproducible across repeated captures of the same room, since pose error grows with distance from the origin. Plan >= 3 captures.Primary sources (all fetched 2026-08-11): Polycam capture modes - Polycam export file types - Polycam splats to Unity - Polycam pricing - PolyCam/polyform - Scaniverse App Store - Niantic Spatial pricing - Scaniverse FAQ - nianticlabs/spz - KIRI export formats - KIRI local processing FAQ - KIRI pricing - Luma 3D Capture App Store - Luma changelog - Postshot Capturing Guidelines - Postshot Importing Images - Postshot Rdnc Field node - nerfstudio custom data - nerfstudio_dataparser.py - camera_utils.py - exporter.py - record3d_utils.py - gsplat LICENSE - ArthurBrussee/brush - PlayCanvas splat formats - playcanvas/splat-transform - SuperSplat - mkkellogg/GaussianSplats3D - Open3D TransformationEstimationPointToPoint - RealityKit GaussianSplatComponent - Stray Scanner format - jc211/NeRFCapture