Show HN: ViSOR – two-plane neural scenes you can fly through (code+demo)

I’ve been experimenting with ways to turn NeRF‑style view synthesis into something you can actually move around in, at interactive frame‑rates, without a data‑center GPU.

The result is ViSOR (View-interpolated Sparse Occlusion Refraction), an open‑source prototype that compresses a whole scene into two learned planes:

Occlusion sheet — stores diffuse colour, soft alpha, specular highlights.

Refraction sheet — fires three slightly bent sub‑rays through a learned micro‑prism to add parallax and a bit of chromatic sparkle.

Because everything lives in those planes, rendering is just two texture‑lookups and a tiny MLP each, so:

≈15 fps at 512 × 512 on an RTX 4090 (current unoptimised Python path).

1–2 GB VRAM while flying around.

Core network weights are only 3.4 MB; a baked spherical‑harmonics lighting grid adds 200 MB if you want nicer global illumination.

The code is in PyTorch with tiny‑cudann hash‑grid latents, Apache 2.0 licence. A single 309 MB checkpoint (network + SH grid) is included for a plug‑and‑play demo — python visor_view.py demo.pt, then WASD to fly.

What works well so far

Glassy or highly specular objects — the micro‑prism trick gives decent highlights.

Memory scales with “number of planes,” not scene size, so multiple ViSORs can tile a larger world.

What’s missing

Training is still slow (single‑scale loss, no fused CUDA kernels).

Only tested on synthetic Blender images; real photos will need exposure/tone‑mapping work.