-
-* Put agents in 3D world
+* Features
+** Put agents in 3D world
- Agent has body that resembles human
- TODO: find good design for humanoid
-
-* World is built around intent
+** World is built around intent
- you start with the world and you name intent for it
- you can inherit copy of existing apps into new space or star with empty space
+* 3D rendering
+
+Roadmap and measurement notes from the 2026-09-13 perf study
+(Fallout 4 environment, downtown Boston spawn — camera
+"7146.48,-7989.42,-5598.31,-2.49,-0.83,0.00", 3840x2091). Resume
+work here.
+
+** Where we are — measured baseline
+
+Single-threaded frame at the Boston spawn (ProfProbe harness):
+transform 120 ms (21%) + painter sort 40 ms (7%) + paint 411 ms
+(72%) = 571 ms/frame. Paint decomposes (linear fit across
+resolutions): per-triangle setup 51%, per-span 25%, pixel fill 24%.
+SETUP dominates, not fill.
+
+Triangle funnel: 1.27M loaded -> 894k reach paint. Of those:
+- 46% backfacing (painted, then covered)
+- 26% die at the vertical clamp AFTER full per-tri setup
+- 20% tinier than 2x2 px at 4K (44% at 960x540!)
+- overdraw 3.28x (every pixel written 3.28 times)
+- frustum culling: ZERO composites in the FO4 scene — the engine's
+ AABB frustum culling never engages (shapes added flat to root)
+
+Live app was at 0.7 fps (bugreport-20260913-030135) vs ~1.75 fps
+single-thread probe: heap histogram shows 3.6 GB texture int[] +
+1.4 GB G1 filler arrays in 12.4/16 GB heap -> GC pressure is a
+separate live-session problem.
+
+Nanite viability floor (NanoTriBench, 24-core tiny): pixel-sized
+triangles rasterize at 26 Mtri/s single-thread, 126 Mtri/s on 22
+threads (8 ns/tri); texture fetch free at that size. Java CAN feed
+a micro-triangle pipeline. Modelled Nanite frame at 4K: ~110 ms
+(~9 fps) without occlusion culling, ~60 ms (~16 fps) with HZB —
+and that number is FLAT with world size.
+
+NOTE (2026-09-13 evening): fo4 default farRadius is now 2, not 4
+(bugreport-20260913-165044 — the terrain-only FAR doughnut rings
+2-4 read as "empty near, detailed far"; BTO object LOD now starts
+at ring 3). Baselines above were probed with farRadius=4 (81
+streamed cells at the Boston spawn); reruns now see 25 cells and a
+different tri mix — re-baseline ProfProbe before comparing.
+
+** Road to Nanite — evolutionary stages
+
+Each stage ships independently and keeps its value; nothing is
+thrown away later. The seam of the whole plan is Stage 1 (touches
+every rasterizer); after that everything is additive.
+
+*** TODO Stage 0a: per-cell composite frustum culling
+Wrap each streamed cell's shapes in a CompositeShape with an AABB
+so the engine's existing frustum test engages. Est -20-30% frame.
+Already the top item in sixth-environment-fo4/AGENTS.org.
+
+*** TODO Stage 0b: per-material backface culling
+MEASURED -11% paint (-De3d.backface=true exists in the engine,
+global, default off). Production version must key off the NIF
+two-sided shader flag (cutout trees/fences are double-sided), so
+wire a flag from NifFile shader properties into TexturedTriangle.
+
+*** TODO Stage 1: z-buffer (w-buffer) + two-pass rendering
+Opaque pass: depth test+write. Transparent pass (glass, GUI, text):
+back-to-front painter, test but NEVER write depth (avoids the
+"insert between merged layers" problem — see Design decisions).
+Cutout alpha (fences, foliage) joins the opaque pass with texel
+test. Store 1/z (w-buffer): our perspective path already
+interpolates 1/z per pixel, depth is a byproduct we throw away;
+hyperbolic precision suits the 130k-unit horizon.
+Seam: every span writer in the engine must learn depth (same class
+of sweep as the stereo X-clipping change — grep all
+drawHorizontalLine variants). Needs demo app + golden re-render
+with VISION review (z intentionally changes pixels where painter
+was wrong — road mottling, depthBias hack, dithered _lod statics,
+patchy distant buildings all die in this commit).
+
+*** TODO Stage 2: opaque near-to-far (early reject)
+Flip the comparator for the opaque pass. Est -12% frame from
+killing the 3.28x overdraw. Hours once Stage 1 exists.
+
+*** TODO Stage 3: HZB occlusion culling
+Downsample depth into a coarse mip pyramid per frame; test
+cell/object AABBs against it before streaming/drawing; cull whole
+cells — skips transform+sort+setup+fill. Nanite's two-phase trick
+applies: reuse LAST frame's pyramid, draw last frame's visible set
+first, then test the rest. View-dependent: moderate at the aerial
+spawn, large at street level.
+
+*** TODO Stage 4: tiny-tri stamp fast path
+At paint entry: screen bbox < 2x2 px -> stamp one average-colored
+quad (nearest mip texel at centroid) instead of full setup. 20% of
+triangles at 4K; est -7-10% paint. This is the SEED of the
+micro-rasterizer grown inside the existing one. Object-level
+variant: drop whole placements under ~2 px bbox at transform time
+(with hysteresis against pop-in).
+
+*** TODO Stage 5: cluster DAG remeshing (Nanite proper)
+The pipeline can't tell where triangles came from — a cluster DAG
+is just another shape source, so build it mesh-by-mesh, no flag
+day. Offline tool: cluster ~128-tri groups, simplification DAG
+with boundary locking; emit clusters; render through the same
+scene graph. Prototype on one kit (Red Rocket shell), measure,
+then batch. Naive vertex-clustering decimation = days; good
+quadric-error version = the multi-week meshoptimizer-class nut,
+but isolated offline. FO4's own _lod.nif levels are a quality
+reference. End state: ~15 fps at 4K FLAT with world size; all
+hand/baked LOD (_lod.nif, BTR, BTO, near ring) replaced uniformly;
+distant patchiness gone.
+
+*** Optional fork: visibility buffer
+Write triangle-ID + depth, resolve materials once per visible
+pixel in a second pass. Worth it when shading dominates; borderline
+at 4K already. Cheap to try once Stage 1 exists (ID write rides
+with the depth write).
+
+** Other optimization proposals (not on the Nanite road)
+
+- Heap/GC diet (live-session 0.7 fps cause): 3.6 GB texture int[]
+ (mip chains) + G1 humongous fragmentation. Options: bigger heap,
+ drop deepest mips for distant-only textures, 16-bit pixel
+ storage. Own work item, independent of rendering changes.
+- Temporal coherence: cache per-cell visibility/culling decisions
+ while the camera stays in a cell; re-evaluate on cell change or
+ large rotation.
+- Coverage mask (painter-era occlusion, NO z-buffer): rasterize
+ nearest ~10-20% opaque tris into a coarse coverage bitmap
+ front-first, skip far objects fully covered. Exact under painter
+ rules (only nearer+opaque sets bits). SUPERSEDED by Stage 3 if
+ the z-buffer lands — kept here in case we stay painter-only.
+- Terrain horizon culling: heightfield horizon-angle test; cells
+ behind hills culled nearly free. Big in hilly country, minor in
+ flat Boston.
+- BSP exact ordering: engine already has BSP-ranked shapes (skip
+ Z-sort). Correctness + sort time, not occlusion. Pairs with
+ everything above.
+- Resolution honesty: ~900k tris at 4K on CPU won't hit 60 fps by
+ any of these. Internal-render-at-lower-res + upscale is the
+ blunt instrument (540p -> 4K saves ~40% paint, looks soft).
+
+** Exotic techniques survey (verdicts from the discussion)
+
+All complexity-independent techniques share one idea: query a
+prebuilt spatial index per pixel instead of drawing per triangle;
+cost = pixels x log(world).
+
+- BVH ray tracing: THE complexity-independent endpoint. Subsumes
+ occlusion/frustum/backface/overdraw/painter-sort by never
+ touching invisible geometry. Static world = ideal (build once).
+ Java CPU: ~10-50M rays/s -> a few fps at 1080p, ~1 at 4K, flat
+ as world grows; shadows/GI nearly free per extra ray. Hybrid
+ variant: rasterize near ring, ray-trace past it. Aligned with
+ user's taste (chose real RT over lightmap upscaling for GI).
+- Surfel/point-cloud (QSplat/Potree): the 4px-blob idea
+ industrialized. Octree of representative points, screen-error
+ LOD walk, hard POINT BUDGET per frame -> world-size independent.
+ Most painter-friendly; cheapest exotic to try.
+- Voxel octree raymarching (SVO, Teardown; Euclideon was the
+ marketing version): geometry forgotten after voxelization.
+ Niche for us: voxelize the coarse BTR/BTO horizon blocks.
+- 3D Gaussian splatting: differentiable painter's algorithm;
+ beautiful but a CAPTURE representation (from photos), wrong
+ input format for game meshes.
+- SDF ray marching: procedural content only; converting FO4's
+ detailed meshes to distance fields is lossy. Bad fit.
+
+** Design decisions (resolved discussions, don't re-litigate)
+
+- Z-buffer speedup mechanics: z alone gives CORRECTNESS, not
+ speed; speed comes from flipping opaque order near-to-far so
+ behind-fragments become cheap depth-test rejects. Per-pixel
+ reject ~1/10 of a shaded write.
+- Transparent polygons under z: two passes (opaque test+write;
+ transparent back-to-front test-only). One depth slot per pixel
+ cannot represent a stack of see-through layers.
+- REJECTED: single-pass alpha-accumulation (framebuffer color +
+ per-pixel alpha + depth, compose on arrival). Math is valid
+ (front-to-back "over" is associative, A' = A + a(1-A)) but
+ requires strict per-pixel near-to-far arrival; per-triangle
+ centroid sorting violates it routinely (existing road mottling
+ proves the error rate), and once layers merge to alpha=1 a later
+ fragment landing between them gets weight 0 -> OPAQUE geometry
+ can vanish. Two-pass fails cosmetically (wrong tint between
+ glasses, rare); one-pass fails structurally (missing geometry,
+ common). Real descendants if ever needed: depth peeling,
+ weighted blended OIT.
+- User preference noted: evolutionary stages with each step
+ shippable, over big-bang rewrites.
+
+** Harnesses and flags (for resuming work)
+
+- ProfProbe.java — per-stage frame profiler at the bugreport pose
+ (world load like Fo4Shot, times transform/sort/paint, prints
+ counters). NanoTriBench.java — micro-triangle floor bench.
+ Both in /tmp/hermes-verify-redrocket/ — VOLATILE (tmpfs);
+ recreate or move into the repo before reboot.
+- Engine instrumentation (UNCOMMITTED in sixth-3d): -De3d.prof=true
+ enables TexturedTriangle counters (tris/backface/offscreenY/
+ tiny/spans/pixels, profReset()); -De3d.backface=true sets the
+ backface-culling default. Probe classpath: prepend
+ sixth-3d/target/classes so the instrumented engine wins over the
+ installed jar.
+- FO4 goldens: Fo4Shot <out.png> spawn|<pose> [w h]; spawn pose
+ lives in ~/.config/sixth/config.properties (fo4.spawnAt).
+- Project docs: sixth-environment-fo4/AGENTS.org (streaming/LOD
+ details), skills n0-sixth-environment-fo4, n0-sixth-3d-engine.
projects / sixth.git / commitdiff