OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields

Haim Sawdayee, Amir Vaxman, Amit H. Bermano

#SignalProcessing/INRs #ComputerGraphics/ObjectReconstruction #CVPR/2023

Reconstruction from cross-sections

Parallel planes setting
Any orientation or distribution of planes

Problem Setting

cross-sections

$P = {P_{1}, \dots, P_{k}}$ : 2D planes embedded in $R^{3}$ with arbitrary offsets and orientations
Each plane $P_{i}$ contains an arbitrary set of (softly) non-intersecting oriented contours $C_{i} = {c_{i, 1}, \dots, c_{i, l_{i}}}$ that consistently partition the plane into regions of “inside” and “outside” of an unknown domain $Ω \subset R^{3}$ with boundary $\partial Ω$

How to model it by INR

The target output of our method is an indicator function $Y : R^{3} \to R$ define $Ω$ as :

Y (x) = {\begin{cases} 1 & x \notin Ω \\ 0 & x \in Ω \\ 0.5 & x \in \partial Ω \end{cases}

CSLC # header
15 2  # number of planes, number of labels (should be at least 2 - inside and outside)

1 78 1 0.0 0.0 1.0 -0.86 # plane index (1-indexing, please state planes in order), number of vertices in the plane image (a hole is counted as another component), number of connected components, plane parameters A,B,C,D, such that Ax+By+Cz+D=0

0.10 0.08 0.86 # The vertices in x,y,z coordinates, should be on the plane.
0.09 0.08 0.86
0.08 0.09 0.86
0.07 0.09 0.86
[...] # rest of vertices

78 1 0 1 2 3 4 5 6 7 8 9 10 11 [...]  # image component: starts with the number of vertices, then label of the component (in case of a hole, h should be added and the index of the component contains the hole), then the indices of vertices that form a contour of the inside label, ordered CCW.
[...] # rest of components

gen1 by foxwarze on Sketchfab

gen2 by foxwarze on Sketchfab

gen3 by foxwarze on Sketchfab

gen4 by foxwarze on Sketchfab

gen5 by foxwarze on Sketchfab

Overview

Loss:

L (x, θ) = \sum_{i = 0}^{N - 1} BCE (Y_{i} (x)) + λ max (0, ‖ \nabla f_{N - 1} (x) ‖ - α)

OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields

Reconstruction from cross-sections

Problem Setting

Overview

Sampling

Results gallery

Qualitative comparisons

Quantitative comparisons

Increasing number of slices

OReX naturally completes regions with missing samples

Comparison of OReX to a Reinforcement Learning work by Ostono