Load a tsdf from disk (stored as npz). Args: fname: path to archive voxel_types: list of strings specifying which vo

Move tensors to a device

Extract a mesh from the TSDF using marching cubes If TSDF also has atribute_vols, these are extracted as vertex_attributes. The mesh

Open a json info_file and do a bit of preprocessing

Returns a list of info_file paths Args: splits (list of strings): each item is a path to a .json file or a path to a .txt fi

Wraps the network output into a TSDF data structure Args: batch: dict containg network outputs Returns:

Applies a 3x4 linear transformation to the TSDF. Each voxel is moved according to the transformation and a new volume is constructed

3d meshgrid of given size. Args: voxel_dim: tuple of 3 ints (nx,ny,nz) specifying the size of the volume Returns: torch lon

Get a normalization module for 3D tensors Args: norm: (str or callable) out_channels Returns: nn.Module or None: th

Backprojects image features into 3D and accumulates them. This is the first half of the network which is run on every frame. Only pa

Reset the accumulators. self.volume is a voxel volume containg the accumulated features self.valid is a voxel volume contain

1x1x1 convolution

3x3x3 convolution with padding

Computes 2d metrics between two depth maps Args: depth_pred: mxn np.array containing prediction depth_trgt: mxn np.array con

Compute Mesh metrics between prediction and target. Opens the Meshs and runs the metrics Args: file_pred: file path of prediction

Wraps inference1() and inference2() into a single call. Args: batch: a dict from the dataloader Returns: se

Gets a transform to preprocess the input data

Package the TSDF volume into a TSDF data structure Args: label_name: name key to store label in TSDF.attribute_vols

Refines accumulated features and regresses output TSDF. This is the second half of the network. It should be run once after all fram

Accumulate a depth map (and color/label) into the TSDF Args: projection: projection matrix of the camera (intrinsics@extrinsics)

rescales TSDF values to weight voxels near the surface more than close to the truncation distance

Load images and metadata for a single frame. Given an info json we use this to load the images, etc for a single frame Args: frame:

Load TSDFs from paths in info. Args: info: dict with paths to TSDF files (see datasets/README) data: dict to add TSDF data to

for each vertex in verts2 find the nearest vertex in verts1 Args: nx3 np.array's Returns: ([indices], [distances])

Normalizes the RGB images to the input range

Apply a 3x4 linear transform to the world coordinate system. This affects pose as well as TSDFs.

Takes 2d features and fills them along rays in a 3d volume This function implements eqs. 1,2 in https://arxiv.org/pdf/2003.10432.pdf Each pi

Builds 2D feature extractor backbone network from Detectron2.

Makes sure TSDFs are voxel aligned so we can directly compare values

Convert a config node to dictionary

backprojects depth maps to point clouds Args: projection: 3x4 projection matrix depth: hxw depth map Returns: tensor

Evaluates COLMAP inference compared to ground truth Args: info_file: path to info_json file for the scene pathout: path where in

Compute TSDF metrics between prediction and target. Opens the TSDFs, aligns the voxels and runs the metrics Args: file_pred: file p

Use TSDF fusion with GT depth maps to generate GT TSDFs Args: path_meta: path to save the TSDFs (we recommend creating a pa

Get the ids of the frames to load

Returns: dict with TSDFs

Computes the L1 distance between 2 TSDFs (ignoring unobserved voxels) Args: tsdf_pred: TSDF containing prediction tsdf_trgt:

Transfer instance labels from ground truth mesh to TSDF For each voxel find the nearest vertex and transfer the label if it is close enough

Returns a dict mapping scannet category label strings to scannet Ids scene****_**.aggregation.json contains the category labels as strings

Returns a dict mapping scannet Ids to NYU40 Ids Args: path: Path to the original scannet data. This is used to get scannetv

Scannet images are 1296x968 but 1296x972 is 4x3 so we pad vertically 4 pixels to make it 4x3

Generates a json file for a sample scene in our common format This wraps all the data about a scene into a single common format that is reada

Create all derived data need for the Scannet dataset For each scene an info.json file is created containg all meta data required by the data

Generates a json file for a scannet scene in our common format This wraps all the data about a scene into a single common format that is read

Generates txt files for each dataset split Creates a folder 'data' in the current working directory. In that folder we create a split.txt fi

Run Colmap dense reconstruction with ground truth pose. Copies and creates the necessary file structure required by Colmap. Then runs Colmap

Run the netork on a scene and save output Args: info_file: path to info_json file for the scene model: pytorch model that implem

Transfers attributs from from_mesh to to_mesh using nearest neighbors Each vertex in to_mesh gets assigned the attribute of the nearest vert

Returns: dict of meta data and images for a single frame

Load images and TSDF for scene i

Args: voxel_dim: tuple of 3 ints (nx,ny,nz) specifying the size of the output volume random_rotation

Args: voxel_size: metric size of voxels (ex: .04m) origin: origin of the voxel volume (xyz position of voxel (0,0,0))

Args: voxel_dim: tuple of 3 ints (nx,ny,nz) specifying the size of the volume voxel_size: metric size of each voxel (

Args: info_file: path to json file (format described in datasets/README) transform: transform object to preprocess da

Args: info_files: list of info_json files num_frames: number of frames in the sequence to load transform: