3D reconstruction with own data

[slz929]

I want to construct my own sequnce data captured by cameras mounted on a vehicle.
I use the script in pipelines/reconstruction.py

steps:

1. I prepared 2500 images with size of 1280 * 720, extract NetVLAD vectors, get 20 paris from retrieval for every image.

global_descriptors = extract_features.main(retrieval_conf, dataset, outputs)
pairs_from_retrieval.main(
    global_descriptors, sfm_pairs, 20,
    query_list= images, db_list= images, db_descriptors= global_descriptors)

2. I extract the Superpoint and match by NN.

'superpoint_zgc': {
        'output': 'feats-superpoint-n4096-r1024',
        'model': {
            'name': 'superpoint',
            'nms_radius': 3,
            'max_keypoints': 4096,
        },
        'preprocessing': {
            'grayscale': True,
            'resize_max': 1280,
        },
    },

'NN-superpoint': {
        'output': 'matches-NN-mutual-dist.7',
        'model': {
            'name': 'nearest_neighbor',
            'do_mutual_check': True,
            'distance_threshold': 0.7,
  },

3. Then I construct with single camera.

reconstruction.main(sfm_dir, dataset, sfm_pairs, feature_path, match_path, 
        colmap_path= 'colmap/build/src/exe/colmap', single_camera= True)

But I get the output, it seems that very few images were used or stored in bin file. And I get 7 sub-models with small size.

Elapsed time: 904.367 [minutes]
[12/29/2021 08:55:22 INFO] Reconstructed 7 models.
[12/29/2021 08:55:22 INFO] Largest model is #0 with 1 images.
[12/29/2021 08:55:22 INFO] Statistics:
{'mean_reproj_error': 1.312922,
 'mean_track_length': 6.724747,
 'num_input_images': 2569,
 'num_observations': 34619,
 'num_observations_per_image': 2884.916667,
 'num_reg_images': 12,
 'num_sparse_points': 5148}

Compared to the output of Aachen dataset.

[11/24/2021 16:03:11 INFO] Statistics:
{'mean_reproj_error': 1.322519,
 'mean_track_length': 5.709355,
 'num_observations': 5203032,
 'num_observations_per_image': 1202.179298,
 'num_reg_images': 4328,
 'num_sparse_points': 911317}

And my result with my own data cause the wrong localization and query image retrieval.
Looking forward to your reply! Thanks sincerely.

[sarlinpe]

If you have sequence data, you should change the way the image pairs are selected and use both:

• sequential pairs between each image and its K previous ones. We don't have a script for that, you're welcome to PR one with a new script pairs_from_sequence.
• loop closure pairs with image retrieval, triggered every N frame (depending on the framerate + motion magnitude).

Overall, you want to make sure that there are more constraints from the sequence and few sparse loop closures.

[ashfaqueck]

I was trying to reconstruct using camera model as PINHOLE. But it fails to get most of the points in such a route. Any idea how to perfectly get it out as PINHOLE camera output? So that I can take that for dense and mesh to OpenMVS or alternates.

I am doing this in the notebook
import cv2
from pathlib import Path

def infer_camera_params_from_image_dir(img_dir: Path, fx=None, fy=None):
sample_img = next(img_dir.glob("*"))
img = cv2.imread(str(sample_img))
h, w = img.shape[:2]
cx, cy = w / 2, h / 2
fx = fx if fx else 1.2 * w # a safe default guess
fy = fy if fy else fx
return f"{fx},{fy},{cx},{cy}"

import pycolmap
from pathlib import Path
import cv2

sfm_dir = outputs / "sfm_superpoint+lightglue"

Setup image reader options

imgs=Path("datasets/lion/images/")
params = infer_camera_params_from_image_dir(imgs)

image_options = pycolmap.ImageReaderOptions()
image_options.camera_model = "PINHOLE"
#image_options.camera_params =params

#model = reconstruction.main(sfm_dir, images, sfm_pairs, feature_path, match_path,image_options=image_options)
model = reconstruction.main(sfm_dir, images, sfm_pairs, feature_path, match_path)

@sarlinpe
@mihaidusmanu