MV2GF

Multi-View Pedestrian Detection (MVPD) aims to detect pedestrians in the form of a bird's eye view map from multi-view images. Recent MVPD methods adopt a unified framework that projects 2D image features into a 3D world space and aggregates them into a single feature. Although they are effective, they struggle to generalize to unseen camera configurations during training due to two main issues. First, they are difficult to capture accurate visual geometry across views in unseen camera configurations. Second, they make detection models highly dependent on distortion patterns during training arising from their image feature projection. To address these, we leverage a visual geometric foundation model and propose MV2GF. This foundation model has exhibited strong generalization in capturing visual geometry across views and predicting accurate 3D attributes in diverse camera configurations. MV2GF fuses task-specific features with general-purpose geometric features extracted by the foundation model to effectively capture the visual geometry even in unseen camera configurations. Furthermore, MV2GF projects each pixel in the image features to an appropriate 3D location using 3D pointmaps predicted by the foundation model, preventing the detection model from depending on distortion patterns during training. Our experiments demonstrate the effectiveness of leveraging a visual geometric foundation model for MVPD and that MV2GF generalizes better than existing methods.

MV2GF leverages multi-view features and 3D pointmaps from a visual geometric foundation model (e.g., Depth Anything 3) through two newly introduced components: Task-specific and Geometric information Fusion (TGF) and Feature Pointmap Aggregation (FPA).

• Task-specific and Geometric information Fusio (TGF) generates informative high-resolution image features for individual views by fusing task-specific features extracted by a trainable ResNet and general-purpose geometric features extracted by a frozen geometric foundation model.
• Feature Pointmap Aggregation (FPA) generates a BEV feature while avoiding shadow-like distortions by projecting each pixel in TGF image features to an appropriate location based on 3D pointmaps from the geometric foundation model.

When camera configurations during testing are different from those during training, MV2GF achieves significantly better detection performance than previous state-of-the-art methods. All models are trained on GMVD-D training split.

Even when camera configurations during testing are identical to those during training, MV2GF achieves detection performance comparable to that of previous state-of-the-art methods. All models are trained on the training split of the same dataset used for testing.

MV2GF reduces both false positives and false negatives compared to MSMVD on GMVD-D.

MV2GF exhibits fewer distortions around pedestrians than MVFP and precisely represents pedestrian locations.