MFE‐MVSNet: Multi‐scale feature enhancement multi‐view stereo with bi‐directional connections

Abstract

Abstract Recent advancements in deep learning have significantly improved performance in the multi‐view stereo (MVS) domain, yet achieving a balance between reconstruction efficiency and quality remains challenging for learning‐based MVS methods. To address this, we introduce MFE‐MVSNet, designed for more effective and precise depth estimation. Our model incorporates a pyramid feature extraction network, featuring efficient multi‐scale attention and multi‐scale feature enhancement modules. These components capture pixel‐level pairwise relationships and semantic features with long‐range contextual information, enhancing feature representation. Additionally, we propose a lightweight 3D UNet regularization network based on depthwise separable convolutions to reduce computational costs. This network employs bi‐directional skip connections, establishing a fluid relationship between encoders and decoders and enabling cyclic reuse of building blocks without adding learnable parameters. By integrating these methods, MFE‐MVSNet effectively balances reconstruction quality and efficiency. Extensive qualitative and quantitative experiments on the DTU dataset validate our model's competitiveness, demonstrating approximately 33% and 12% relative improvements in overall score compared to MVSNet and CasMVSNet, respectively. Compared to other MVS networks, our approach more effectively balances reconstruction quality with efficiency.

Keywords

• computer vision
• image reconstruction
• learning (artificial intelligence)