The Incremental structure from motion(ISFM) method realizes 3D reconstruction based on unordered images and plays an important role in fields such as fine modeling, 3D recording of realistic scenes and 3D reconstruction of Internet images. When facing with the complexity of the scene structure, the ISFM method has some problems, such as poor robustness and low accuracy, which often leads to unsatisfactory 3D reconstruction results and even failure of 3D reconstruction, which severely limits its development and application. An incremental structure from motion method of robustness enhancement and accuracy improvement is constucted. The main improvements are:① Facing the problem that there are many error points in stereo image feature matching, a parameter adaptive RANSAC method which takes into account the feature response value is proposed, which can effectively remove the mismatch and keep the correct matching point to the maximum extent. ② A strategy of adding the next image to avoid robustness and reconstruction accuracy is proposed, which makes the process of reconstruction more reasonable. ③ The elimination of outer points into the adjustment process is introduced, which significantly improves the robustness and accuracy of the adjustment. Finally, by using UAV low-altitude image data, close-range shooting data and Internet-downloaded image data respectively, the experimental results show that this method can effectively eliminate the mismatch points, optimize the image reconstruction sequence and weaken the effect of error points on the adjustment results.