This study aims to develop a quantitative surface color measurement method of agricultural products under natural lighting conditions. We then analyzed the relationship between the color appearance differences of the images taken under the standard and natural lighting conditions based on the illuminating radiance spectral characteristics. Based on the photosensitive characteristics of five kinds of tested cameras, the applicability was studied using the images of tomato fruits whose surface color dynamically changes from green to red. As the results, the RGB values of the images of the red, green, and blue sections on the virtual standard color chart could be calculated using the simple spectral features of the various natural lighting conditions, and consistently agreed with the actual values. In addition, the images of tomato fruits acquired under the natural lighting conditions using the tested cameras were successfully calibrated based on the color parameter differences between the standard color images taken under the standard and natural lighting conditions.