Background: Pupillometers have been proposed as clinical assessment tools. We compared two pupillometers to assess measurement agreement. Materials & methods: We enrolled 30 subjects and simultaneously measured the pupil diameter and light reflex amplitude with an iPhone pupillometer and a portable infrared pupillometer. We then enrolled 40 additional subjects and made serial measurements with each device. Results: Failure occurred in 30% of attempts made with the iPhone pupillometer compared with 4% of attempts made with the infrared pupillometer (Fisher’s exact p = 0.0001). Method comparison of the two devices used simultaneously showed significant disagreement in dynamic measurements. Conclusion: The iPhone pupillometer had poor repeatability and suggests that it is not a practical tool to support clinical decisions.