The effects of large round‐off errors on the performance of control charts for the mean when the quality characteristic is normally distributed with a known variance

Abstract

Abstract This research discusses the effects of large round‐off errors on the performance of control charts for means when a process is normally distributed with a known variance and a fixed sample size. Quality control in practice uses control charts for means as a process monitoring tool, even when the data is significantly rounded. The objective of this research is to demonstrate how ignoring the round‐off errors and using a standard Shewhart chart affects the quality control of a measured process. The first part of the research includes theoretical calculations for estimating the values of alpha and beta, relating to the unrounded data and the large‐rounded data. For the rounded data, normality can no longer be assumed because the data is discrete; therefore, the multinomial distribution is used. The results show that given an in‐control process, alpha indicates that false alarms are more frequent, whereas given an out‐of‐control process, the influence on beta is minor and inconsistent. For some rounding levels, there is a decline in the control chart performances, and in others, there is an improvement. In the second part, a simulation study is used to evaluate the performances of the control chart based on a single sample, checking whether the conclusion (reject or fail to reject) for a sample is consistent for rounded and unrounded data. The results of the simulation match the theoretical calculations.

Keywords

• measurement errors
• measurement theory
• normal distribution
• process monitoring