c© Heldermann Verlag Economic Quality ControlISSN 0940-5151 Vol 16 (2001), No. 1, 5 – 15

ARL Comparisons Between Neural Network Models

and x̄-Control Charts for Quality Characteristics

that are Nonnormally Distributed

Junsub Yi, Victor R. Prybutok and Howard R. Clayton

Abstract: One widely used control chart, the x̄-chart, is based on the assumption thatmeans of samples drawn from the process are normally distributed. When the normalityassumption is not valid, control chart users may choose from several different courses ofaction. These include using Box-Cox power transformations on the original data to yield anapproximate normal distribution, increasing the size of the samples drawn from the processuntil the distribution of the sample means is considered normal, and modifying the x̄-chartto employ asymmetric control limits instead of limits that are equidistant from the processtarget mean. Since none of the remedies for handling nonnormal processes is completelysatisfactory, we build on previous neural network research by developing a neural network tocontrol nonnormal processes. Comparison of the performance of our neural network modelwith that of traditional x̄-control charts shows that the neural network model is superior tothe traditional x̄-control charts.

Key Words: Statistical Quality Control, Statistical Process Control, Control Charts, NeuralNetworks, Simulation

1 Introduction

The control chart was introduced in 1925 by Shewhart in the Journal of the AmericanStatistical Association. Since then control charts have become a powerful tool to monitorprocess means and variances [12]. When used appropriately, control charts help determineif processes are “in control” or “out of control” A process found to be out-of-control canbe corrected, thereby reducing the variability of the process means [1].

One of the most widely used control charts, the x̄-chart, is based on the assumption thatthe distribution of the means of samples drawn from the process is sufficiently close to anormal distribution. However, in many industrial processes, this assumption is invalid.The nonnormality of sample means can have a significant effect on the performance of thex̄-chart [21]. If the x̄-chart is used when the normality assumption is invalid, the chanceof wrongly interfering with an in-control process or allowing an out-of-control process togo uncorrected can be significant. Consequently, design considerations for the x̄-chartmust include recognition of the degree of nonnormality of the underlying data [18, 23].

6 Junsub Yi, Victor R. Prybutok and Howard R. Clayton

When the normality assumption is not valid, control chart users may choose from severaldifferent courses of action. Rigdon [19] suggested using Box-Cox power transformationson the original data to yield an approximate normal distribution. The usual charts couldthen be used on the transformed data. The shortcomings of the transformation approachare first, the difficulties in determining the appropriate transformation, and second, whena transformation has been chosen, the difficulties in implementing the procedure andjustifying the transformation [13, 23]. Yourstone and Zimmer [23] suggested anothercourse of action that requires increasing the size of the samples drawn from the processuntil the distribution of the sample means is considered normal. However, large samplesizes are often not operationally feasible and can be costly. A third course of action is tomodify the x̄-chart. One way to modify the chart is to employ asymmetric control limitsinstead of limits that are equidistant from the process target mean. The difficulty with thisapproach is that it requires at least a good approximation of the underlying distributionof the process data. The cumulative sum (CUSUM) control chart is an alternative to thex̄-chart for controlling nonnormal data [9]. Despite its usefulness in detecting small shiftsin the process mean, the CUSUM chart has been found to be inferior to the x̄-chart indetecting relatively large shifts. In general, none of the remedies for handling nonnormalprocesses has been found to be completely satisfactory.

Previous researchers have developed economic designs for x̄, CUSUM , and asymmetriccontrol charts. For example, the economic design of an x̄-chart entails determining theoptimal sample size, sampling interval, and control limit coefficients that minimize thecosts associated with designing the chart [9, 17, 5]. As in the case of the previouslymentioned remedies, none of these designs is uniformly satisfactory for dealing with thenonnormality problem.

A number of authors [7, 10, 14, 15, 16], and others, have used neural networks to monitorprocess means when data are normally distributed. We build on the previous research bydeveloping a neural network to control nonnormal processes. We compare the performanceof our neural network model with that of traditional x̄-control charts. In the next section,we begin with a description of the model for generating process data then we describe ourneural network model, the training of the neural network, and the performance measuresfor the control charts. In the last section we present and discuss the results of our research.

2 Research Methodology

2.1 Data Generation

For our investigation we generated process data from the Burr distribution which wasutilized in a previous simulation study by Yourstone and Zimmer [23]. The distributionfunction FX(x) of the Burr distribution is given by

FX(x) =

{0 for x < 0

1 − 11+xc

kfor x ≥ 0

(1)

Comparisons Between Neural Network Models and x̄-Control Charts 7

where c, k ≥ 1.

A wide variety of probability distributions, which may be used in industrial environments,can be approximated with the Burr by judicious choice of c and k values. In accordancewith Yourstone and Zimmer, we used skewness and kurtosis to measure the degree ofdeparture from normality of a given distribution. Burr [3, 4] tabled c and k values thatrelate to various combinations of skewness and kurtosis. We found this table very usefulto generate distributions with a desired skewness and kurtosis.

Table 1: The Simulated Processes

Description skewness kurtosis c kCase 1 Almost uniform distribution 0 2 -18.1484 0.0629Case 2 Normal 0 3 -11.2519 0.1463Case 3 Symmetric and peaked 0 4 27.0689 1.3258Case 4 Positively skewed 1 3 -13.0680 0.0301Case 5 Positively skewed and peaked 1 4 -7.1760 0.0787Case 6 Positively skewed and peaked 1 5 2.3471 4.4286Case 7 Highly skewed, highly peaked 2 6.2 -21.4163 0.0074Case 8 Highly skewed, severely peaked 2 7.2 -7.5077 0.0270Case 9 Almost exponential distribution 2 8.8 -5.8643 0.0442

Table 1, above, describes the skewness and kurtosis of nine simulated distributions inves-tigated in our study along with the c and k values used in their generation from the Burrdistribution. As can be seen, we chose distributions that were symmetrical, moderatelyskewed and peaked, as well as severely skewed and peaked. Although some of these situ-ations may not regularly be encountered in monitoring real processes, we included themfor the sake of comparing the neural network and control charts.

2.2 Neural Network Description

In this study we used a fully-connected feed-forward network with five input neurons,five hidden neurons, and three output neurons. Choosing five input neurons was logicalonce we decided upon using samples of size five because each neuron would represent anobservation in the sample. The simplicity of this neural network architecture guaranteeda realistic training time. We implemented the back-propagation learning algorithm forour neural network. Although the slowness of learning is usually a major concern withback-propagation networks, the learning speed of the training sets was fast enough foruse in our study.

In addition to the architectural features of number of hidden layers, number of hidden layerneurons, and number of output neurons, factors such as the learning rate, momentum,learning rule, and type of transfer function are crucial to the performance of a trainedneural network We used a learning rate of 0.1 and a momentum of 0.6 after testing morethan 30 different combinations of the parameters. A sigmoid transfer function was usedbecause of its desirable features. It is monotonic, differentiable, and semi-linear and is

8 Junsub Yi, Victor R. Prybutok and Howard R. Clayton

partially insensitive to noise. We pilot-tested on the data the two most popular learningrules: the generalized delta and cumulative delta. We employed the generalized deltalearning rule because it yielded better results.

3 Training of the Neural Network

For the neural network to satisfactorily monitor a process, the range of shifts in the processmean utilized in the training procedure should be representative of the whole spectrumof the shifts of interest. However, the determination of the range of mean shifts usedin training is subject to the required recognition capability, and the expected trainingtime. The magnitude of shifts in the process mean was measured in units of the standarddeviation of the mean (σx̄). When small magnitudes of process changes were includedin the training set the neural network performed poorly, registering a high false alarmrate (an increase in commitment of Type I errors). This poor performance for smallshifts in the process mean was probably due to the high overlap between the in-controlobservations and out-of-control observations. As a result, data with small shifts (.25, .5,and .75 σx̄) were not included in the training sets.

Consistent with previous simulation studies on control charts [6, 20, 22] we used a targetmean (µ0 ) of zero in this study. The number of training examples was determined throughexperimentation. For each combination of skewness and kurtosis we used 900 examples ina training set. The 900 comprised 100 examples of in-control processes and 100 exampleseach of out-of-control processes with positive and negative shifts ranging from ±1 to ±4units. The value of the three output neurons were set to [0, 0, 1] to signal an in-controlprocess, [0, 1, 0] to signal an out-of-control process with a positive shift, and [1, 0, 0] tosignal an out-of-control process with a negative shift.

In neural network training, the weights connecting the nodes were adjusted after a cycleconsisting of a fixed number of presentations of training samples. The number of trainingsamples between each adjustment is defined as the epoch which can be related to thespeed of training process. We tested several epochs. Our results showed that the epochhad no effect on the speed of learning. Another concern, related to training time, isthe determination of the optimal stopping point of learning process. During the trainingprocess, the performance of the neural network on the training data set continuouslyimproves. However, the longer the neural network learns the training data set, the closerthe neural network comes to memorizing the training data set. Thus, the neural networkmay make a very accurate prediction on the training data set but may not be able togeneralize well on a fresh data set. There exists no method of optimizing the selection of astopping point or the selection of parameters. This means that any neural network solutionmight be sub-optimal. We relied on the software (SAVEBEST utility in NeuralWare 1993)to determine the stopping point.

Comparisons Between Neural Network Models and x̄-Control Charts 9

3.1 Data Collection and Performance Measures

Following the convention established in previous research, comparison of the neural net-work and x̄-control charts was based on average run length (ARL) which is defined asthe average number of subgroups (samples) that are observed from a process before anout-of-control signal is detected. For a given process, this average was based on 10,000simulated replications. When the process is in-control (not shifted), it is desirable to havea large ARL. However, if the mean of the process is out-of-control (shifted), it is desirableto have a small ARL. In other words, the monitoring process should generate signals asquickly as possible (i.e., give shorter ARLs) if the production process is out of control,and as late as possible (i.e., give longer ARLs) if the production process is in control.

As mentioned in the training of the neural network, the binary digits (0, 1) were usedon the output neurons to signal in-control and out-of-control states. We recognized that,owing to the use of the sigmoid function to train the neural network, the output neuronscould assume any value between 0 and 1. Therefore we had to determine cut-off values foroutput neurons a and b in the output vector [a, b, c] that could be deemed close enoughto 1 to signal, respectively, an out-of-control process due to a negative or a positive shift.(Recall that target values a = 1, b = 0, c = 0 were selected to designate out-of-control dueto a negative shift and a = 0, b = 1, c = 0 to designate out-of-control due to a positiveshift).

Table 2: Cutoff points for neural network models

Case Neural network cutoff point (a) Neural network cutoff point (b)1 0.8236 0.92242 0.7675 0.87383 0.5235 0.85784 0.7001 0.87185 0.8700 0.48506 0.8375 0.51707 0.5185 0.63708 0.6400 0.53309 0.8400 0.3820

Selecting low cutoff values would increase the probability of committing a Type I error (therate of false alarm; concluding the process is out-of-control when it is really in-control),but would reduce the probability of committing a Type II error (concluding the processis in-control when it is really out-of-control). On the other hand, selecting high cutoffvalues would increase the probability of Type II error, but reduce the probability of aType I error. To obtain fair comparisons between the x̄-chart and neural network model,the cutoff values for a and b were determined by trial and error simulations so that thein-control ARLs of the neural network would match those of the x̄-charts for each one ofthe nine simulated processes. The cutoff values are shown in Table 2.

10 Junsub Yi, Victor R. Prybutok and Howard R. Clayton

4 Simulation Results

For the sake of brevity, only the results for Cases 2, 4, 6, and 9 will be presented anddiscussed. These cases were chosen to illustrate any effect caused by (1) increasing theskewness at a given kurtosis level (Case 2 vs. Case 4), (2) increasing the kurtosis at agiven skewness level (Case 4 vs. Case 6), and (3) increasing both skewness and kurtosislevels (Case 6 vs. Case 9). The results for the four chosen cases are illustrated in Tables 3- 6 which contain the ARL values for the neural network and x̄-control chart for variousshifts in the process. In addition, the tables list ratios (denoted by ρ) of each x̄-controlchart ARL to the corresponding neural network ARL. For any shift in the process, valuesof ρ substantially larger than one indicate that the neural network is better than the x̄-control chart while values of ρ substantially smaller than one indicate the x̄-control chartto be better than the neural network. In all the cases the in-control ARLs of the twocontrol procedures were kept the same for ease of comparison of the out-of-control ARLs.

Table 3: Comparison of ARL between and neural network (skewness=0, kurtosis=3)

Shift x̄-chart NN ρ Shift x̄-chart NN ρ0.00 372.28 371.96 0.000.25 280.16 272.85 1.03 -0.25 282.48 292.57 0.960.50 157.1 151.09 1.04 -0.50 153.81 163.16 0.940.75 83.27 79.48 1.05 -0.75 81.06 85.70 0.941.00 44.9 43.48 1.03 -1.00 43.94 45.87 0.961.50 15.01 14.73 1.02 -1.50 14.99 15.68 0.952.00 6.28 6.17 1.02 -2.00 6.31 6.51 0.973.00 1.98 1.97 1.01 -3.00 2.02 2.05 0.994.00 1.19 1.19 1.00 -4.00 1.18 1.19 0.99

where ρ = ARL of x̄-chartARL of neural network

.

4.1 Case 2 (skewness=0, kurtosis=3), Table 3

In Case 2, we examine a normal process in which process shifts in positive and negativedirections must be detected equally well because of the distribution’s symmetry. Thisproperty is known as the directional invariance property. Table 3 shows roughly similarout-of-control ARLs for the neural network and x̄-control chart for both positive andnegative shifts in the process mean. As a result the ρ values are all close to one, indicatingno advantage in utilizing the neural network over the x̄-control chart.

4.2 Case 4 (skewness=1, kurtosis=3), Table 4

Here we have a process distribution with the same kurtosis as in the normal (previous)case but with a positive skew. In Table 4 we observe ρ values close to one for positiveshifts but much larger than one for negative shifts. This indicates that for positivelyskewed processes the neural network model gives an out-of-control signal much quickerthan the x̄-control chart when the process mean shifts off target in a direction opposite the

Comparisons Between Neural Network Models and x̄-Control Charts 11

skew. For example, the 4.97 for the ρ value at mean shift -0.25 tells us the neural networkmodel reacts nearly five times as fast to a genuine shift than the standard x̄-control chart.Comparing the results in Table 3 and Table 4 points to the effect on the match-up betweenthe neural network and x̄-control chart by a change only in the skewness of the process.We observe an impressive ascendance of the neural network model.

Table 4: Comparison of ARL between and neural network (skewness=1.0,kurtosis=3.0)

Shift x̄-chart NN ρ Shift x̄-chart NN ρ0.00 256.70 256.38 0.000.25 142.29 142.80 1.00 -0.25 477.90 96.25 4.970.50 81.20 81.71 0.99 -0.50 936.22 29.52 31.710.75 48.70 49.01 0.99 -0.75 1876.82 13.70 136.991.00 30.23 30.79 0.98 -1.00 392.54 7.73 50.781.50 12.74 12.79 1.00 -1.50 19.17 3.46 5.542.00 6.13 6.22 0.99 -2.00 5.99 2.07 2.893.00 2.15 2.16 1.00 -3.00 1.88 1.25 1.504.00 1.20 1.21 0.99 -4.00 1.21 1.05 1.15

4.3 Case 6 (skewness=1.0, kurtosis=5.0), Table 5

The effect of a change in only the kurtosis of the process data can be observed by compar-ing the results in Table 5 with those in Table 4. Table 5 shows ρ values larger than onefor negative shifts but not nearly as large as those in Table 4. We can infer that althoughthe neural network once more exhibits superiority over the x̄-control chart, signaling asmuch as 82% faster for a -0.75 shift, the neural network’s ascendancy over the x̄-controlchart is only moderate for an increase in kurtosis.

Table 5: Comparison of ARL between and neural network (skewness=1.0,kurtosis=5.0)

Shift x̄-chart NN ρ Shift x̄-chart NN ρ0.00 193.76 193.81 0.000.25 121.69 123.69 0.98 -0.25 297.93 267.52 1.110.50 76.02 77.54 0.98 -0.50 336.35 223.12 1.510.75 48.06 48.73 0.99 -0.75 203.60 112.00 1.821.00 30.60 31.45 0.97 -1.00 88.08 49.66 1.771.50 13.23 13.66 0.97 -1.50 19.19 12.77 1.502.00 6.41 6.57 0.98 -2.00 6.46 4.85 1.333.00 2.11 2.15 0.98 -3.00 1.90 1.69 1.124.00 1.18 1.19 0.99 -4.00 1.18 1.14 1.04

12 Junsub Yi, Victor R. Prybutok and Howard R. Clayton

Table 6: Comparison of ARL between and neural network (skewness = 2,kurtosis = 8.8)

Shift x̄-chart NN ρ Shift x̄-chart NN ρ0.00 116.86 117.04 0.000.25 77.69 77.55 1.00 -0.25 175.65 57.31 3.060.50 52.81 52.43 1.01 -0.50 265.88 12.72 20.900.75 36.03 35.74 1.01 -0.75 400.02 6.31 63.391.00 24.84 24.72 1.00 -1.00 597.13 3.9 153.111.50 12.28 12.32 1.00 -1.50 74.26 2.2 33.752.00 6.36 6.36 1.00 -2.00 6.16 1.54 4.003.00 2.27 2.27 1.00 -3.00 1.77 1.13 1.574.00 1.2 1.20 1.00 -4.00 1.18 1.03 1.15

4.4 Case 9 (skewness=2, kurtosis=8.8), Table 6

Comparing the results in Table 6 with those in Table 5 gives an opportunity to observethe effect of an increase in both skewness and kurtosis of the process data. Table 6 showsa substantial increase in ρ values for negative shifts. Because the extent of the increase inρ values is about the same as was observed when only the skewness was increased (Case2 vs. Case 4), it is difficult to delineate the role of increased kurtosis here. We can say,however, that once more the neural network shows vast superiority over the x̄-controlchart in detecting shifts in the process opposite to the direction of the skew. As for allthe other cases, we observe no difference between the two monitoring systems in detectingpositive shifts in the process.

5 Discussion and Conclusion

In this paper the performance of the neural network was evaluated by estimating theARLs. The neural network approach presented here offers a competitive alternative toexisting control schemes. Because the performance of the neural network depends onthe selection of training samples, the results reported in this research are not necessarilyoptimal. However, we have demonstrated the feasibility of applying a neural networkmodel to monitoring a process.

The comparisons showed that use of the neural network model appeared to be a bettercontrol procedure for detecting sudden changes in the process mean than the x̄-controlchart. The neural network developed in this study was primarily designed to detect asudden shift in the process mean. Nevertheless there are other characteristics of a process,such as trends, cycles, systematic and stratification patterns and mixtures, that may causeone to regard the process as out-of-control. In future research these characteristics oughtto be considered and might need to be redefined under nonnormality.

Comparisons Between Neural Network Models and x̄-Control Charts 13

Many authors [2, 8, 14] suggest that neural networks offer advantages over traditionalstatistical methods. Among these advantages is the employment of a pattern recogni-tion approach by neural networks as opposed to a complex statistical approach. Otheradvantages of neural networks over control charts for detecting out-of-control conditionsare their flexibility and high speed computation. In practical terms this flexibility allowsan analyst to design a desired control scheme corresponding to the particular productionprocess. According to Hwarng and Hubele [11], flexibility in training and high-speedcomputation resulting from the parallel structure of the networks enable neural networkmodels to be used in real-time applications.

Even though several researchers [7, 10, 11, 14, 15, 16] have developed neural networks asalternatives to the standard x̄-control charts for monitoring process means, these studiesare limited in number and the extent of their coverage. The studies were conductedunder the assumption the process being monitored is normally distributed while in manymanufacturing environments such an assumption may be invalid. An infinite number ofpotentially complicated nonnormal situations can develop from process data possessingdifferent combinations of skewness and kurtosis. The need to better define approachesfor handling nonnormal data was a major motivation for our research. One of the majorcontributions of this paper is to demonstrate a competitive alternative to standard controlcharts under conditions of nonnormality.

References

[1] Aft, L. S. (1988): Quality Improvement Using Statistical Process Control. HarcourtBrace Jovanovich, San Diego.

[2] Archer, N. and Wang, S. (1993): Application of the Back Propagation Neural Net-work Algorithm with Monotonicity Constraints for Two-Group Classification Prob-lem. Decision Sciences 24, 60-75.

[3] Burr, I. W. (1967): The effect of non-normality on constants for and R charts.Industrial Quality Control, May, 563-569.

[4] Burr, I.W. (1973): Parameters for a general system of distributions to match a gridof α3 and α4. Communications in Statistics 2, 1-21.

[5] Collani, E.v. (1997): Determination of the Economic Design of Control ChartsSimplified. In Optimization in Quality Control. Eds. Khaled S. Al-Sultan and M.A.Rahim. Kluwer, Boston, 89-143.

[6] Crowder, S. V. (1987): A Simple method for studying run length distributions ofexponentially weighted moving average charts. Technometrics 29, 401-407.

[7] Davis, S. and Illingworth, B. (1989): Neural Network Simulation Applied to Statis-tical Process Control. Internal Paper, Texas Instruments Incorporated, Dallas.

14 Junsub Yi, Victor R. Prybutok and Howard R. Clayton

[8] Hill, T., Marquez, L., O’Connor, M. and Remus, W. (1994): Artificial Neural Net-work Models for Forecasting and Decision Making. International Journal of Fore-casting 10, 5-15.

[9] Ho, C., and Case, K. E. (1994): Economic Design of Control Charts: A LiteratureReview for 1981-1991. Journal of Quality Technology 26, 39-53.

[10] Hwarng, H. B. and Hubele, N. F. (1993): Back-propagation Pattern Recognizers forx̄ Control Charts: Methodology and Performance. Computers Industrial Engineer-ing 24, 219-235.

[11] Hwarng, H. B. and Hubele, N. F. (1993): barX control chart pattern identificationthrough efficient off-line neural network training. IIE Transactions 25, 27-38.

[12] Keats, J. B. and Hubele, N. F. (1989): In Statistical Process Control in AutomatedManufacturing. Marcel Dekker, New York.

[13] Montgomery, D. C. (1991): In Design and Analysis of Experiments. 3rd ed, Wiley,New York.

[14] Prybutok, V. R., Sanford, C. C. and Nam, K.T. (1994) A Comparison of NeuralNetwork to Shewhart X-bar Control Chart Applications. Economic Quality Control9, 143-164.

[15] Pugh, G. A. (1989): Synthetic Neural Networks for Process Control. Proceedings ofthe 13th Annual Conference of Computers Industrial Engineering 17 (1-4), 24-26.

[16] Pugh, G. A. (1991): A Comparison of Neural Networks to SPC Charts. Proceedingsof the 15th Annual Conference of Computers Industrial Engineering 21 (1-4), 253-255.

[17] Rahim, M. A. (1985): Economic Model of x̄-Charts Under Non-normality AndMeasurement Errors. Computers and Operations Research 12, 291-299.

[18] Ramsey, P. P. and Ramsey, P. H. (1990): Simple tests of normality in small samples.Journal of Quality Technology 22, 299-309.

[19] ] Rigdon, S. E., Cruthis, E. N. and Champ, C. W. (1994): Design Strategies forIndividuals and Moving Range Control Charts. Journal of Quality Technology 26,274-287.

[20] Runger, G. C. and Pignatiello, H. J. Jr. (1991) Adaptive sampling for process con-trol. Journal of Quality Technology 23, 135-155.

[21] Ryan, T. P. (1989): Statistical Methods for Quality Improvement. Wiley, New York.

[22] Walker, E., Philpot, J. W. and Clement, J. (1991): False signal rates for the She-whart control chart with supplementary runs tests. Journal of Quality Technology23, 247-252.

Comparisons Between Neural Network Models and x̄-Control Charts 15

[23] Yourstone, S. A. and Zimmer, W. J. (1992): Non-normality and the Design ofControl Charts for Averages. Decision Sciences 23, 1099-1113.

Junsub YiDept. of Management Information SystemsKyungsung University110-1 Daeyeon-dong, Nam-gu,Pusan608-736, South Koreajunsub@star.kyungsung.ac.kr

Victor R. PrybutokUniversity of North TexasDenton TX 76203-5249prybutok@unt.edu

Howard R. ClaytonAuburn UniversityAuburn, AL 36849-5241hclayton@business.auburn.edu