A MULTI-CRITERIA BASED ROBUST DESIGN APPROACH

Arun Kunjur

Graduate Research Assistant

Sundar Krishnamurty

Associate Professor

Department of Mechanical Engineering

University of Massachusetts

Amherst, MA 01003

Ph: (413) 545-0297

Fax: (413) 545-1027

Email: skrishna@ecs.umass.edu

Abstract

The overall value of a product is typically determined by its performance with respect to multiple measures. As such the product design task is very much simplified if all these performance measures were optimized simultaneously rather than sequentially. Another significant factor that determines product quality is its sensitivity to external or uncontrollable variations. To effectively address these issues in product design, this paper presents a methodical approach that integrates multi-criteria optimization concepts with statistical robust design techniques. In this approach, which includes a systematic treatment of constraints, the results are presented as a set of non-inferior design solutions. These solution sets are obtained from the initial design space based on a statistical design of virtual experiment set-up, that utilizes ANOVA results to quantify the relative dominance and significance of design factors.

Introduction

Robust design techniques based on the concept of building quality into a design, are increasingly finding new applications in the industry mainly because of their simplicity and practicality. Taguchi's Methods (Roy, 1990; Kacker, 1985) have been extensively applied in process design, wherein mathematical models for performance do not exist and experiments are typically conducted to determine optimum settings for design and process variables. However, Taguchi's methods have not been fully exploited in the parametric design of products (Kota and Chiou, 1993 ; Otto and Antonsson, 1991). Conventional optimization techniques employed in product design such as direct search and gradient based methods generally rely on a large number of function and gradient evaluations to determine the optimum point. This is particularly true if multiple objectives are involved. Taguchi based methods have been used in product design primarily to reduce the number of function evaluations or to obtain a good starting point for other optimization techniques. Here, special performance measures called signal-to-noise ratios, that will lead to a reduction in the sensitivity of the product to uncontrollable variations, are employed in the orthogonal array experiments towards achieving a robust design. However, the main drawback with the application of the basic form of Taguchi's method, to product design, is its inability to address two important issues - (1) a systematic treatment of constraints and (2) a formulation for multiple objectives.

Multiple criteria optimization techniques can be broadly classified into preference based methods and generating methods (Cohon, 1985). Preference based methods attempt to quantify the decision makers preferences or relative importance for each criterion and use this information to obtain a single super criterion. Standard optimization techniques are then used to isolate a single optimum design. Goal programming (Charnes and Cooper, 1961) and utility analysis (Keeny and Raiffa, 1976 ; vonNeuman and Morgenstern, 1947) are some examples of preference based methods. Goal programming requires the allocation of a goal for each criterion and a weight that reflects the relative importance of deviations from that goal. The problem then is to find a solution that is as close as possible to the set goals. Utility analysis requires quantification of attribute levels, which may not be known a priori, to reduce the objective in a probabilistic sense, to a single overall utility function. A salient feature of utility analysis is that it can be used to evaluate alternatives under uncertainty. Generating methods (Cohon, 1985), on the other hand, have been developed to enumerate the exact non-inferior set or an approximation of it. Here a feasible solution to a multi-criteria optimization problem is considered non-inferior (Pareto-optimal) if there exists no other feasible solution that will yield an improvement in one criterion without causing a degradation in at least one other criterion. This information is then presented to the decision maker, who is required to select a design that is most suitable. A major drawback of this method is that most real world problems are too large to allow the exact non-inferior set to be found and, even if it were generated, the set would include too many alternatives for the decision maker's consideration. The weighting and constraint methods (Cohon, 1985) are the most widely used generating techniques. They operate by converting the multiple objectives into a single-criterion using certain parameters, which are then varied to obtain different Pareto-optimal solutions.

In this paper, Taguchi's method has been further developed to handle multiple objective product design problems using generative techniques and a rational approach to treat constraints in such problems is presented. Application of Taguchi Method for constrained problems has primarily been dealt with by introducing a penalty function in the performance parameter (Otto and Antonsson, 1991). This is not possible for most problems as the inclusion of a penalty function may make the performance parameter non-additive or even discontinuous. Furthermore, the optimization of multiple performance parameters by using weighting functions to arrive at a single measure of performance can lead to erroneous results because of the difficulty involved in making the trade-off decisions to arrive at this single expression.

This paper addresses these problems and proposes a methodology that further develops Taguchi's method to incorporate multiple objectives and constraints in product design. In this methodology, statistical analysis (ANOVA) concepts are used to obtain a well diversified, though not necessarily exhaustive, set of Pareto-optimal solutions. It is expected that this integrated procedure will provide a rational platform for the systematic identification of robust designs in a multiple objective domain.

Robust Design For A Single Response

The Taguchi concept of robust design is based on maximizing performance measures called signal-to-noise ratios by running a partial factorial set of experiments using orthogonal arrays. The signal-to-noise ratio is typically given by

S/N = -10 log [MSD] [1]

where MSD refers to Mean Square Deviation of objective function.

The S/N ratio aims at achieving the separability of design factors into control factors and signal factors. A robust optimum design is identified by finding the optimum setting of the control factors to reduce variation and then adjusting the signal factors to shift the mean.

Additivity of factor effects is an important consideration in statistical design of experiments. It ensures that the performance measure is not adversely affected by the non-linearities of the objective function. Note that this form of the S/N ratio does not guarantee separability and additivity for all types of objective functions, and thus the use of transformations such as -transformations may become necessary (Box, 1988).

The application of the Taguchi concept to product design, which is usually characterized by continuous design variables, requires that the design space be discretized by splitting each variable range into a desired number of intervals. The non-linear effects of the response can be captured by using at least three levels (two intervals) for each factor. Standard orthogonal arrays, developed by Taguchi, are used to design the experiments (inner array) and to simulate the noise effects (outer array), if any. The objective function is evaluated, and the S/N ratio or a suitably transformed performance measure (henceforth generalized to S/N) is determined for all the experiments. ANOVA is performed on this S/N ratio to determine the effects of different factors on performance. Specifically, the sum of squares, mean squares, F-values, and percentage contribution is computed to determine which factors contribute significantly to the variance and mean response. This information is then used to predict the optimum settings for the various design factors.

Robust Design For Multiple Responses With Constraints

Taguchi's method, in its as is form, does not handle multiple objective formulations or constraints. This paper extends Taguchi's method to address such cases. The problem setup for the multiple responses case is similar to the single response case except that each objective and constraint function will result in a separate ANOVA table. A two step procedure is then followed to identify the non-inferior set :

1) Eliminate all factor levels that cause any of the constraints to be violated.

2) Identify those factor levels, in the reduced space, that significantly effect any of the objectives.

A factor is assumed to have a dominant effect on an objective function if its percentage contribution (or F-value) is considerably higher than that of all other factors for that objective. It has a significant effect if the percentage contribution is greater than a designer specified cut-off value. Details on this cut-off value and its selection are addressed later in this section.

The signal-to-noise ratio for the constraint functions are set up for the larger is better case if it is a 'lesser than' constraint (and vice versa), so that the factors that cause a violation of the constraint can be easily identified. Using the ANOVA table, the effect of the various factor levels on the constraint functions are first considered. If the ANOVA of a constraint function indicates that it is significantly effected by a factor, the level of the factor causing the violation is identified. If the mean effect of this level on the constraint function is beyond the constraint limit, then this level is eliminated from further consideration. Note that before comparing constraint mean effects with the constraint limit, the latter has to be transformed by the same transformation that is used to obtain the S/N ratio from the constraint equation. In this manner, the non-feasible designs are filtered out from the initial design space.

The ANOVA of each of the objective functions are then considered, and the design space is further reduced according to the following rules :

1) If a factor has a significant effect on all objective functions, then all the levels that optimize at least one objective are selected.

2) If a factor has a dominant effect on a single objective, the factor level that optimizes this objective is selected regardless of its significance on other objectives.

3) If a factor has insignificant effect on all the objectives, then the designers discretion is used to determine the objective most effected by this factor and the best level for the factor is identified.

This procedure thus attempts to identify non-dominated designs, but note that it does not ensure the elimination of all dominated designs from the final set. The proportion of dominated designs in the final set depends to a large extent on the designer determined cut-off value for identifying significant factors. A high value for this cut-off indicates that the designer does not want any dominated designs in the final set. In this case, the designer is willing to eliminate some non-dominated designs to reduce the cost of processing the final set. Alternatively, a low cut off value will result in a large final design set, and consequently a sizable number of dominated designs, that have to be filtered out by further processing. Depending on the choice of the designer, further processing would mean either another iteration on the reduced design space or filtering out the dominated designs using a standard procedure such as the technique of dominated approximations (Majchrzak, 1989). Thus the cut-off value is a trade-off between the number of non-inferior designs that the designer requires and the amount of effort he/she is willing to put in to identify these.

The procedure outlined above does not guarantee an exhaustive non-inferior set, but it should be noted that enumeration of such a set may not be preferred in most practical design situations as it will only compound the complexity of the final selection process. On the other hand, providing a representative set in a systematic manner as shown here will greatly facilitate the identification of appropriate designs as either the final design or for further refinement.

Problem Formulation And Results

The application of this methodology to multi-criteria optimization problems is demonstrated by a beam design problem (Osyczka, 1985) involving two objectives and a single constraint. The objective of this problem is to determine the dimensions of a beam (Figure 1), that minimizes the cross section area for a given beam length and the deflection of the beam under given loads. The various parameter values for the problem are:

Permissible bending stress of the beam material is 16 kN/cm2.

Young's Modulus of Elasticity (E) is 2*104 kN/cm2.

Maximal bending forces P = 600 kN and Q = 50 kN.

Length of the beam (L) is 200 cm.

The problem can be mathematically expressed as follows:

Minimize Cross Section Area = f1(x) = 2 x2 x4 + x3 ( x1- 2 x4 )

Minimize Vertical Deflection = f2(x) = P L3 / 48 E I

Subject to the constraint

Maximum Stress = f3(x)

where I is the Moment of Inertia of the beam cross section.

The geometric constraints are :

10 x1 80, 10 x2 50, 0.9 x3 5, 0.9 x4 5.




Figure 1 : Beam design problem

To capture non-linear effects of the various responses, four levels were initially chosen for each of the four design variables, within the specified range. Thus, neglecting interaction effects, the total degrees of freedom of the system is 12. The L16 orthogonal array was used to design the experiments for this problem. It was found from the mean effects table, that both the area and the deflection equations satisfy the additivity condition and thus do not require any transformation or study of interactions. The constraint equation was found to be non-additive, but this does not effect the result as the analysis of constraint equations is done only to eliminate invalid designs and not to predict constraint values for the optimum design. Dimensional variation in the design variables were treated as the appropriate noise factors and an L9 array was used to simulate these variations. The experiments are set up as shown in Table 1.

The objective functions are computed for each combination of design variables and for each combination of noise values. The S/N ratios for the various experiments are shown in Table 1. The mean response of each factor level is then computed for each objective function and constraint equation. These are listed in Table 2. Table 3 lists the results of ANOVA for each objective function and constraint equation.

It is seen from the ANOVA of the constraint equation (Table 3c) that the factors x1, x2, and x4 have a significant effect on the constraint, assuming a 1% cut off for significance. The levels of these factors that maximize the constraint value are found from Table 2c to be the first level for all three factors. It is further observed that the mean
x4
0.99 1 1.01 1.01 0.99 1 1 1.01
0.99
x3
0.99 1 1.01 1 1.01 0.99 1.01 0.99
1
x2
0.99 1 1.01 0.99 1 1.01 0.99 1
1.01
x1
0.99 0.99 0.99 1 1 1 1.01 1.01
1.01
Exno
x1
x2
x3
x4
SN1
SN2
SN3
1
10
10
0.9
0.9
f1
24.87
25.29
25.71
25.36
25.29
25.49
25.36
25.56
25.48
-28.09
-21.62
48.89
f2
12.54
12.33
12.13
12.07
12.11
11.95
11.86
11.70
11.74
f3
286.72
283.09
279.52
277.40
278.59
278.57
272.92
273.07
274.25
2
10
23.3
2.3
2.3
f1
117.22
119.37
121.54
119.48
118.76
120.55
118.88
120.67
119.93
-41.56
-9.55
38.49
f2
3.12
3.08
3.04
3.02
3.01
2.97
2.95
2.91
2.91
f3
86.60
85.47
84.37
84.42
84.28
83.41
83.25
82.39
82.27
3
10
36.7
3.6
3.6
f1
268.86
273.96
279.11
274.03
272.04
276.86
272.14
276.96
274.91
-48.77
-4.45
33.72
f2
1.74
1.72
1.70
1.68
1.67
1.65
1.64
1.62
1.61
f3
49.98
49.43
48.90
48.89
48.54
48.06
48.01
47.54
47.21
4
10
50
5
5
f1
490.05
499.50
509.04
499.45
495.51
505.00
495.51
505.00
500.95
-53.98
-1.59
30.98
f2
1.25
1.24
1.22
1.21
1.20
1.19
1.18
1.16
1.15
f3
36.48
36.13
35.78
35.75
35.39
35.07
35.02
34.70
34.35
5
33.3
10
2.3
3.6
f1
129.40
131.26
133.14
131.86
132.08
132.15
132.68
132.74
132.95
-42.41
11.75
15.36
f2
0.27
0.26
0.26
0.26
0.26
0.26
0.25
0.25
0.25
f3
6.02
6.23
6.42
5.54
5.56
6.41
4.87
5.75
5.76
6
33.3
23.3
0.9
5
f1
248.92
253.67
258.47
253.86
251.94
256.09
252.15
256.30
254.34
-48.10
19.66
17.65
f2
0.11
0.11
0.10
0.10
0.10
0.10
0.10
0.10
0.10
f3
7.86
7.77
7.68
7.61
7.63
7.63
7.47
7.48
7.50
7
33.3
36.7
5
0.9
f1
219.11
221.90
224.69
223.46
224.57
222.65
226.16
224.20
225.31
-46.99
15.66
20.45
f2
0.17
0.17
0.17
0.16
0.16
0.16
0.16
0.16
0.16
f3
10.85
10.78
10.71
10.43
10.42
10.72
10.07
10.38
10.37
8
33.3
50
3.6
2.3
f1
326.69
332.12
337.60
333.13
332.22
334.59
333.26
335.61
334.66
-50.46
21.93
16.52
f2
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
f3
6.90
6.80
6.70
6.69
6.72
6.69
6.60
6.57
6.61
9
56.7
10
3.6
5
f1
262.78
266.08
269.39
267.75
269.16
267.44
270.86
269.10
270.51
-48.57
25.80
11.15
f2
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
f3
3.71
3.48
3.26
3.78
3.75
3.28
4.05
3.57
3.54
10
56.7
23.3
5
3.6
f1
407.00
412.43
417.88
414.88
416.42
414.46
418.92
416.91
418.44
-52.37
30.57
3.18
f2
0.03
0.03
0.03
0.03
0.03
0.03
0.03
0.03
0.03
f3
1.48
1.51
1.54
1.38
1.39
1.55
1.25
1.42
1.42
11
56.7
36.7
0.9
2.3
f1
211.42
215.20
219.01
215.65
214.53
216.93
215.01
217.39
216.24
-46.68
28.66
11.72
f2
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
f3
3.97
3.93
3.88
3.81
3.85
3.89
3.73
3.78
3.82
12
56.7
50
2.3
0.9
f1
211.97
214.97
217.98
216.22
216.67
215.91
217.95
217.16
217.61
-46.70
26.17
14.04
f2
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
f3
5.19
5.15
5.11
4.97
4.99
5.13
4.82
4.96
4.98
13
80
10
5
2.3
f1
414.58
419.00
423.42
422.77
426.54
419.69
430.35
423.42
427.23
-52.53
33.91
15.35
f2
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
f3
6.04
5.84
5.66
5.93
5.81
5.84
5.88
5.92
5.80
14
80
23.3
3.6
0.9
f1
317.02
320.58
324.14
323.39
325.92
321.06
328.76
323.85
326.40
-50.20
32.42
17.50
f2
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
f3
7.72
7.41
7.11
7.72
7.71
7.05
8.01
7.36
7.35
15
80
36.7
2.3
5
f1
517.49
526.16
534.90
527.73
526.17
530.06
527.80
531.65
530.03
-54.45
41.32
-0.46
f2
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
f3
0.98
0.97
0.97
0.93
0.94
0.98
0.89
0.93
0.94
16
80
50
0.9
3.6
f1
417.05
424.80
432.62
425.42
422.64
428.46
423.30
429.11
426.27
-52.58
40.90
2.48
f2
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
0.01
f3
1.37
1.36
1.34
1.32
1.33
1.34
1.29
1.31
1.32

Table 1 : Experiment set-up and S/N ratios

effects corresponding to these levels of x1 and x2 are above the transformed constraint limit given by 10*log(162) = 24.08. Consequently, these two levels are eliminated from further consideration. The ANOVA of the objective functions (Tables 3a and 3b) show that all the four factors (x1 - x4) have a significant effect on the cross section area (objective 1) whereas only the factors x1, x2, and x4 significantly influence the deflection of the beam (objective 2). Additionally, x1 is a dominant factor for beam deflection as its percentage contribution for deflection (92.7%) is much greater than that of any other factor. The level of x1 that optimizes deflection (level 4) is thus singled out for further consideration, regardless of its effect on the weight of the beam. The factors x2 and x4 have a significant effect on both the objectives and therefore a range of levels is obtained for these factors by identifying the levels that optimize each of the objective functions.
level 1
level 2
level 3
level4
x1
-43.0977
-46.98891
-48.57791
-52.439
x2
-42.89953
-48.05369
-49.22116
-50.92913
x3
-43.86072
-46.28051
-49.49674
-51.46554
x4
-42.99387
-47.80442
-49.03123
-51.27399

a) Objective 1
level 1
level 2
level 3
level4
x1
-9.30278
17.24992
27.80081
37.1364
x2
12.46102
18.27419
20.29889
21.85025
x3
16.89814
17.42388
18.92493
19.6374
x4
13.15836
18.73512
19.69314
21.29773

b) Objective 2
x1
38.02051
17.49263
10.02144
8.718296
x2
22.68757
19.20507
16.35471
16.00554
x3
20.18324
16.85627
19.72158
17.49179
x4
25.2187
20.52064
13.68373
14.82981

c) Constraint

Table 2 : Mean response for all objectives and constraints
level 1
level 2
level 3
level4
SS
DOF
MS
F0
% Cont
x1
29718.58
35327.32
37757.01
43997.58
179.5706
3
59.85686
11.14517
26.25771
x2
29445.92
36946.51
38763.57
41500.42
143.5511
3
47.85038
8.9096
20.47161
x3
30780.2
34270.16
39198.84
42379.23
136.5584
3
45.51947
8.47559
19.34831
x4
29575.56
36564.2
38464.98
42064.36
146.7245
3
48.90818
9.106559
20.98138
Residual
16.11196
Error
16.11196
3
5.370654
12.94099
Total
622.5166
15
100

a)Objective 1
level 1
level 2
level 3
level4
SS
DOF
MS
F0
% Cont
x1
1384.669
4760.957
12366.16
22065.8
4832.267
3
1610.756
2172.463
92.76362
x2
2484.433
5343.137
6592.718
7638.934
202.677
3
67.55899
91.11835
3.849803
x3
4568.753
4857.468
5730.45
6170.037
19.54822
3
6.516072
8.788375
0.332715
x4
2770.281
5616.074
6205.115
7257.493
150.1122
3
50.03741
67.4866
2.840268
Residual
2.224327
Error
2.224327
3
0.741442
0.213597
Total
5206.829
15
100

b)Objective 2
level 1
level 2
level 3
level4
SS
DOF
MS
F0
% Cont
x1
23128.95
4895.874
1606.869
1216.139
2198.467
3
732.8224
104.2603
80.33045
x2
8235.614
5901.353
4279.622
4098.835
115.3661
3
38.45536
5.471134
3.478278
x3
6517.812
4546.144
6223.049
4895.401
32.11156
3
10.70385
1.522862
0.406756
x4
10175.72
6737.548
2995.911
3518.772
343.4986
3
114.4995
16.29012
11.89481
Residual
21.08632
Error
21.08632
3
7.028774
3.889705
Total
2710.53
15
100

c)Constraint

Table 3 : Analysis of variance for all objectives and constraints

The factor x3 has a significant effect only on the weight of the beam, which is optimized by

setting x3 to the first level as seen from Table 2a. The factor levels that optimize the respective objectives, after eliminating the levels that violate the constraints, are shown highlighted in Tables 2a and 2c. Subsequently, the design space is narrowed down and the new levels (bounds) for the design factors are identified as follows:

x1 = 80, 10 x2 50, x3 = 0.9, 2.3 x4 5.

At this point, the designer has the option of either (1)Performing another iteration on the reduced space to further narrow down the search or (2)Identifying the non-inferior set by combining the various levels of each factor. In the case of another iteration, the reduced bounds (as for x2 and x4), or the neighborhood (as for x1 and x3) of the predicted levels can be used to formulate new design levels. Alternatively, the combination of the various levels in the reduced design space results in a design set as shown in Table 4. The graphical representation of the resulting objective functions for all the non-dominated designs is shown in Figure 2. All the designs in this set, for this particular example, are Pareto-optimal designs. This will not be true in general and the final solution set may require further processing to identify the non-dominated designs. Note that a lower cut-off value would have resulted in fewer factor levels being eliminated and consequently the final design set would have had a larger number of designs, possibly including some dominated designs.
Exno
x1
x2
x3
x4
f1
f2
f3
1
80
10
0.9
2.3
113.86
0.049213
20.41247
2
80
10
0.9
3.2
130.24
0.040216
13.59076
3
80
10
0.9
4.1
146.62
0.034253
9.955928
4
80
10
0.9
5
163
0.03002
7.719161
5
80
23.3
0.9
2.3
175.04
0.025778
0.185818
6
80
23.3
0.9
3.2
215.36
0.020007
0.487314
7
80
23.3
0.9
4.1
255.68
0.016491
0.590065
8
80
23.3
0.9
5
296
0.01413
0.629231
9
80
36.7
0.9
2.3
236.68
0.01742
1.760328
10
80
36.7
0.9
3.2
301.12
0.013282
1.447877
11
80
36.7
0.9
4.1
365.56
0.010832
1.241703
12
80
36.7
0.9
5
430
0.009215
1.09808
13
80
50
0.9
2.3
297.86
0.013179
1.858706
14
80
50
0.9
3.2
386.24
0.009959
1.45284
15
80
50
0.9
4.1
474.62
0.00808
1.20751
16
80
50
0.9
5
563
0.00685
1.044104
MIN
113.86 0.00685

Table 4 : Non-inferior design set

Figure 2 : Graph of cross section area against deflection

Conclusions

In this paper, an efficient framework for the identification of a non-dominated design set for multi-criteria optimization problems with constraints is presented. Its application to engineering design problems is discussed with the aid of an illustrative beam design problem. This work employs statistical robust design techniques. In particular, it utilizes ANOVA results to uniquely identify a Pareto-optimal design solution set based on the relative dominance and significance of the factors on the objective functions. The nature and size of this final solution set is a function of the cut-off value specified by the designer for the identification of significant factors. The determination of this value is, in general, a much simpler task than making the trade off decisions necessary for computing weights to reduce the multiple objectives to a single expression. For the purposes of optimization, these salient features make this efficient robust design method an effective solution procedure in engineering design, especially for constrained multiple objective problems.

Acknowledgement

The authors gratefully acknowledge the support of the National Science Foundation under Grant No. CMS-9402608.

References

Bagchi T. P., 1993, "Taguchi Methods Explained - Practical Steps to Robust Design," Prentice-Hall of India, New Delhi.

Box G., 1988, "Signal-to-Noise Ratios, Performance Criteria, and Transformations," Technometrics, Vol. 30, No.1 pp 1-17.

Charnes A. and Cooper W. W., 1961, "Management Models and Industrial Applications of Linear Programming, Vol. 1" Wiley, New York.

Cohon J. L., 1978, "Multiobjective Programming and Planning," Academic Press, New York.

Cohon J. L., 1985, "Multicriteria Programming: Brief Review and Application," in Design Optimization, Gero John S (Ed.), Academic Press, New York.

Dehnad K., 1989, "Quality Control, Robust Design, and the Taguchi Method," Wadsworth and Brooks/Cole, California.

Kacker R. N., 1985, "Off-Line Quality Control, Parameter Design, and the Taguchi Method," Journal of Quality Technology, Vol. 17, No. 4, pp 176-188.

Keeney R. L. and Raiffa H., 1976, "Decisions with Multiple Objectives : Preferences and Value Tradeoffs," Wiley and Sons, New York.

Kota S. and Chiou S. J., 1993, "Use of Orthogonal Arrays in Mechanism Synthesis," Mechanism and Machine Theory, Vol. 28, No. 6, pp 777-794.

Leon R. V., Shoemaker A. C., and Kacker R. N., 1987, "Performance Measures Independent of Adjustment - An Explanation and Extension of Taguchi's Signal-to-Noise Ratios," Technometrics, Vol. 29, No. 3, pp 253-265.

Majchrzak J., 1989, "A Methodological Guide to the Decision Support System DISCRET for Discrete Alternatives Problems," in Aspiration Based Decision Support Systems, Lewandowski A. and Wierzbicki A. P., (Eds.), Springer-Verlag, New York.

Osyczka A., 1985, "Multicriteria Optimization for Engineering Design," in Design Optimization, Gero John S (Ed.), Academic Press, New York.

Otto K. N. and Antonson E. K., 1991, "Extensions to the Taguchi Method of Product Design," ASME Design Theory and Methodology, DE Vol. 31.

Roy R. K., 1990, "A Primer on the Taguchi Method," Van Nostrand Reinhold, New York.

Song A. A., Mathur A., and Pattipati K. R., 1995, "Design of Process Parameters Using Robust Design Techniques and Multiple Criteria Optimization," IEEE Transactions on Systems, Man and Cybernetics, Vol. 25, No. 11, pp 1437-1446.

Thurston D. L., Carnahan J. V., and Liu T., 1991, "Optimization of Design Utility", ASME Design Theory and Methodology, DE Vol. 31.

Tribus M. and Szonyi G., 1989, "An Alternative View of the Taguchi Approach," in Quality Progress, May 1989.

Von Neumann J. and Morgenstern O., 1953, "Theory of Games and Economic Behavior," 3rd Ed., Princeton University Press, Princeton, N.J.