## Chapter 13 - Experimentation

This chapter tells you all about the experimentation features in GPSS World, and how to use them. A general overview is presented in the chapter introduction, followed by background material, then a detailed presentation of all the features relating to experimentation. We conclude the chapter with a list of practical tips that address problems you may encounter.

You should be able to get started after reading the introduction if you are willing to use the Help facilities, the Tutorial Manual, and refer to details in this manual. If you are unfamiliar with some of the nomenclature you may want to read section 13.2.2 first.

However, to get to most out of what GPSS World offers you need to become familiar with what lies in the rest of this chapter. Also helpful are several new sample models including OneWay.gps, Multiway.gps, ExperEther.gps, LatinSquare.gps, GraecoLatin.gps and Lessons 19 and 20 in the Tutorial Manual.

## 13.1 Introduction

The experimental phase of a simulation project assumes the existence of a fully developed, well-tested, GPSS World simulation that embodies the influences of each of the factors whose effects are to be measured. Only then, should you proceed with the experimental phase of your project. Conceptually, GPSS World supports three different approaches to experimentation: Screening Experiments, User Experiments, and Optimizing Experiments. There are features that support each of these.

The Experiments generated by GPSS World use partial factorial experiments with a fixed number of treatment levels for each factor. When you create your own experiments, you do not have this restriction. You are limited to 6 factors, each of which may have any number of treatment levels. Your job then is to define a one-way or an orthogonal experiment, fill a Result Matrix with the yield of each run, and pass it to the ANOVA library procedure.

## 13.1.1 Screening Experiments

A Screening Experiment is usually used to identify the most important factors affecting the simulated system. This information is crucial for directing the rest of the investigation in the most efficient way. The results of a screening experiment show which factors are not effective and should receive low priority with respect to further study. Also, the sensitivity of the yields of an experiment to one or more of the screened factors raises a flag that assumptions made in this part of the simulation should be verified carefully. GPSS World has an automatic experiment generator that can create Screening Experiments for you. To use it, you fill in a dialog that is accessed from the Edit Menu of the Main Window. This results in PLUS code being inserted into your Model Object. Optionally, this process will also load a Function Key with the appropriate CONDUCT Command for the simulation. After that, your role is to create the Simulation Object (Ctrl+Alt+S), start the experiment (normally F11), and then analyze the results.

## 13.1.2 User Experiments

User Experiments are more flexible, but you must create and run them yourself. Even so, GPSS World provides a lot of support along the way. The ultimate goal is for you to provide the data needed by the GPSS World ANOVA library procedure, which will analyze your experiments with up to 6 factors, including 2 and 3-way interactions between factors. The main requirement of GPSS World ANOVA is that you must pass it the name of the GPSS Matrix where the results of your experiment have been saved.

There are two important things to know about Result Matrices. First, before you begin your experiment you should initialize your Matrix's elements to the UNSPECIFIED state. Here's a typical Statement to do this:

INITIAL MyResultMatrix,UNSPECIFIED

This makes it clear to the ANOVA routine when a run of the experiment has not been completed.

The second thing is that if your experiment has more than one factor, you need to make it symmetrical so that the GPSS World ANOVA routine can cleanly analyze the variance. The technical term for this special symmetry is "orthogonality". All you have to do to achieve orthogonality is to look at all pairs of factors in your experiment. If, within each pair, each Treatment Level of the first Factor appears the same number of times within each Treatment Level of the second Factor, the experiment will be orthogonal. If you make everything symmetrical, it will be orthogonal. Don't worry, GPSS World will tell you when it's not.

GPSS World provides the PLUS Language that you can use to write programmable experiments. A good way to get started is to look at chapters 19 and 20 in the Tutorial Manual, or to use the automatic experiment generators in the Edit Menu to create samples for you to follow. Actually, you don't really need to use PLUS. You can set up your own Command lists in Include-files or enter them manually. All you need is some way to get the yield(s) of each run in your experiment into one or more Result Matrices that can be passed to the ANOVA library procedure.

If you do decide to write your experiment in PLUS, you should consider using the same programming style as the automatic experiment generators, which is discussed later in this chapter. The use of a Run Procedure is a case in point. A Run Procedure is a simple PLUS Procedure that is called every time a run in the experiment is to be performed. Since the Run Procedure is called by a PLUS Experiment, it is permitted to invoke the DoCommand library procedure, giving it the power to issue RMULT, START, RESET, and other Commands. It is then convenient to place all the Commands needed to set up each run into the Run Procedure. To reset variables in the Run Procedure, you should use the CLEAR ,OFF form of the CLEAR Command so that your Result Matrix is preserved. To vary the Random Number seeds, generated experiments pass a unique run number as an argument to the Run Procedure. However, you may prefer to do this in some other way. One last hint: since the command string passed to the DoCommand library procedure is eventually Translated in global scope, you must avoid using local argument and temporary variable names in it. If you need to use values from arguments or temporary variables, just build the command string dynamically before invoking DoCommand, as is done in the generated experiments. This is discussed further, below, in Section 13.3.2.

## 13.1.3 Optimizing Experiments

Optimization and the quantitative prediction of the behavior of a system are often the primary objectives of a simulation project. Both of these are supported directly by GPSS World. A Response Surface is an equation that predicts the results of a simulation. It is often desirable to establish a Response Surface for abbreviating results, providing a predictive methodology, quantifying the sensitivity of results to numerical inputs and other assumptions, and determining optimal treatment levels. GPSS World’s programmable Experiment feature provides the basis for a variety of Response Surface methodologies. A PLUS Experiment that calculates a Response Surface can be generated automatically, based on the input from a User Dialog. The resulting Experiment attempts to use the Method of Steepest Ascent and a Method of Local Exploration to find an optimum value. If successful, it reports the mathematical description of the best fit Response Surface and the predicted optimum conditions in the Journal Window of the Simulation Object.

As with Screening Experiments, the automatic generation of an Optimizing Experiment begins in the Edit Menu of the Main Window of GPSS World. When a Model Object is selected, you can click on Edit / Insert Experiment / Optimizing ... in order to open the dialog. After you fill in the blanks, and edit the Run Procedure, GPSS World inserts the finished PLUS Experiment into your model and optionally loads the appropriate CONDUCT Command into the F12 Function Key. Then, all you must do to start the experiment is to Create the Simulation Object (Ctrl+Alt+S) and press F12.

## 13.2 Experimentation and The Analysis of Variance

Next we explore some of the technical considerations that are relevant to experimentation in GPSS World. This section presents a discussion of the main issues that need to be considered when you invoke the ANOVA Procedure or when you generate an Experiment.

## 13.2.1 Motivation

Simulations are not completely faithful to the real world system they are intended to represent. Instead, they are merely representations intended to capture the most important behavioral characteristics of the target system.

One major difference with the real world is that simulations are perfectly repeatable. That is, if we repeat a simulation many times, we get precisely the same results every time. Such a thing is extremely unlikely in the natural sciences. In fact, it is so unnatural that we intentionally introduce variability into our simulations in order to make them look more realistic. In practice, we often find simulations in which the random variation in modeled processes, artificially introduced by us, is essential for capturing the behavior of the target system.

Another source of variability that does not occur in simulations is that due to the measurement itself. Whereas in the real world some measurement tools are very noisy and troublesome, in the simulated environment we enjoy a god-like perspective where all things are ultimately knowable without disturbing the measured environment. The point is that the observations from simulations are unnaturally crisp, and may occasionally reveal effects that exist in the real world but are extremely difficult to observe there.

Unlike experiments conducted in the real world, in simulation we have much better control over the variability introduced to mimic the apparent randomness of repeated real world measurements. Generally, we introduce one or more streams of "pseudo" random numbers, which are able to pass certain statistical tests of randomness. We go even further when we select probability distributions thought to accurately reflect the target system’s behavior. GPSS World provides over twenty of these, to be selected and used by the simulation analyst. In our quantitative analysis of simulation results, we will assume we have successfully modeled the random variance of measured values in the real world in this way.

The Analysis of Variance, or ANOVA, is a highly developed methodology for extracting information from the results of an experiment. In essence, it breaks up the variance of observations, and associates the pieces with the experimental factors and their interactions. One part, called the Error term, is associated with the unavoidable intrinsic randomness of the observations. In one sense, ANOVA attempts to account for all the variation of the observations from their average. That which is left over, the "Error", is a variation that has not been accounted for by association with the experimental conditions. It is an estimate of the intrinsic variability of observations called the Standard Error. Usually, only those effects that exceed the magnitude of the Standard Error are presumed to be real and not due to random fluctuations. A test using the "F Statistic" is normally used to ascertain this.

We must be careful to distinguish the variance introduced for realism from the variance encountered in our statistical models, the Standard Error. Generally, we do not want to distort the former because it will often cause our simulation to miss important real world behavior. On the other hand, the purpose of Design of Experiments is to reduce the unaccounted variance in our statistical analysis. This is quite a different matter. Variance Reduction Techniques, which distort variance introduced for realism, are to be avoided in the methodology we are presenting here. Our goal is to model real-world randomness accurately, not to reduce it.

We are, however, motivated to find ways to reduce the variability of the observations in the Analysis of Variance, as long as we do not distort the way we represent natural randomness. Reducing the estimate of the Standard Error in an Analysis of Variance allows significant effects to emerge above the statistical noise level. Two methods of doing so, which we will discuss below, improve the residual data used in the estimate of the Standard Error. The first by increasing the number of observations, the second by changing the underlying statistical model. These are discussed in Section 13.2.3.

## 13.2.2 Nomenclature

When we conduct an experiment we begin by selecting one or more metrics that quantify the state of the system or some other outcome of interest. Measurements of these quantities are called "observations" or "yields", and the set of all observations comprises the "results" of the experiment.

We will examine one or more forces called "factors" that are believed to influence the value of some of the observations. We will assign values called "treatment levels" to the factors when we specify the conditions for each execution of the simulation. When there are multiple factors, we say that the conditions are specified by "treatment combinations", since multiple treatment levels must be specified. If the influence of some factor differs when the treatment level of some other factor is varied, we say that there is an "interaction" between the two factors. Since we will be using an additive model, when the effects of two factors together is not the sum of their separate effects, we say that there is an interaction between them. There can be distinct interactions involving any number of factors.

We will simulate the target environment multiple times calling each instance a "run". The treatment combination specifies the conditions of the run, and one or more observations form the results of the run. When the conditions of a set of runs are the same except for randomization, we say that the runs in the set are "replicates", and form a "cell" of the experiment.

In the course of the Analysis of Variance, done for you by GPSS World, the observations are partitioned into components, called "effects" which are presumed to be due to the influence of the factors and their interactions. How this partition is done depends on an underlying additive model called the "statistical model". An intrinsic random deviation causes the observations to differ slightly from the sum of effects in the statistical model. The "error term" in an observation is derived by subtracting estimates of the effects corresponding to terms in the statistical model. Since the error is the quantity left over, it is often called the "residual".

For multiway experiments, that is those with more than one factor, the Analysis of Variance in GPSS World requires that the experiment be orthogonal in order to complete the analysis. This means that the estimators within the analysis must be uncorrelated. In practice, providing the same number of runs within each treatment combination of a balanced design guarantees orthogonality.

Effects, as denoted by letter groups, have important uses in the design of fractional factorial experiments, discussed below. To fractionate a factorial design, a small number of effects are chosen by the designer as "Generators" in order to specify the set of runs in the experiment. Unfortunately, this also causes some effects to become indistinguishable, or "aliased", with others. In GPSS World, the set of generators separated by equal (=) signs is called the "defining relation" of the experiment.

GPSS World supports experimentation through internal library procedures and through PLUS, the embedded Programming Language Under Simulation. PLUS is both a low-level procedural language accessible from within simulations, and a high-level control language that can direct the conditions and sequence of runs in an experiment.

## 13.2.3 ANOVA

The Analysis of Variance is a tool, pioneered by Sir Ronald Fisher, which is able to extract much of the information available in a set of measurements. In it, we quantify the variation of observations from the overall average, and then break it into pieces, each of which has a separate cause. If any experimental factor cannot be found to induce variability in a measurement, we say that it does not have a significant effect on it. On the other hand, if a factor does appear to induce variability, we compare the amount of it to an estimate of the intrinsic variability of the observation, the Standard Error. We do this to rule out apparent effects that are nothing more than random fluctuation. Our standard of comparison is that the variation from any source must be much larger than the Standard Error in order to be deemed a significant effect. The F test, named for Fisher, is used for this purpose. We use the F test as the criterion by which we declare the effects of experimental factors and their interactions to be statistically significant.

Implicit in the use of ANOVA is the existence of an additive mathematical model used to explain the components of variation in the observations. We will call this the "statistical model". The simplest statistical model is given in Figure 13-1.

yi = m + ei

Figure 13-1. A Simple Statistical Model

In this example, each observation is broken down into only two components, the grand mean of all observations, m, and the random component, e. Each observation has a starting point, the grand population mean, and then incurs a random deviation leading to its final value. The grand mean does not vary from observation to observation, whereas the random component does, and is subscripted accordingly. Although this model is suggestive, it allots all variation of the observations to random sources and none to factors, and so is not very useful in analyzing the results of an experiment.

Next, we turn to a statistical model used to analyze the data from an experiment with a single factor, namely, factor A.

yi,j = m + ai + ei,j

Figure 13-2. One Factor Statistical ModelIn Figure 13-2, notice the introduction of the subscripted alpha term, which denotes the effect of the ith treatment level of the one and only factor considered in the model. The experiment would include one or more runs at each of the treatment levels of factor A. All observations at a given treatment level are analyzed using the same value for a. Since there is only one factor in this experiment, the number of treatment combinations is just the number of treatment levels of that factor. An Analysis of Variance based on this statistical model will result in an ANOVA table that partitions the variation of the observations into that due to treatment A, and that due to the random variation.

yi,j,k = m + ai + bj + i(ab)i,j + ei,j,k

Figure 13-3. Two Factor Statistical ModelIn Figure 13-3, we move to a model where two factors are considered. Notice that we now include a term for the interaction between factor A and factor B, denoted i(ab). This term may differ at each treatment combination, and therefore is subscripted twice since there are two factors. Recall from above that when the effects of two factors together is not the sum of their separate effects, there is an interaction between them. The strength of the interaction between factors A and B is denoted "AB" and would be reported in a line of the ANOVA table.

In a complete factorial experiment all interaction terms would be included. Just as the complete 2-factor model in Figure 13-3 has 5 (RHS) terms, a complete 3-factor model has 8 terms, and a complete 4-factor model has 16 terms. All this information is normally presented in the resulting ANOVA table. GPSS World can handle up to 6-factor models with up to and including 3-way interactions.

Figure 13-4, below, presents an ANOVA table reported by GPSS World. As explained above, the single capital letters denote factors and capital letter combinations denote interactions.

First, look at the bottom of the table. The Total Sum of Squares is to be partitioned, and the components are to be associated with the effects of factors and their interactions. Anything that is left over, that is, the residual sum of squares, is shown in the previous line, labeled "Error". The Mean Sum of Squares of the error term is used to estimate the Standard Error of the experiment.

Each Sum of Squares has a divisor associated with it called the Degrees of Freedom. From statistical considerations, the Degrees of Freedom is that divisor that must be used to create an unbiased estimator of the Standard Error, in the absence of other effects. For our purposes, it is sufficient to think of Degrees of Freedom as the proper divisor associated with a Sum of Squares in the ANOVA table. GPSS World will always calculate the Degrees of Freedom for you.

Figure 13-4. An ANOVA Table

Each factor and interaction in the statistical model is represented by a distinct line in the ANOVA table. In each line we have first the Sum of Squares and the Degrees of Freedom associated with that estimate. These are the basics from which the other numbers are derived. A simple division results in the Mean Square, and by dividing that quotient by the Mean Square of the Error, from the bottom row of the table, we get the F statistic for that effect.

Now we are ready to draw some conclusions. We must decide if the F value is large enough to declare that the effect is significant. The threshold value we will use to make the comparison is called the "Critical Value of F" and is placed just to the right of our F statistic on the same line. If our F value exceeds the Critical Value, we conclude that we are dealing with a significant effect. If not, we conclude that the effect is not significant and we disregard any associated variation in observations as only due to random noise. The larger the F value the stronger the effect. The ANOVA table in Figure 13-4 shows the effect of factor A to be significant and the effects from factor B and the AB interaction to be not significant.

Sometimes an experiment will fail to detect an effect even though one actually exists. One of our goals is to make this as unlikely as possible. According to the ANOVA table, there are two ways to make an experiment better able to detect real effects. To get more positive results we would need either a larger F statistic or a smaller Critical Value of F.

It is desirable to remove any part of the Sum of Squares of the Error that is due to any important effect we have not included in the analysis. If we can do this, our F statistic will generally be larger. In experimentation in the natural sciences, this is done by making comparisons in as homogenous an environment as possible, a technique known as "blocking". However, in simulation studies this goal is perhaps best approached by identifying additional factors that must be included in the experiment.

Two other approaches are directed at increasing the Degrees of Freedom of the Error term. The first is simply to increase the number of replicates in the experiment. This is usually the most expensive approach but it can be quite effective. The second possibility relates to the design of the experiment and the statistical model behind the Analysis of Variance. The Mean Square of the Error is actually a residual term left after all other squares have been removed. If we can find an acceptable way to allow more of the data to remain after the effects are removed, we will have an estimate of the Standard Error that has more degrees of freedom. The resulting Critical Value of F will be smaller, thereby increasing the power of the analysis. This is what we are attempting to do when we choose to ignore some of the interactions.

None of the techniques we are considering will distort the randomness introduced to simulate real-world randomness. Variance Reduction Techniques that do should not be used here. Intentionally reducing the intrinsic variability of the observations would cause the F statistics to be overstated.

In many multifactor experiments we will chose to remove the highest order interactions in order to improve the information available on the main factors and the low order interactions. You can do this using the third argument of the ANOVA Procedure. If it is true that these interaction effects do not exist, this will allow GPSS World to use the extra degrees of freedom to achieve a better estimate of the F statistic. In addition, more Degrees of Freedom means a smaller critical value of F, as well. However, we must realize that removing terms from the statistical model assumes that there is no high-order effect. If there is a reasonable chance of one, we would be better off adding replicates to improve the statistics rather than limiting the analysis to low order interactions. Replicates are made known to the ANOVA Procedure in its second argument.

yi,j,k = m + ai + bj + ei,j,k

Figure 13-5. Two Factor Statistical Model without Interactions

Figure 13-5 shows a statistical model in which the 2-way interaction term was removed. If it is true that the interaction effect does not exist, we have improved the crispness of the Analysis of Variance by providing more Degrees of Freedom for the Mean Square of the Error. When you choose to apply this technique, you simply change the 3rd argument of the call to the GPSS World ANOVA library procedure to remove higher order interactions from the analysis. This built-in procedure can handle up to 3-way interactions, and allows you to give up 2-way or 3-way interactions, and above, to improve the significance testing of the other effects.

## 13.2.4 Selecting Factors

When you use the ANOVA library procedure you must define a GPSS Matrix Entity, called the Result Matrix, to store the individual results of each run. If you were analyzing more than one metric, you would have more than one Result Matrix. Since GPSS World Matrix Entities can have up to 6 dimensions, each of any size, the Result Matrix is also limited to 6 dimensions.

When you write your own GPSS World Experiment, the most important decision you will make is to select the factors of the experiment. These are the quantities that you can control in order to optimize your system. Each factor is to be represented by a User Variable that takes on treatment levels as values. After you run each simulation in the experiment, you must place the resulting value into a Results Matrix so that the data can be analyzed by the ANOVA Procedure.

The position in the Result Matrix for each result is determined by the treatment combination. For example, if the experiment considers 4 machine types and two speeds of each, the result of simulating the third machine at high speed would go into the Result Matrix at position [3,2]. Actually, since replicates are usually tagged onto the end, the first run at this treatment combination would go into element [3,2,1] of the Result Matrix. Each dimension of the Result Matrix is analyzed as a factor by the ANOVA library procedure. The size of the dimension in the Result Matrix must be at least as large as the number the treatment levels of that factor, or of the maximum replicate count.. There is no arbitrary limit to the number of treatment levels within a factor, only that imposed by the virtual memory of your computer.

We can examine any effect by giving it it’s own dimension in the Result Matrix. If we so desire, we could even separate random number streams and treat them individually as distinct factors. Each random number seed would represent a distinct treatment level. This would provide information on the relative importance of each random input process. Usually, though, we use a single dimension for replicates, and vary all seeds from one replicate to the next within a single treatment combination. We then make known to the ANOVA Procedure which dimension is used for replicates. This is done in the second argument.

Not all experiments need a Replicate Dimension. The ANOVA is often able to extract an estimate of the Standard Error without one. The Latin Square sample model, "Latin Square.gps" is an example of this. However, depending on the design of the experiment, sometimes there is not enough residual data to estimate the Standard Error. Then the Analysis of Variance will not be able to calculate F statistics for the ANOVA Table. In such cases, you will have to either add replicate runs to the experiment or remove high level interactions from the statistical model in order to provide enough residual data for the estimate of the Standard Error. Both of these methods require a change in the call to the ANOVA library procedure.

## 13.2.5 The Random Number Streams

A guideline often used in the design of experiments is this: control all factors that you can; what you can’t control, block; what you can’t block, randomize. This applies to the use of the random streams in a discrete event simulation, as well.

To treat random number streams as factors is to allocate a dimension for each stream, and to use each distinct seed as a treatment level. Although it is possible to view the random number streams as factors in this way, and to examine their relative effects, it may be best done as a separate study. For the mainstream investigation of the main effects and interactions, we are more interested in using our random number streams to represent sources of natural and unavoidable variability, reserving the dimensions of the Result Matrix for the factors being studied.

The most common approach combines all random number streams into a single pseudo-factor, where each treatment level is a set of distinct seeds for all random number streams. A single dimension, the Replicate Dimension, is then allocated from the Result Matrix for this purpose and is made known to the ANOVA library procedure. This has some similarities with blocking, where all comparisons are performed within each homogeneous environment. The ANOVA library procedure can then treat this Replicate Dimension separately to better estimate the Standard Error of the experiment. The experiment then consists of an experimental "cell" for each treatment combination, with several replicate runs within each cell.

If these approaches are too expensive, or there isn’t a dimension to spare in the Result Matrix, the runs should be "completely randomized". This means that randomization should not be restricted in any way, and that the random number seeds should not be repeated within a single random number stream. If this procedure fails to provide enough Degrees of Freedom for the analysis, you may need to limit the level of interactions to be included in the statistical model, as well.

## 13.3 GPSS World Features

We turn now to the to the specific features of GPSS World which can be used in any of the experimentation phases of a simulation project. First, we consider the features provided for the analysis of User Experiments. GPSS World provides for two kinds of automatically generated experiments that are designed for you, and User Experiments that are designed by you.

## 13.3.1 User Experiments

The immediate goal of a User Experiment is to provide the information needed by the built-in ANOVA library procedure. That means that you must fill a Result Matrix with all the results of the experiment. If you prefer, you can do this directly through the use of Command Lists and manual entries. However, the PLUS Language, which allows you to write programmable experiments can make your life a lot easier if your experiment is complex.

You should begin by selecting the factors whose effects you will be studying. For example, you may want to measure the effect of the number of phone lines on the average waiting time in a telephone exchange. Factor A of your experiment would be the number of phone lines, and you would assign it to the first dimension of your Result Matrix. Next we decide how many treatment levels (i.e. number of phone lines) will be studied. Let's say that we want to simulate exchanges with 4, 8, and 16 lines. We would then need 3 slots in the Result Matrix for these three conditions. Therefore the first dimension in the Result Matrix would have a size of 3.

We continue to add factors to the experiment in this way, using the number of treatment levels as the size of the corresponding dimension in the Result Matrix. Normally, we use the last dimension for replicates. The size of the Replicate Dimension must obviously be large enough to contain the results from any cell (i.e. treatment combination) in the experiment. The only restriction is that the total number of dimensions cannot exceed 6. When you analyze the data, you must explicitly tell the ANOVA routine which dimension is used for replicates.

Remember to initialize the Result Matrix to UNSPECIFIED, before you begin storing results into it. That way, it is clear to the ANOVA routine when a run has not been performed.

When you call the ANOVA routine with your results, you must also decide if you want to exclude 3-way, or even 2-way, interactions from the analysis. If you can justifiably do this, you will improve the statistics in the ANOVA table.

That is a rough outline of the creation of User Experiments. In the remainder of this section we will look at the steps in more detail. We will consider the embedded PLUS programming language, the requirements and restrictions of the Result Matrix, and the use of the ANOVA library procedure. Lesson 19 of the GPSS World Tutorial Manual guides you through an example.

Experiments created using the automatic experiment generators are not considered User Experiments, and are discussed later. However, you may find it useful to study the programming of a generated experiment before you attempt to do it yourself.

## 13.3.2 PLUS Experiments

PLUS is a simple but powerful programming language that is an important part of the GPSS World simulation environment. It began as a source of custom programmable subroutines to be accessed during GPSS simulations, and has progressed to become a control language that can direct the details of the runs in an experiment, and report on the results. The complete description of PLUS is in Chapter 8 of this manual. Lesson 17 of the GPSS World Tutorial Manual covers it, as well.

An Experiment is a PLUS Procedure where the keyword PROCEDURE is replaced by the keyword EXPERIMENT. Experiments can do everything that normal Procedures can, and a whole lot more. An Experiment is invoked by a CONDUCT Command which names the invoked Experiment and passes it any arguments it might require.

The power of a PLUS Experiment is based on its ability to invoke the "DoCommand" library procedure. Nearly all GPSS Commands can be executed by a DoCommand invocation with a Command String as an argument. Not only can an Experiment invoke DoCommand, but also so can any normal PLUS Procedure that has been called during the execution of an Experiment. This adds a lot of flexibility to the programming of complex experiments.

An example of a DoCommand invocation is:

DoCommand("START 1000");

The DoCommand Library Function is peculiar in that its argument string is compiled in global scope. This means that you cannot reference local variables in an argument of a DoCommand invocation. Instead, if you need to pass local values to DoCommand, you must use PLUS string commands to create an argument string including evaluated results. As an example of this, here's the "Run Execution Procedure" created by the Automatic Experiment Generators to log run information to the Journal:

*******************************************************

* The Run Execution Procedure *

*******************************************************

PROCEDURE SEM_GetResult() BEGIN

/* Run Simulation and Log Results. */

/* Treatments have already been set for this run. */

TEMPORARY CurrentYield,ShowString,CommandString;

/* Run Procedure Call */

RunProc(SEM_NextRunNumber);

/* Place Run Results in Journal. */

ShowString = PolyCatenate("Run ",String(SEM_NextRunNumber),". ", "" );

ShowString = PolyCatenate(ShowString," Yield=",String(CurrentYield),". ");

ShowString = PolyCatenate(ShowString," Factor1=",String(Factor1), ";" );

ShowString = PolyCatenate(ShowString," Factor2=",String(Factor2), ";" );

ShowString = PolyCatenate(ShowString," Factor3=",String(Factor3), ";" );

CommandString = PolyCatenate("SHOW """,ShowString,"""", "" );

DoCommand(CommandString);

SEM_NextRunNumber = SEM_NextRunNumber + 1;

RETURN CurrentYield;

END;

Figure 13-6. Using a DoCommand in a PLUS Procedure

This procedure issues a SHOW Command in 3 steps. It records the result and conditions of the run in the Journal. First it creates a temporary string, ShowString, which contains variables evaluated locally. Second, it builds the overall string with the completed SHOW Command in the temporary variable CommandString. Finally, it invokes the DoCommand library procedure. Normal Procedures such as this can only invoke DoCommand when a CONDUCT Command is in effect. In other words, such Procedures must be called either directly, or indirectly, by a PLUS Experiment.

In experiments with many runs, it is convenient to use a PLUS Procedure to do the set up of each simulation run in the experiment. Such a procedure initializes the random number generators and issues commands that control the simulation. Then, you can set up your high level PLUS Experiment to call the Run Procedure each time a simulation is to be run.

An example of the default Run Procedure created by the Experiment Generators is in Section 13.4. 2. Even when GPSS World Creates the Experiment for you, it is up to you to modify the Run Procedure to fit the run conditions of your own simulation project.

## 13.3.3 The Result Matrix

The primary source of information from an experiment is kept in the Result Matrix. In order for the ANOVA library procedure to analyze the results of an Experiment, you must assure that the Result Matrix has been set up appropriately.

In GPSS World, a Matrix Entity can have up to 6 dimensions. There is no arbitrary limit on the size of each dimension other than that imposed by the virtual memory of the computer. However, one of the most important jobs in setting up a GPSS World Experiment is the selection of factors. Each factor to be included in the Experiment is assigned to one of the dimensions of the Result Matrix. In addition, depending on the design of the experiment, you will probably want to use one dimension for replicates, which must be large enough to handle the runs in the largest cell. For each dimension used for a factor, the size at least as large as the number the treatment levels of that factor. That is because there must be a place in the Result Matrix for each run. There can be any number of these treatments up to the limits imposed by the virtual memory of your computer.

If your experiment has more than one result metric, you need only define a separate Result Matrix for each one, and then utilize the ANOVA Procedure for each.

Viewing a Result Matrix

GPSS World Matrices can be viewed in a Matrix Window or printed in a Standard Report. Since GPSS World is unable to view all 6 dimensions at one time, it presents only a two dimensional slice in a Matrix Window. A small Dialogue Box is used to select the slice of the Matrix that is to appear.

Figure 13-7. The Matrix Cross Section Selection Dialog

Figure 7 presents the dialog that allows the selection of a slice through a higher order Matrix Entity. In this case we have a 3 by 4 by 2 3-dimensional Result Matrix. To use it, first select the two dimensions to form the Rows and Columns of the Matrix Window, then use the index of the remaining dimensions to select deeper slices, other than the first. It may be helpful to think of a loaf of bread seen head-on. The row and column numbers indicate where on the slice we are, and the index of the third dimension chooses the slice. In Matrices with 4 or more dimensions, an index value would be needed for each dimension not participating as a row or column.

Once the Matrix Window, itself, is open you will have the chance to scroll to any location within that slice. Matrix Windows are updated dynamically, and any number of Matrix Windows can be open at the same time.

The other way to view the elements of a Matrix Entity is to include the results in the Standard Report. Page 2 of the Settings of a Model or Simulation Object ( Edit / Settings ) has a checkbox which when checked, puts a line for each element of the Matrix in subsequent Standard Reports.

Initialization of a Result Matrix

Variables within GPSS World can take on a variety of data types without user intervention. Behind the scenes, values are converted from one form to another as needed. For example, a string is converted to a real number if the operator using that value requires a real argument.

Variables and Matrix elements can take on the value UNSPECIFIED to indicate that no value has yet been assigned to it. UNSPECIFIED is not a string and will cause an Error Stop if an operator encounters it while evaluating an expression.

For compatibility reasons, when GPSS Matrices are created or CLEARed, they are given the value of 0. However, a Result Matrix should have all elements initialized to the UNSPECIFIED state before the Experiment is executed. In this way, the ANOVA library procedure can detect when runs are missing. Otherwise, a 0 value is taken to be the result of a run of the experiment. The INITIAL Command now supports the initialization of a Matrix to UNSPECIFIED.

INITIAL

MatrixName,UNSPECIFIEDThis Command causes all elements in the matrix to be set to the UNSPECIFIED state. Normally, you should place the initialization statement outside the Experiment, itself. This is what GPSS World does when it generates a Screening Experiment.

In addition to showing which data is missing, the use of the UNSPECIFIED state in the Results Matrix allows an Experiment to be saved and restarted without repeating runs that have already been completed. To be restartable, the Experiment should test the destination element for UNSPECIFIED before calling the Run Procedure. Automatically generated Screening Experiments use this method.

Here's another example of the creation and initialization of a Result Matrix.

MyResults MATRIX ,3,5,4,3

INITIAL MyResults,UNSPECIFIED

Figure 13-8. GPSS Statements Initializing a Result Matrix

In this example a Result Matrix is created which can hold the results of an experiment with 4 factors, or one with 3 factors with replicates. Factor A can have up to 3 treatment levels, factor B can have up to 5, and so on,

The INITIAL Statement sets all elements in the Matrix to "UNSPECIFIED". This allows the ANOVA routine to detect elements in the matrix where no results are available.

## 13.3.4 ANOVA Library Procedure

The final step in a User Experiment is usually the analysis of results. By calling GPSS World’s ANOVA library procedure, most of the work is done for you. The ANOVA Procedure can handle a multiway Analysis of Variance considering up to 6 factors and up to 3-way interactions of all combinations of main factors.

There are 3 arguments to the ANOVA Procedure. The first is the name of the Result Matrix, which is described in more detail, below.

The second argument is an optional dimension of the Result Matrix to be used for replicates. Each level in this dimension represents a run with distinct random number seeds. The ANOVA Procedure uses all information associated with the replicate dimension as part of the estimate of the Standard Error. This provides an estimate with more information and more Degrees of Freedom than had a replicate dimension not been used.

Some experimental designs do not use a replicate dimension. In that case, use a 0 as the second argument.

The third argument controls the factor interactions to be included in the statistical model. If the argument is 2, only 2-way interactions are included in the analysis. If the argument is 1, no interaction terms are included. A value of 3 or greater causes main effects, 2-way interactions, and 3-way interactions to be included in the statistical model. Four way interactions are not supported by GPSS World. The reason you would want to limit the interaction terms in the statistical model is this: when interaction terms are removed from the model, additional information and degrees of freedom become available for a better estimate of the Standard Error.

The main goal of the ANOVA Procedure is to create a standard ANOVA Table in the Journal, which indicates the resulting F statistic and its critical value. In addition, when you call the ANOVA Procedure from a PLUS routine, the Standard Error is returned when the ANOVA Procedure completes.

The ANOVA Table

Here we have an ANOVA table generated automatically by the library procedure in GPSS World.

Figure 13-9. An ANOVA Table

Each factor and interaction in the statistical model is represented by a distinct line in the ANOVA table. In each line we have the basic calculations and the F statistic for that effect.

The larger the F value the stronger the effect. It’s considered significant if it exceeds what is called "the critical value of F", which is calculated for you by GPSS World. The ANOVA table above shows the effect of factor A to be significant and the effects from factor B and the AB interaction to be not significant.

It’s important to avoid Variance Reduction Techniques when using Analysis of Variance. They distort the variability introduced to simulate real-world randomness. Reducing the intrinsic variability of the observations causes the F statistics to be overstated.

When you do an Analysis of Variance, you should investigate the assumptions behind the analysis, such as normality and unchanging variance, which statisticians call ‘homoscedasticity’. GPSS World provides another feature for this purpose. If you create a GPSS Table Entity with a specific name, the ANOVA Procedure will fill it with the residuals of the Analysis of Variance. This is discussed in the next section.

The cell statistics report follows the ANOVA table, and it gives ranges and 95% Confidence Intervals for each Treatment Combination in the experiment, based on a pooled estimate of the Standard Error.

Although the Confidence Intervals associated with treatment combinations are not independent, the ranges can still be used as a check on the assumptions behind the ANOVA. When these vary wildly the homogeneity of variance in the residuals is in doubt. In that case, you may need to examine the data more closely before drawing your final conclusions. One way is to define a GPSS Table to collect the residuals calculated in the Analysis of Variance. The next section tells you how to do this.

## 13.3.5 The Table of Residuals

When the ANOVA Procedure performs its analysis, it searches for a GPSS Table Entity of a special name. It looks for a Table Name the same as the name of the Result Matrix except that the string "_Residuals" is appended. If it finds a Table Entity of the derived name, the ANOVA Procedure tabulates the residuals of the analysis for the purpose of checking the assumptions used by ANOVA. For example, if you have a GPSS Table entity named MyResults_Residuals defined when you call ANOVA with a Result Matrix named MyResults, GPSS World will clear the Table and fill it with the residuals from the Analysis of Variance.

Skew and outliers can sometimes be detected in the Table of Residuals, and the lack of homogeneous variance (heteroscedasticity), when it occurs, is often apparent here and in the in the descriptive statistics report which follows the ANOVA Table.

## 13.4 The Automatic Experiment Generators

GPSS World can generate either screening or optimization experiments for you. You can use them to rule out irrelevant factors or to find the best treatment combination. The process inserts a PLUS Experiment into your Model Object and optionally loads a Function Key with the appropriate CONDUCT Command. It takes just 4 easy steps:

1. Fill out the dialog (Edit Menu) and Click OK.

2. Edit the Run Procedure and Click OK.

3. Translate the model (Ctrl+Alt+S).

4. Press the Function Key.

That's all there is to it. The PLUS Experiment will run, reporting its status to the Journal. When it completes, it will create a full report, as well. In the case of an Optimizing Experiment, the report will include a mathematical description of the Response Surface around the optimum. Then, analyzing the Report is up to you.

Both types of generated PLUS Experiment contain invocation of a PLUS Run Procedure. It is your responsibility to ensure that this procedure sets up the run conditions appropriately. The Run Procedure is called each time the Experiment begins a new simulation run. Optionally, you can have the Experiment Generator create a template Run Procedure for you. Even so, you must still adapt it to your own conditions. The next section tells you how.

## 13.4.1 The Run Procedure

The Run Procedure is the flexible connection between the generated experiment and the user’s simulation. Both Screening Experiments and Optimizing Experiments use them. Generated Experiments repeatedly call the Run Procedure that the user names in the creation dialog in order to execute each run in the experiment.

Every experiment should have a Run Procedure written by you or generated by GPSS World. A Run Procedure is used to establish the measurement period during the simulation that is to be observed. It sets up each run in the Experiment according to the particular requirements of your simulation and is invoked once for each run in the experiment. Run Procedures are PLUS Procedures that are called by Experiments and therefore may contain DoCommand calls.

Normally the Run Procedure contains setup commands for the simulation such as RMULT Commands to control the Random Number Generators, and START and RESET Commands to set up the measurement period of the run.

Here are the most important things you should take care of in the Run Procedure:

· Use CLEAR OFF to remove old transactions between replication but still preserve the Results Matrix.

· If the Experiment does not set the treatment levels elsewhere, do it here based on arguments of the Run Procedure. This is not necessary for generated experiments. They set the treatment levels for you.

· Use RMULT to set the Random Number Generators according to an argument in the run procedure. If you have not defined a replicate dimension in the Result Matrix, you should "completely randomize" and use distinct random number stream seeds for each run.

· Use PLUS String Procedures to create the DoCommand argument.

· Use START to begin the start-up period.

· Use RESET to reset the statistics accumulators prior to the measurement

period.

· Use START to begin the measurement period.

· Unless the Experiment stores results in the Result Matrix using a PLUS assignment statement, do it here based on arguments of the Run Procedure. This is not necessary for generated experiments.

When an experiment is generated automatically, the generator optionally creates a template Run Procedure which can be modified as desired. As an example, here is the default Run Procedure created by the Experiment Generators:

PROCEDURE RunProc(Run_Number) BEGIN

DoCommand("CLEAR OFF"); /* Must use OFF to preserve results. */

/* EXPAND THIS RMULT IF YOU HAVE MORE RNGs. */

/* All Random Number Streams must have new seeds. */

TEMPORARY CommandString;

/* Evaluate before passing to DoCommand. */

CommandString = Catenate("RMULT ",Run_Number#111);

/* DoCommand compiles the string in Global Context. */

DoCommand(CommandString);

/* SET UP YOUR OWN RUN CONDITIONS. */

DoCommand("START 100,NP"); /* Get past the Startup Period. */

DoCommand("RESET"); /* Begin the Measurement Period. */

DoCommand("START 1000,NP"); /* Run the Simulation. */

END;

Figure 13-10. The Default Run Procedure

When you let GPSS World create a Run Procedure, it is up to you to customize it to fit your own situation. You can edit it before, or after, it is inserted into your Model Object.

## 13.4.2 Screening Experiments

Screening Experiments are used to look for factors which must be investigated more thoroughly or those which should be considered to be irrelevant. However, the combinatorial nature of measuring main effects and all the possible interactions often makes the complete set of runs, called a full factorial experiment, too lengthy and/or expensive to run. To make the procedure more manageable, we restrict the number of treatment levels to two per factor, and we look for ways to run only a fraction of the possible runs without sacrificing the most important information. The Screening Experiment Generator of GPSS World does most of the work for you.

You must decide which information you can give up in order to reduce the number of runs in the experiment. Normally it would be the higher-level factor interactions that are usually insignificant and can be ignored. Even if you suspect otherwise, remember that the purpose of the screening experiment is to identify the important factors to be studied in more depth later. When you ask GPSS World to reduce a full factorial screening experiment to a fractional set of runs, some effects get mixed with others to the point where you can’t distinguish them in the Analysis of Variance. GPSS World will partition the set of all possible effects subsets called Alias Groups. Only the combined result of all effects in the group is reported in the Analysis of Variance. This means that if factor A and factor B are in an Alias Group, you will have lost any ability to tell which is responsible for the observed effect. Perhaps even worse than that, it is possible for two strong effects in the same Alias Group to cancel each other, thereby concealing the actual effects of each factor.

As you can see, it is extremely undesirable to have more than one main effect in an Alias Group. However, it is usually possible to arrange Alias Groups so that each main effect and the most important 2-way interactions are in Alias Groups of their own, with only higher order interactions as companions. GPSS World provides default Alias Groups which isolate main effects, if possible, and it makes it easy for you to change the Alias Groups if the defaults are not suitable. In any case, when you generate a GPSS World Screening Experiment, you should always examine the Alias Groups before you begin the runs. If you haven’t separated the most important effects into distinct groups, the interpretation of results will be difficult.

Generating A PLUS Screening Experiment

The first step in creating a Screening Experiment automatically is to open the dialog. You access it by clicking "Insert Experiment" in the Edit Menu of the Main Window, and clicking on the "Screening Experiment ... " item. You must have an Open Model Object active in order to be able to do this.

Now just fill in the blanks and click OK when you're done. Figure 13-11 shows the Screening Experiment Dialog.

Figure 13-11. The Screening Experiment Generator Dialog

To specify the PLUS Experiment that is about to be generated, we simply fill in the fields of the Dialog. The "Experiment Name" and the "‘Run Procedure’ Name" fields are used as the PROCEDURE names in the generated experiments. The EXPERIMENT calls the Run Procedure repeatedly.

The Factors of the Experiment come next. Each factor name is actually the name of a User Variable and must obey the GPSS World naming conventions. It must begin with an alphabetic character and it must not clash with a keyword, SNA, or SNA class. Since screening experiments in GPSS World are full or fractional 2k experiments, there are two treatment levels to be specified for each factor. You must specify the names and two treatment levels for between 1 and 6 factors, inclusively. Factors must be specified consecutively starting with the A factor. The choice of treatment levels is crucial. You may want to do some preliminary experimentation so that the choices you make are good ones. Choose levels that are far apart to elicit a change of behavior and try to avoid levels where the effects are hidden by other factors.

The "Fraction" group is next. It allows us to specify what faction of the full 2k experiment is to be run. The run count that will result is specified to the right. Choose a smaller fraction to decrease the run count.

The "Result Expression" is required. You must specify an expression to be evaluated as the metric of the simulation.

Next we have two checkboxes that allow us to select additional options. First, you can have GPSS World generate a template Run Procedure to go along with the generated experiment. You can edit this Run Procedure to suit your needs. Second, you can have GPSS World set up a setting which loads the appropriate CONDUCT Command into the setting for Function Key 11. If you choose this, after you create the Simulation Object, you only have to press the F11 Function Key to run the experiment.

Before we generate the experiment we should examine the Alias Groups. To do so, click the Alias Groups button to examine the consequences. This brings up the Alias Groups Dialog.

Figure 13-12. The Alias Group Dialog. Improved Experimental Design.

From the Alias Groups Dialog we can see that this experimental design isolates all the main factors, does not confound them with 2-way interactions, and it even isolates the 2-way interactions of the first few main effects.

Take a closer look the effects used as "Generators" in the Alias Group Dialog. We can change the partition of the effects into a different set of Alias Groups if we so desire by simply using a different set high level interactions as generators. To do this, change the generators and click on the Generate Button. Before you do this there are several things to know about the generators. First, you should know that when GPSS World creates Alias Groups it first expands the set of generators by forming all possible modulo 2 products and adding them to the set of generators, now called the "Expanded Set" of generators. For example if ABCD and BCDE are generators, their product, AE, will be in the Expanded Set of generators (ABCDE * BCDE = ABBCCDDE = A (BB) (CC) (DD) E = AE ). Note that any effect in the Expanded Set of generators will not appear in the report of effects. This means that you must not choose generators that result in an important effect occurring in the Expanded Set of generators. You can return to the default generators suggested by GPSS World by clicking on the Use Default Button. Click the Generate Button to create the new list of Alias Groups.

The next step is to insert the PLUS code comprising the experiment into your Model Object. When you have made the best compromise between the number of runs and the aliasing of effects, click the OK Button in the Alias Group Dialog and the Insert Experiment Button in the Screening Experiment Generator Dialog. If you have asked GPSS World to generate a Run Procedure, next we see a Dialog that allows you to personalize it. The details about personalizing the Run Procedure are in the previous section.

In any case, you may want to edit the generated experiment itself in your Model Window. You can change the Run Procedure there, as well. Just click OK and all the new GPSS World Statements comprising the Screening Experiment are placed at the bottom of your Model. A sample of a generated PLUS Experiment follows in the next section.

The Generated Experiment

*******************************************************

*

* ScreenEthernet

* Fractional Factorial Screening Experiment

*

*******************************************************

Global_Delays MATRIX ,2,2,2,2,2

INITIAL Global_Delays,UNSPECIFIED

EXPERIMENT ScreenEthernet() BEGIN

/* Run 1 */

Node_Count = 100;

Min_Msg = 512;

Max_Msg = 12144;

Fraction_Short_Msgs = 600;

Intermessage_Time = 1.0;

IF (StringCompare(DataType(Global_Delays[1,1,1,1,1]),"UNSPECIFIED")’E’0)

THEN BEGIN

/* Run Procedure Call */

DoTheRun(1);

Global_Delays[1,1,1,1,1] = QT$Global_Delays;

END;

/*********** Runs 2-15 would go here ************/

/* Run 16 */

Node_Count = 200;

Min_Msg = 1024;

Max_Msg = 24288;

Fraction_Short_Msgs = 300;

Intermessage_Time = 1.0;

IF (StringCompare(DataType(Global_Delays[2,2,2,2,1]),"UNSPECIFIED")’E’0)

THEN BEGIN

/* Run Procedure Call */

DoTheRun(16);

Global_Delays[2,2,2,2,1] = QT$Global_Delays;

END;

/* Aliased Effects in Fractional Factorial Experiment */

EFFECTS(Global_Delays,"I=ABCDE");

END;

*******************************************************

* Run Procedure *

*******************************************************

PROCEDURE DoTheRun(Run_Number) BEGIN

DoCommand("CLEAR OFF"); /* Must use OFF to preserve results. */

/* EXPAND THIS RMULT IF YOU HAVE MORE RNGs. */

/* All Random Number Streams must have new seeds. */

TEMPORARY CommandString;

/* Evaluate before passing to DoCommand. */

CommandString = Catenate("RMULT ",Run_Number#111);

/* DoCommand compiles the string in Global Context. */

DoCommand(CommandString);

/* SET UP YOUR OWN RUN CONDITIONS. */

DoCommand("START 100,NP"); /* Get past the Startup Period. */

DoCommand("RESET"); /* Begin the Measurement Period. */

DoCommand("START 1000,NP"); /* Run the Simulation. */

END;

*******************************************************

Running the Experiment

CONDUCT is the GPSS World Command that is used to invoke an Experiment. The syntax is similar to that of a Procedure Call. But we have a better way. In the Experiment Generation dialog, we can choose to have GPSS World load the F11 Function Key with the appropriate CONDUCT Command. You can see this in the Model Object's Settings. (see Edit / Settings / Function Keys).

To start the execution of the experiment, we must do two things. First we must Translate the Model, thereby creating the Simulation Object. Click on Command / Create Simulation to do this, or just press Ctrl+Alt+S.

Then we press the F11 Function Key. This enters the CONDUCT Command. The Experiment goes to work, reporting status and output to the Journal Window of the newly created Simulation Object. Figure 13-13 shows the results of a Screening Experiment.

Figure 13-13. Results of Screening Experiment.

The Screening Experiment Report shows the strength of each effect and, if it can, calculates an F statistic for those alias groups with main effects as members. From it you can plan future actions such as ignoring insignificant factors, and more fully exploring the important ones.

## 13.4.3 Optimizing Experiments and Response Surfaces

Optimization and the quantitative prediction of the behavior of a system are often the primary objectives of a simulation project. Both of these are supported directly by GPSS World's Optimizing Experiment Generator.

Response Surface Methodology is a set of statistical techniques for empirical model building and model exploration. Although most often used to estimate optimum treatment levels, it also can provide important information for general predictions of the yield expected under untested conditions. GPSS World supports several important Response Surface Methodologies in a way intended to make them easy to use. The simulation analyst sets up the initial conditions in a few User Dialogues and GPSS World then automatically creates a response surface experiment that seeks an optimum value. Ideally, you can start the experiment and let it run unattended until it finds the optimum value of the chosen metric. When you fill in the dialog windows you are, in effect, defining a box (actually a "hyperbox") called the Local Experimental Region that wanders over the set of permissible factor levels. If you make this box too small, the experiment will require more runs than necessary to move to the optimum. If you make it too large, the estimates of the optimum treatment levels will be less precise than they could be.

GPSS World applies several different methodologies as it explores the response surface of a GPSS World Simulation Object. The primary concept is that of a Local Experimental Region which is defined by you in the setup dialog. To define it, each of up to 5 factors to be studied are associated with two values defining the extent of the Local Experimental Region. Then, it is the whole Region that moves around the factor space. The goal, which is not always achieved, is to contain the optimum within the Local Region, perform a confirmation run, and then to report the optimum conditions and the mathematical model that was used.

GPSS World uses several experiments of increasing complexity to get the information it needs. As results are available, it attempts to fit either a linear model or a second order model (including two-way interactions) to the data. It begins by using the Method of Steepest Ascent as it leaves its starting point. The direction is determined by the gradient of the first model that fits the data. If the goodness-of-fitness test fails for the first order model, the second order model is used.

Once a gradient is calculated, the center of the Local Experimental Region moves until it encounters either diminishing returns or a limit asserted in the setup dialog. Each step of the movement along the gradient never exceeds the width of the local experimental region, as defined in the Optimizing Experiment Generator Dialog. If the range of treatment levels is too narrow, it will take many unnecessary steps to climb the gradient. Each step in the move requires a simulation at the center of the Local Experimental Region, and its results are reported in the Journal. If, when movement stops, the local experimental region does not then contain an optimum, the experiment attempts to move in a new direction. This redirection can be suppressed by the Redirection Limit in the Optimizing Experiment Generator Dialog.

Figure 13-14. The Optimizing Experiment Generator Dialog.

Figure 13-14 shows the dialog used to generate Optimizing Experiments. Just as with the Screening Experiment Generator it is accessed through the Edit Menu of the Main Window.

To specify the PLUS Experiment that is about to be generated, we simply fill in the fields of the Dialog. The "Experiment Name" and the "‘Run Procedure’ Name" fields are used as the PROCEDURE names in the generated experiments. The EXPERIMENT calls the Run Procedure repeatedly.

The Factors of the Experiment come next. Each factor name is actually the name of a User Variable and must obey the GPSS World naming conventions. It must begin with an alphabetic character and it must not clash with a keyword, SNA, or SNA class. The two treatment levels of each factor define the Local Experimental Region which moves around searching for the optimum conditions. You must specify the names and two treatment levels for between 1 and 5 factors, inclusively. Factors must be specified consecutively starting with the A factor. The choice of treatment levels is crucial.

Several important trade-offs are implicit in the way you define the Local Experimental Region. First, it forms the starting point of the experiment. You should make it as close to the optimum as you can. Second, making it small usually makes the mathematical model fit better. Third, when the experiment moves one step toward the optimum it moves to an adjacent region. Therefore, a larger experimental region will take fewer steps. As you can see, the issues conflict with each other. It is up to you to choose compromises. Even further, don't overlook the fact that it is easy to modify a generated experiment after it has been inserted into your Model.. It's worthwhile to examine the generated PLUS code to look for ways to improve the experiment. You may want to do some preliminary experimentation so that the choices you make are good ones.

You should apply limits to the Optimizing Experiment. The section of the dialog for Move Limits gives you a chance to restrict the movement of the Local Experimental Region. If possible you should enter limits for each of the factors so that time won't be wasted on infeasible simulations. The Redirection Limit is another way to restrain the search. This value sets a limit to the number of times the experiment can start moving in a new direction in the factor space. This limit can prevent an unending cycle from occurring.

The "Result Expression" is required. You must specify an expression to be evaluated as the metric of the simulation.

Finally, we have two checkboxes that allow us to select additional options. First, you can have GPSS World generate a template Run Procedure to go along with the generated experiment. You can edit this Run Procedure to suit your needs. Second, you can have GPSS World set up a setting which loads the appropriate CONDUCT Command into the setting for Function Key 12. If you choose this, after you create the Simulation Object, you only have to press the F12 Function Key to run the experiment.

The next step is to insert the PLUS code comprising the experiment into your Model Object. When you have filled out all the appropriate items in the Optimizing Experiment Generator Dialog, click Insert Experiment to create the PLUS code and insert it into your model. If you have asked GPSS World to generate a Run Procedure, next we see a Dialog that allows us to personalize it. The details about personalizing the Run Procedure are in the beginning of this section.

In any case, you may want to edit the generated experiment itself in your Model Window. You can change the Run Procedure there, as well. Just click OK and all the new GPSS World Statements comprising the Optimizing Experiment are placed at the bottom of your Model.

Running the Experiment

CONDUCT is the GPSS World Command that is used to invoke an Experiment. The argument to the CONDUCT Command is similar to a Procedure Call. But we have a better way. In the Experiment Generation dialog, we can choose to have GPSS World load the F12 Function Key with the appropriate CONDUCT Command. You can see this in the Model Object's Settings. (see Edit / Settings / Function Keys).

To start the execution of the experiment, we must do two things. First we must Translate the Model, thereby creating the Simulation Object. Click on Command / Create Simulation to do this, or just press Ctrl+Alt+S.

Then we press the F12 Function Key. This enters the CONDUCT Command. The Experiment goes to work, reporting status and output to the Journal Window of the newly created Simulation Object. Figure 13-15 shows the results of an Optimizing Experiment.

Figure 13-15. Results of Optimizing Experiment.

When successful, the Optimizing Experiment reports the results of each run. From this can be seen the results of the sub-experiments and the movement of the local experimental region. The generated PLUS Experiment attempts to move the local experimental region so that it encloses the optimum conditions. The Experiment ends when this occurs, or when the redirection limit or a movement limit is reached.

When movement stops, the results of the goodness of fit test and the equation for the fitted response surface are also reported in the Journal Window. Then, if not already done, the PLUS Experiment attempts to verify the predicted optimum conditions by running a simulation.

## 13.5 Operating Tips

We end the chapter with some helpful hints that can be useful as well as some common pitfalls to be avoided while you are engaged in an experimentation project.

· GPSS World uses User Variables to take the treatment level of each factor as values. When you create names for your variables, they must not clash with existing keywords, SNAs, or SNA classes. Names with more than two letters, followed by at least one digit or underscore, are safe. The factor names in experiments are simple User Variable names and take on the indicated treatment levels as values during the experiment.

· By default, GPSS World uses the # symbol to denote multiplication and reserves * for SNA indirection. If you prefer to use * for multiplication, you can switch the role of * and # by a checkbox in the first page of the Model Settings. This is accessed through the Edit Menu.

· When you create your own names, they must not be the same as existing keywords or SNA Classes. Names including underscores are safe.

· Settings are inherited from the parent Model Object, including Function Key assignments (see the Edit menu). Optionally, the Experiment Generators load a Function Key with the appropriate CONDUCT command. The Screening Experiment Generator optionally loads F11, the Optimizing Experiment Generator, F12. After creating the Simulation, to run the experiment, just press the Function Key.

· The DoCommand library procedure allows an Experiment, or any Procedure called by an Experiment, to execute any GPSS World Command, except the CONDUCT Command.

· DoCommand translates its argument string in global context. Don’t use temporary variable names in it. If you need to pass the values of local variables or arguments to the DoCommand library procedure, you should build the string first using PLUS string procedures. That will cause the local variables to be completely evaluated before entering the scope of the DoCommand library procedure. The bottom line is that you must use values, not names, of temporary variables in the string you pass to the DoCommand Procedure. Global Variable names may be used, however. For examples, see the Run Procedure in Section 14.3.2.

· You can place INCLUDE Statements anywhere but inside a PLUS Procedure. Commonly, you would want to place all Commands after your last INCLUDE Statement.

· If you have more than one metric describing your observations, you will need to edit any generated experiment and define additional Result Matrices and ANOVA invocations.

· If find that in an ANOVA Table the F Statistics could not be calculated, try reducing the level of interactions in the analysis if you can. Otherwise, you will have to add replicates.

· Simulations can have any number of Experiments. Use the CONDUCT Command to invoke an experiment.

· Do not pass Procedure arguments or Temporary Variables to DoCommand without evaluating them first. This is discussed above. The DoCommand argument string is compiled in global scope, which cannot access local variables. You can create the string before the call to DoCommand, and then, in a separate step, pass it as an argument.

· Don’t forget that PLUS statements can only appear inside PLUS Procedures.

· Inserting a screening Experiment does not remove an existing Experiment of the same name from your Model Object, but it does replace it in your Simulation Object because being inserted at the end of your Model it is registered last in the Simulation Object.

· You can replace any Procedure, including Experiments, in an existing simulation object. Since they are longer than simple Commands, you would probably use an INCLUDE statement as a custom command, which points to a Text Object containing the Procedure.

· Initialize your Result Matrix to UNSPECIFIED. The default value of a GPSS Matrix Entity is 0. Initializing to UNSPECIFIED prevents 0 values from being used as results. Doing the initialization outside the PLUS Experiment allows your experiment to be restartable (see the next tip).

· If you want your Experiments to be restartable, inside your PLUS Experiment before each run you should test the Result Matrix element for the "UNSPECIFIED" Data Type. Only if that test is true would you call the Run Procedure for that run of the Experiment. Then, when the Experiment is restarted after being HALTed (and possibly SAVEd), completed runs are skipped. To see an example of this, refer to any generated Experiment.

· Generated Experiments are inserted at the bottom of your Model. An experiment replaces any like-named experiment registered previously. You are responsible for removing any old generated experiments you no longer need.

· You should always complete a thorough test of the base simulation before beginning the experimentation phase.

· When you get compilation errors, use the "Next Error" command in the Search Menu. When you get "run time" errors, use the "Go To Line" command in the Search Menu to go to the location where the error was detected.