INEDUCATIONAL RESEARCH - EXPERIMENTAL DESIGN
INTRODUCTION:
Experimental design refers to the
conceptual framework within which the experiment is conducted. An Experimental
design serves two functions: 1. it establishes the conditions for the
comparison required by the hypotheses of the experiment, 2. it enables the
experimenter through statistical analysis of the data to make a meaningful
interpretation of the results of the study.
The most important criterion is that
the design be appropriate for testing the particular hypotheses of the study.
The mark of a sophisticated experiment is not complexity or simplicity but
rather appropriateness. Will this design do the job it is supposed to do? Thus
the first task for the experimenter is to select the design that best arranges
the experimental conditions to meet the needs of the particular experimental
problem.
A second criterion is that
the design must provide adequate control so that the effects of the independent
variable can be evaluated. Unless the design controls extraneous variables, one
can never be confident of the relationship between the variables of the study.
Therefore the best advice is to select a design that utilizes randomization in
as many aspects as possible.
Validity of Research Design:
A very significant contribution to the evaluation of
research designs has been made by Campbell and Stanley, who suggest that there
are two general criteria of research design.
§ Internal validity
§ External validity
Internal validity:
Internal validity is essentially a problem of control. The
design of appropriate controls is matter of finding ways to eliminate
extraneous variables (i.e.) variables that could lead to alternative
interpretations. Anything that contributes to the control of a design
contributes to its internal validity.
External validity:
External validity refers to the generalizability or representativeness
of the findings. When experimental controls are tightened to achieve internal
validity, the more artificial, less realistic situation may prevail, reducing
the external validity or generalizability of the experiment. Some compromise is
inevitable so that a reasonable balance may be established between control and
generalizability between internal and external validity.
Threats
to Internal Experimental Validity:
In educational
experiments, or in any behavioral experiments, a number of extraneous variables
are present in the situation or are generated by experimental design and
procedures. These variables influence the results of the experiment in ways
difficult to evaluate. A number of factors jeopardize the power of the
experimenter to evaluate the effects of independent variables unambiguously.
Campbell and Stanley have
discussed these factors in their excellent definitive treatment. They include
the following:
Maturation
Subjects change (biologically and
psychologically) in many ways overtime, and these changes may be confused with
the effect of the independent variables under consideration. During the course
of a study, the subjects may become more tired, wiser, hungrier, older and so
on.
History
Specific
external events occurring between the first and second measurements and beyond
the control of the researcher may have a stimulating or disturbing effect on
the performance of subjects.
Testing
The processing of pretesting at the beginning of an
experiment can produce a change in subjects. Pretesting may produce a practice
effect making subjects more proficient in subsequent test performance. Testing
presents a threat to internal validity that is common to pretest-posttest
experiments. Of course, an equivalent control group would be affected by the
test in a similar way as the experimental group.
Unstable Instrumentation
Unreliable instruments or techniques used to describe and measure
aspects of behavior are threats to the validity of an experiment.
Statistical Regression
If groups are selected on the basis of
extreme scores, statistical regression may operate to produce an effect that
could be mistakenly interpreted as an experimental effect. This regression
effect refers or moves toward the common mean on subsequent measures.
Selection Bias
Selection bias is represented by the
nonequivalence of experimental and control groups, and its most effective
deterrent is the random assignment of subjects to treatments. Selection bias is
likely when, on invitation, volunteers are used as members of an experimental
group.
Interaction of Selection and
Maturation
This type of threat to the internal validity of a study is
not the same as selection bias. The interaction of selection and maturation may
occur whenever the subjects can select which treatment they will receive. Even
though the groups may be equivalent on the pretest and on other cognitive
measures, the reasons some choose one treatment over another may be related to
the outcome measures.
Experimental Morality
There may be differential loss of respondents from the
comparison groups. If a particular type of subject drops out of one group
during the course of the experiment, this differential loss may affect the
outcome of the study.
Experimenter Bias
This type of bias is introduced when the researcher has some
previous knowledge about the subjects in an experiment. This knowledge of
subject status may cause the researcher to convey some clue that affects the
subjects’ reaction or may affect the objectivity of his or her knowledge.
Experimental Design:
Experimental design is the blueprint of
the procedures that enable the researcher to test the hypotheses by reaching
valid conclusions about relationships between independent and dependent variables.
Selection of a particular design is based on the purposes of the experiment,
the type of variables to be manipulated, and the conditions or limiting factors
under which it is conducted.
The adequacy of experimental designs
is judged by the degree to which they eliminate or minimize threats to
experimental validity. Three categories are presented here:
1. Pre-experimental design is the least effective, for it
provides either no control group or no way of equating the groups that are
used.
2. True-experimental design employs randomization to provide for
control of the equivalence of groups and exposure to treatment.
3. Quasi-experimental design provides a less satisfactory degree
of control, used only when randomization is not feasible.
R - random assignment of subjects to
groups or treatment
X - exposure of a group to an
experimental variable
C - exposure of a group to the control
condition
O - Observation
1. Pre-Experimental Design:
The pre-experimental design does not provide a control or the
equivalent of a
Control
group.
A. The Single-Group, Posttest Design:
Here a group is exposed to some form of intervention
and then subsequently tested. This design might be illustrated as shown below:
Experimental Group -
Processed Observed or Measured
(X)
(O)
Carefully studied results of a
treatment are compared with a general expectation of would have happened if the
treatment had not been applied. This design provides the weakest basis for
generation.
In a test
administered after the showing of the film, the mean score was 86. The mean
score was higher that it would have been had the film not been viewed and, as
he recalls, higher than the mean score of a test that he had administered to a
similar class several years before. He concludes that the film has been
effective in reducing racial prejudice.
The reader has no way
knowing if a change has occurred because lack of pretest or if a similar group
who had not seen the film (a control group) would have scored differently from
the group viewing the film. This design is the poorest available and should not
be used.
B. The Single-Group,
Pretest-Posttest Design:
This design is an improvement over the
above design because the effects of treatment (X) are judged by making a
comparison between pre-test and post-test scores. However, no control group is
used in this design.
Experimental group – pre-test O1
- Special Treatment X – Post-test O2
In the same setting we have to administer
a pre-test before showing the film and a posttest after the viewing. He
computed the mean difference between the pretest and the posttest scores and
found that the mean had increased from 52 to 80, a mean gain of 28 score
points. It also apparently detected some temporary improvement in attitude
toward racial integration. He concludes that there has been a significant
improvement in attitude as a result of students viewing the film.
C. The
Static – Group Comparison Design:
This design compares the status
of a group that has received an experimental treatment with one that has not.
There is no provision for establishing the equivalence of the experimental and
control groups, a very serious limitation.
Experimental
Group - (X......O1) Group Tested (O1)
Control Group - Group Tested (O2)
A beginning
researcher administered the 25-minute racial integration film to a group of
elementary teachers in one school. He then administered the attitude scale and
computed the mean score. At another elementary school he administered the
attitude scale to teachers who had not viewed the film. A comparison of mean
scores showed that the teachers who had viewed the film had a higher mean score
than those who had not. He concluded that the film was an effective device in
reducing racial prejudice. The dotted line indicates that the control group
(O1) has not been equated by randomization.
2. True Experimental Design:
In a true experiment the equivalence
of the experimental and control groups is provided by random assignment of
subjects to experimental and control treatments. Although it is difficult to
arrange a true experimental design, particularly in school classroom research,
it is the strongest type of design and should be used whenever possible. There
are three types of experimental designs are discussed here,
A. Randomized Two Groups – Treatment
and Posttest Design:
This design is the most effective and useful
true experimental design, which minimizes the threats to experimental validity.
Randomly picked Special Treatment
(X) Post-test (O1)
experimental group (R)
Randomly picked
Control group (R) No Special Treatment Post-test (O2)
This design is one of the most
effective in minimizing the threats to experimental validity. It differs from
the static group comparison design in that experimental and control groups are
equated by random assignment. At the conclusion of the experimental period the
difference between the mean test scores of the experimental and control groups
is subjected to a test of statistical significance, usually a t test or an analysis of variance. The assumption is
that the means of randomly assigned experimental and control groups from the
same population will differ only to the extent that random sample means from
the same population will differ as a result of sampling error. If the
difference the means is too great to attribute to sampling error, the
difference may be attributed to the treatment variable effect.
B. Randomized Two Groups Pretest, treatment and Posttest Designs:
These types of design have been
described as true experimental designs because they always include the
processes of randomization.
Randomly picked -
Pre-test (O1) -
Special Treatment (X) - Post-test (O2)
experimental group (R)
Randomly
picked - Pre-test (O3) - No Special
treatment - Post-test (O4)
Control group (R)
This design
is similar to the previously described design, except that pretests are
administered before the application of the experimental and control treatments
and posttest at the end of the treatment period. Pretest scores can be used in
analysis of covariance to statically control for any differences between the
groups at the beginning of the study. This is a strong design, but there may a
possibility of the influence of the interaction effect of testing with the
experimental variable.
The Solomon Four-Group Design:
The Solomon Four Group Design
developed by Solomon (1949)is really a combination of the two equivalents –
groups design, namely, the post-test – only design and pre-test – post-test
only design and represents the first direct attempt to control the threats of
the external validity.
Randomized group -
Pre-test - Receives intervention
- Post-test
R (A)
O1 X O2
Randomized group -
Pre-test - No intervention - Post-test
R (B)
O3 O4
Randomized group
- Receives
intervention - Post-test
R (C)
O5
Randomized group
- No intervention - Post-test
R (D)
O6
The design is really a combination of
the two-group designs previously described, the posttest only and the
pretest-posttest. The Solomon Four-Group Design permits the evaluation of the
effects of testing, history and maturation. Analysis of variance is used to
compare the four posttest scores; analysis of covariance may be used to compare
changes in O2 and O4.
This design provides for two
simultaneous experiments, the advantages of a replication are incorporated. A
major difficulty is finding enough subjects to assign randomly to four
equivalent groups.
3. Quasi – Experimental Designs
These designs provide control of when
and to whom the measurement is applied, but because random assignment to experimental
and control a treatment has not been applied, the equivalence of the groups is
not assured.
A. The Non-equivalent Pre-test –
Treatment – Posttest Design:
This design is often used in classroom experiments when
experimental and control groups are such naturally assembled groups as intact
class, which may be similar.
Experimental
Group - Pre-test (O1) - Treatment (X) - Post-test (O2)
Control
Group - Pre-test (O3) - Post-test (O4)
As in the pretest-posttest equivalent
group design, analysis of covariance may be used with the pretest as the
covariate. Because this design may be only feasible one, the comparison is
justifiable, but as in all quasi-experimental studies, the result should be
interpreted cautiously. Difference between the Pretest and posttest (O1 and
O2) of experimental group and the difference between the Pretest and
Posttest of Control group (O3 and O4) will give the
result of the design.
B. Single Group Time-Series Design:
This is the design which consist
number of Pretest and Posttest only for Experimental Group. One of the main
problems with conducting experiments is that the effects of intervention may be
only partially revealed, at the moment of testing.
Experimental Group- Pretest
–Pretest-Pretest – Treatment – Posttest-Posttest
O1
O2 O3 X O4 05
-
Posttest
O6
The diagram showing one X and several Os does not
necessarily represent the relative number of sessions for each. It may be that
each O represents one measurement, and the single X represents an intervention
of several weeks. Although it is better to have several observations, as shown,
it is not always possible to have this many. For instance, a time-series
experiment by a student of the second author used only two pre-intervention and
two post-intervention measures. Because this study was measuring the effect of
a program to reduce the number of criminal visualizations of students with
disabilities, it was necessary to have a 2-month period between measurements in
order to have a sufficient number of victimization for each period measured.
Thus, the intervention appeared successful in reducing crimes committed against
persons with disabilities.
C.
Control Group Time-Series Design:
This design is somewhat related to
the previous design. The previous design consist only Experimental Group. Here
in this design, we are including Control Group without treatment.
Experimental Group- Pretest –Pretest-Pretest – Treatment –
Posttest - Posttest
O1
O2 O3 X O4 05 -Posttest
O6
Control Group- Pretest –Pretest - Pretest ––-- Posttest – Posttest
- Posttest
O1 O2 O3 O4 05 O6
This design proves the effect of
treatment by comparing difference of both the group results. It also gives the
importance of number of pretest and posttest among both the groups.
D.
Counter Balanced Design:
These are designs in which experimental control derives from
having all the subjects receive all the treatment conditions. The subjects are
placed into, in the case of this example, four groups. Each of the groups then
receives all four treatments but in different orders.
A Pretest T1 T 2 T3 T4 Posttest5
B
Pretest T 2 T3 T4 T1 Posttest5
C Pretest
T3 T4 T1 T2 Posttest5
D
Pretest T4 T1 T2 T3 Posttest5
A counterbalancing design in which
four treatments have been randomly given four groups at four different
occasions. Variables like history, maturation, testing, instrumental morality
posing threats to internal validity are well controlled by the counterbalanced
design.
Conclusion
Experimentation is a
sophisticated technique for problem solving and may not be an appropriate
activity for the beginning researcher. It has been suggested that teachers may
make their most effective contribution to educational research by identifying
important problems that they encounter in their classrooms and by working
cooperatively with research specialists in the conduct and interpretation of classroom
experiments.
Reference:
1. Research in Education -
John W. Best and James V. Kahn
2. Research
Methods in Education - Dr. Radha Mohan
Prof. EG. Parameswaran
0 comments:
Post a Comment