ASPECTS OF EDUCATIONAL TECHNOLOGY

Introduction

Flight Lieutenant A.W. Panton, M.A. Dip. Ed. served in the Royal Air Force Education Branch from 1970 to 1974. Influenced by the RAF School of Education at RAF Upwood, Ramsey, Hunts, the Branch emphasised the importance of a technological or systematic approach to education. In preparing for his promotion examinations to Squadron Leader in November 1973, Flight Lieutenant Panton prepared notes on the following subjects set out below. These notes give a flavour of the kind of systematic approach which was encouraged within the Education Branch at that time. Where some of the points below may appear opaque, at this distance in time it is not necessarily possible for them to be adequately clarified now.

1.  FACTORS INFLUENCING CHOICE OF INSTRUCTIONAL METHOD

1.  Objectives.  Intended terminal behaviour of students will affect method. Different aspects of instructional task will also involve different methods.

2.  Subject Matter.  Stability of content. Verbal or manipulative? Symbolic or Non-Symbolic? Easy to assimilate or not?  

3.  Target Population.  Class Size. Aptitudes. Experience. Educational level.

4.  Instructional staff.  Numbers. Ability.

5.  Facilities.  Accommodation. Equipment. Aids. 

6.  Time.  Total course time. Time of day.

7.  Costs.  Interlinked with all factors. However investment in one method should offset other costs.

2.  THE CHARACTERISTICS OF DIFFERENT INSTRUCTIONAL METHODS

1.  Lecture. Pros: speed, large class size; content under control of instructor, versatile. Cons: communication one way, limited sense appeal, passive audience, student reaction hard to gauge, useless for skill teaching. Uses: introductions, general surveys, indicate rules and policies, recapitulation and summaries. 

2.  Demonstration. Pros: dramatic appeal, shows relations between steps, saves time by reducing explanation. Cons: limited class size. Uses: illustration of manipulative skills, operation of equipment, safety procedures and principles.

3.  Student performance. Pros: realistic application of knowledge, student confidence, student participation, reduces wastage through error, method of evaluation. Cons: outlay of equipment and resources. Uses: teaching manipulative skills, operation of equipment, safety drills, teamwork.

4.  Programmed Instruction.  Pros: meticulous preparation ensures effectiveness; self-pacing of students; forced response and immediate feedback guarantee attention, improve retention and eliminate errors; instruction is standardised; frees instructor from routine and repetitive tasks. Cons: difficulty and cost of programme writing; few programmers around; unsuitable for unstable content; self-pacing causes timetabling problems. Uses: free-flow training and instruction; remedial instruction; filling in gaps, consolidation and practice.

5.  Discussion.  Pros: should stimulate; student participation leads to better retention; use of students' knowledge. Cons: difficult to control, take time, require small selected groups. Uses: problem solving, supplement other methods, application of theory to particular situations.

6.  Assignment.  Pros: can reduce need for classroom capacity; capitalise on individual interests, provide a detailed coverage; motivation. Cons: difficulty in defining objectives, hard to evaluate, no use for standardisation. Uses: advance study, provision for student differences, enrichment.

7.  Tutorial Method.  Pros: one-to-one teacher/pupil ratio very effective; student participation; easy to diagnose student needs. Cons: demanding; expensive use of resources. Uses: teaching of highly complex skills, operation of dangerous and expensive equipment.

8.  Seminar.  Pros: student participation, adaptive instruction. Cons: expensive, difficult to evaluate. Uses: guidance in advanced study, exchange of information.

9. Lesson.  Pros: highly flexible, uses many methods, cooperation between instructor and class. Cons: small groups, students have to keep pace with instructor, adults prefer other methods. Uses: elementary levels, novice instructors, practical skills.

10.  Role Playing / Stimulation.  Pros: interest, motivation > retention, realism. Cons: preparation, effects of gaming. Uses: practice in controlled conditions, special training, team training, interest.



3.  METHODS OF MOTIVATING STUDENTS

A.  Extrinsic motivation - involves context factors (Maslow) imposed on task or student by teacher or external agents. Assoc. with Maslow's 3 lower order needs - physiological, safety and belonging, and with Herzberg's hygiene factors. By removing causes of environmental dissatisfaction from environment, teacher can bring student to minimum acceptable level of learning. Methods of extrinsic motivation:

1.  Show purpose of instruction and connection with external goals.

2.  Organise teaching and its institutions well. Sound policy.

3.  Adopt relaxed teaching style.  Warmth of approach > friendship, identification.

4.  Fair distribution of praise and blame. Gives security and order.

5.  Comfortable classroom conditions. Lighting, temperature and ventilation, size of classes, seating, audio-visual aids.

(Connected with McGregor's X Theory of Management.)

B.  Intrinsic motivation - involves content factors (Maslow) inherent in task or student. assoc.with Maslow's 2 higher order needs - esteem  and self-actualisation, and Herzberg's motivators. Such motivation leads to positive results. Methods of intrinsic motivation:

1.  Encourage curiosity and interest. Make use of elements of realism and variety and surprise > interest = self-actualisation.

2.  Use discovery methods. Desires for challenge and autonomy are strong and lead to sense of achievement and recogntion.

3.  Use activity methods.  Encourages desire to express oneself and to take responsible part in a group. Role-playing and discussions.

4.  Proper use of rivalry and competition.  Group competition methods preferable. Competition > confidence, self-respect and esteem.

5.  Set high standards. Demonstrate examples of high craftsmanship. We learn by initiations, and the standards we expect are set by what we see.

(Connected with McGregor's Y Theory of Management.)



4.  THE PURPOSES AND USES OF AUDIO-VISUAL AIDS

A.  General categories (Powell)

1.  Aids which amplify the mechanics of the transmission and reception of sensations: e.g. microphones, projectors (hardware).

2.  Aids which contribute to the teaching/ learning process: e.g. diagrams, models, gramophone records (software).

B.  Functions of AV aids

1.  Invite cooperation and challenge.

2.  Promote perception: by attracting and holding attention.

3.  Promote understanding: supplement verbal information (very important in technical subjects); illustrate relationship between rule and application by directed observation; illustrate relationship between parts of a whole; promote transfer of training; provide reinforcement or knowledge of results.

4.  Promote consolidation: help retention - use of summaries.

C.  General categories of AV materials (Davies)

1.  Criterion materials:  have to be described, interpreted, identified, etc, by student to show that he has learned: e.g. pictures, maps, real objects.

2.  Mediating materials: help students to gain insight or knowledge of phenomena: e.g. diagrams, transparencies, graphs.

D.  General categories of AV media (Davies)

1.  Enrichment media: provision optional.

2. Necessary media: to realise learning task, whether cognitive, affective, or psychomotor: provision obligatory.



5.  USES OF PARTICULAR AUDIO-VISUAL AIDS

1.  Chalkboard.  The most important aid for general teaching of cognitive objectives. Can be used for (a) drawings, diagrams, maps, graphs, (b) technical words, key words, definitions, (c) summaries and outlines, (d) problems. Drawing and diagrams should be simple. Complex ones such as circuit diagrams should be given on note-sheets. Chalkboards very versatile and intimate.

2.  Felt Board, Plastigraph Board. Useful alternatives to chalkboard. Can be used with large groups. Display of material the matter of a moment, but less versatile than chalkboard.

3.  Magnetic Boards.  Useful for showing effect of moving shapes to different positions, for traffic problems, work study, and economics problems, etc. Can also be used like chalkboards.

4.  Overhead Projectors. Another alternative to chalkboards. Strong visual appeal. Transparency only has to be prepared once. Overlays can build up picture by degrees. Teacher can write facing class.

 5.  35mm. Slide Projectors. Can show individual slides or film-strips. Greater detail possible than in OHP transparencies. Useful for both cognitive and affective objectives. Tonal range of slide makes them preferable to prints and they have enrichment value.

6.  8mm. Loop Projector. For concept films. Easy to present and to integrate, films on casettes can be replayed, and useful for individual project learning.

7.  16mm. Film Projector. Very important for affective and pyschomotor objectives. Can be used to orientate attitudes or to consolidate learning. Usually films should be shown twice. As they take over from teacher they must have high merit.

8.  Wireless, record-players, tape recorders. Have cognitive and affective value. Tape-recorder has psychomotor value. Tape/slide synchroniser units now available for standard presentations. Sound/slide systems also available. Casettes, like loop films, can be used for individual learning.

9.  3-D models.  Mock-ups, cutaways and large models. Useful for cognitive and, in the case of large models, affective objectives. large models often very expensive.

10.  Television.  Can be used for films, but lacks visual appeal of film projectors. CCTVs and VTRs have tremendous possibilities for training. Video casette recorders also.

11.  Teaching machines. Use programmes. Valuable for remedial and individual learning, e.g. reading. Can incorporate sound and loop films.

12.  Simulators.  Used where training on real equipment is too expensive or dangerous, e.g. emergency landing of aircraft. Represents real situations. Students have control  and conditions can be varied. Not only used for psychomotor skills but also for cognitive (chain and m-d learning) and affective objectives. Difficult to integrate.

13.  Language laboratories. Useful for sequential, cumulative development of language skills, each small skill
building on the last. Weak when used as an adjunct to grammar program.



6.  THE STAGES OF INSTRUCTIONAL PREPARATION

1.  Aim.  The overall aims must be ascertained. Training objectives essential in training. These may be cognitive, affective or psychomotor. Aims may affect content and method.

2.  Objectives.  With academic exams. syllabi and past papers specify content and relative weighting. In training a task analysis is required to give objectives.

3.  Content. This will be derived from objectives minus student knowledge calculated from preliminary test. Relative importance of items must also be decided from past papers or task analysis.

4.  Sequence. Learning must be planned to achieve most effective learning in shortest possible time and to cut out duplication and omission. In skill training logical order comes from job performance. In academic or theory teaching the best psychological order (instructional logic) must be found. This will proceed from what student knows to required knowledge, and from simple to complex. Relationships between objectives that the student must know may be by association or discrimination. Sequence must establish the latter. In sequencing a subject  with a logical or natural order the matrices and flow diagrams of programmed learning are useful.

5.  Criteria.  The most appropriate tests have to be selected. Types of test: essay, MCOQs, short answer questions, practical/manipulative, practical written, oral/aural. Tests should be held (a) when an objective has more than one other objective dependent on it, (b) when concepts or combinations of concepts have to be mastered, (c) often enough for previously learned material not to be forgotten. The characteristics of tests that have to be considered are validity, reliability, ease of preparation and ease of marking. Tests ought to be reliable, valid and compatible with learning situation. In skill training validity and reliability are essential. Essays are valuable when flexibility is required.

6.  Instructional plan. The teaching method then has to be selected. Types available: lesson, lecture, demonstration, student peformance, programmed instruction, discussion, project, tutorial, seminar, simulation. Factors influencing choice: objectives, subject matter, target population, instructional staff, facilities, time, costs. Sense appeal (75% sight, 25% hearing) dictates desirability of certain methods. When method has been determined, time has to be allocated to syllabus items on the basis of total time available, item complexity and student ability.

7.  Resources. The provision of resources then has to be planned: books, paper, AV aids, note handouts, past papers, course plans, etc.

8.  Environment. Classroom or lecture hall has to be available, quiet, light, clean, tidy and have enough chairs. It also has to be suitable for the instructional method selected.



7.  THE PHASES OF LESSON STRUCTURE

A.  The Classroom (Theory) Lesson

1.  Introduction - arouse interest and focus attention. Should provide a motive for learning. Should relate to previous learning. Should list objectives and lead smoothly into development phase.

2.  Development - teaching phase. Should aim to present subject matter in manner most easily absorbed and retained. heavy emphasis on participation by question and answer technique. Any questions? 

3.  Recapitulation - tests recall of what has been learned. Monitors success of development phase. In a lengthy development phase, intermediate recapitulations are advisable.

4.  Application - involves application by students of principles and procedures learned. Could lead into homework, including test of earlier phases.

5.  Conclusion - apart from recapitulation and application, conclusion should allow for student queries, the provision of references and setting of homework, and indication of future lines of instruction.

B.  The Skills Lesson

1.  Introduction - arouse interest and focus attention. should provide motive for learning and and relate to previous learning. Should list objectives and lead smoothly into development phase.This explanatory phase should be 10% of lesson.

2.  Development phase - this involves a demonstration, carrying out the whole procedure step-by-step in the correct job sequence. This phase, which involves both talking and demonstrating, takes up 25% of the lesson. It can include a second demonstration.

3.  Recapitulation - a second demonstration,as above.

4. Application (imitation) - involves student performance, practice of the skills under supervision and appraisal of the students' work. Student practice should take up 65% of lesson.



8.  PROGRAMMED LEARNING: DEFINITION, USES, PROS & CONS

1.  Definition - a form of instruction/ learning, in which the following factors are present: (a) clear statement of the objective; (b) itemised and tested material is presented in frames; (c) students follow a sequence of frames fitted to their individual needs; (d) frequent responses from students are required; (e) immediate feedback is given before the student proceeds to the next frame.

2.  Characteristics - (a) PL is an individual learning process in which students accept a wider measure of responsibility; (b) students proceed at their own rate; (c) it requires an active response from student and provides immediate knowledge of results; (d) student is more often successful and his motivation is thereby increased; (e) subject matter is programmed in such a way as to shape learning in a particular manner.

3.  Uses:

a. Remedial instruction, providing training in infrequently practised skills;

b. Filling in gaps caused by late arrivals and absences.

c. Acceleration of able trainees, permitting early completion;

d. Providing common background for subsequent instruction;

e. Consolidation of learning by practice;

f. Provision  of advanced or broader work in a particular field of study; and

g. As a control in the study of learning situations.

4.  Advantages: (a) pre-testing, self-pacing, forced response and immediate feedback characteristics lower failure rate; (b) they also improve retention rate; (c) elimination of unnecessary material, self-pacing and forced attention cuts time; (d) instruction is standardised; (e) no special facilities are required, except teaching machines where used; (f) can be effective in the absence of an instructor; (g) can meet individual or group needs; (h) can be used for classes of any size; (i) frees instructor from repetitive teaching tasks and frees him to devote time to difficult tasks.

5.  Disadvantages: (a) commercial programmes rarely match particular instructional objectives and have to be prepared locally or by contract; (b) only a few trained programmers are available; (c) programmes are very costly, involving either training of programmer, writing and testing of local programmes, or very costly contracts; (d) unsuitable where subject matter is unstable or subject to radical and frequent change; (e) cause admin. and organisational problems: viz. different completion times causing timetable difficulties, and problems in assigning students to jobs at end of training.



9.  PROGRAMMED LEARNING: LINEAR AND BRANCHED

A.  Linear Programmes

1.  Theory:  stem from instrumental conditioning theory of B.F.Skinner. This has 3-fold process: simplification
> reward > reshaping simple into complex responses. Skinner sees rewards as means of getting repetition of correct responses, and thus means of effective learning.

2.  Implications of theory for programmed learning:

i.  We must proceed in small steps in order to adjust the pace of conditioning to behaviour of individual. Too much too soon will lead to learning breakdown.

ii.  We must move slowly so as not to impose too difficult a discrimination task on student whose capacity at outset is limited.

iii.  We must anticipate difficulties and modify approach, relating it to student behaviour.

iv.  We must not reinforce behaviour incompatible with desired learning.

3.  Characteristics: a body of info. is broken down into a sequence of small steps leading logically through subject. Increments of information that students have to absorb from one frame to another are limited. Common element is discriminative stimuli. The more often a student is correct - wrong answers to be avoided - the more he is motivated. Students write down or construct responses. Skinner believes this deepens thought.

B.  Branched Programmes

1.  Theory:  stems from differential school of psychology, to which Norman Crowder belongs. This school believes that learning occurs by exposing students to new material. Learning occurs in a variety of ways depending on abilities of students and nature of subject.

2.  Characteristics:  Crowder uses responses of student to control order of presentation of material. To attain control he uses MCOQs, though constructed answers could be used. Small steps of linear programmes are considered an insult. More ambitious steps are followed by remedial sequences of frames if necessary. An able student will progress much faster than a weak one.

3.  Purposes of MCOQs:

i.  test understanding;

ii.  to select remedial sequence if response is incorrect;

iii.  practice of concept;

iv.  to keep student working actively;

v.  to motivate student when response is correct.



10.  OBJECTIVES IN EDUCATION AND TRAINING 

1.  Different domains:

a.  Cognitive objectives: information and knowledge;

b.  Affective objectives:  attitudes, values, feelings, emotions;

c.  Psychomotor:  skills involving neuromuscular coordination.

2.  Definitions:  'A precise description of intended terminal behaviour'. 'A training objective written in performance terms is a statement of what the student must DO to show that he has learned; it will contain a statement of the conditions under which the performance must be carried out, and a statement of the conditions under which the performance must be carried out, and a statement of the standards to be achieved'.

3.  Different types of learning objectives:

a.  Training objectives differ from the task analysis to the extent that we can consider on-the-job-training and the value of experience in improving performance.

b.  Course objectives are derived from training objectives by the deduction of the required entry behaviour.

c.  Enabling objectives will define the path to be followed towards the attainment of the terminal objectives.

4.  Defects of traditional syllabi:  a. give no guidance to students or instructors as to conditions or standards; b. cannot be tested save in random fashion; c. list contents only. d. are teacher centred; e. over-training = wastage; under-training = ? danger.

5.  Characteristics of objectives:  a. they are written in behavioural or performance terms, using a precisely worded vocabulary of behavioural verbs; b. include a statement of conditions; c. include a statement of standards; d. synonymous with criterion; e. related to task analysis in the case of training. In the case of education, anything on the syllabus should be capable of justification (N.B. affective objectives cannot be directly tested, but could inform teaching methods.(Performance syllabi are both job-related and student centred).

6.  Advantages: a. instructors are clear about standards; b. students know 'what' they are required to learn and can monitor progress; c. objective testing is possible; d. training is efficient.

7.  An example: 'Given a mild steel block approx. 1/8'' oversize, the student must be able to file it down to specified dimensions within a tolerance of 0.002". All faces must be parallel and square to each other and square to the front face'.



11.  THE SYSTEMS APPROACH TO TRAINING

1.  Elements of training:

a.  Input: body of entrants to be trained to a certain level.

b.  Processing: training carried out in a variety of ways over varying periods and using variety of resources.

c.  Output: a trained body of men.

d.  Feedback: experience gained reflected in future selection standards and form of training.

2.  Functions of Systems Approach: To identify, understand and control the interrelationships of the 4 elements and of the processes of which they are contained.

3.  Process of training:  a. instructional system is a compound of interacting, interdependent process. Weaknesses are attributable to absence or misapplication of these processes or the failure to coordinate them. The main processes are:

a.  Job and behavioural analysis:  the preparation of job specifications on the basis of a task analysis, the job specifications indicating the required performance levels.

b.  Statement of training objectives:  the declaration in behavioural terms of training objectives based on the performance requirements; and the design of criteria by which the effectiveness of training can be assessed in relation to those objectives.

c.  Definition in target population:  the calculation of the standards of skill and knowledge possessed by entrants to training.

d.  Design of the training course:  choosing content and sequence; selecting instructional methods, and designing achievement tests to provide immediate feedback on the effectiveness of training.

e.  Implementation of the training course: the processing of instructors, students, time and resources.

f.  Evaluation of training effectiveness: the establishment of an evaluation system to provide long-term feedback from the field on the effectiveness of the overall training system.

4.  Need for the Systems Approach: only by a thorough understanding of the interrelationships of the training processes can the training be controlled so as to stabilise output regardless of input variables.


                              THE 4 INTERRELATED ELEMENTS OF TRAINING


            >      UNTRAINED STUDENTS      >          TRAINING            >          GRADUATES           >      
      i                    (INPUT)                                (PROCESSING)           i    
      i                                                                            i                           i
      i                                                                            i                           i
      i                                                                            i                           i
      i                                                          <                i                   FEEDBACK                                                                                 



12.  PERSONAL QUALITIES OF THE GOOD INSTRUCTOR


1.  Purpose.  Object clear. Keeps to subject but deals with interesting points. Keeps an eye on the clock.

2.  Cognitive validity. The degree to which a teacher possesses and reflects a valid, systematic cognitive structure of concepts and principles of subject. Skilled demonstration. Knows his staff. Instruction backed by experience.

3.  Care.  Preparation of lesson and materials good. Sense of balance and proportion as to what is covered.

4.  Interesting.  Uses drama in proper places. Introduces variety. Timing.

5.  Enthusiasm.  Keen. Infects class with enthusiasm. (Modelling force.)

6.  Warmth.  At ease. Confident. Pleasant voice. Natural. Be reinforcing. (conditioning and modelling force.)

7.  Control of class. Fair. Firm. Friendly. Respected. See  students' viewpoint.

8.  Manner. Voice.  Stance.  Lack of irritating mannerisms.



13.  QUESTION TECHNIQUE

1.  Functions of oral questions:

a.  To make class think:
   
i)   to rouse curiosity and interest.

ii)  to lead class by logical steps to building up a body of knowledge. Learning can be shaped in this way as in a programme.  

b.  To test and confirm knowledge:

i)   to test the assimilation of initial knowledge and the rate of learning.

ii)   to revise and consolidate at the end of a piece of instruction.

iii)  to evaluate learning and the success of instruction either in the long-term or short-term.

2.  Factors to consider in oral questioning:

a.  Name last.

b.  Spread of questions.

c.  Match questions to ability in order to encourage in order to encourage feeling of participation.

d.  Phrase questions carefully.

e.  Use wrong answers as feedback.

f.  Give praise and encouragement.

g.  No big steps from one question to another.

h.  Answers must be heard. Repeat if necessary.

3.  Questions to avoid:

a.  Questions that lack obvious answers.

b.  Questions which have several equally good answers.

c.  Questions that call for 'Yes' or 'No' as an answer.

d.  Questions that have two possible answers.

e.  Questions that the class cannot answer.

f.  Trick questions.

g. Questions that are mainly tests of the power of expression.



14.  OBJECTIVE TESTING: PROS & CONS

1.  Advantages:

a.  Marking is more rapid and does not require skilled examiners.

b.  No subjective element in assessing answers.

c.  Once constructed, exam. can be given to 1,000s with little difficulty.

d.  Large number of short items enable a sampling of whole syllabus.

e.  Not confined simply to facts but can be used for testing higher order strategies.

f.  Not dependent on candidates' facility in writing essay answers.

g.  Difficulty level consistent from exam. to exam. Pre-testing avoids ambiguity.

2.  Disadvantages.

a.  Construction is very elaborate and involves use of psychologists and need for external validation.

b.  The choice of questions and responses introduces subjective element.

c.  Discrepancies will be found in 2 or more tests on same candidate - attributable to particular choice of questions as well as to variations in candidates.

d.  Appear to be too easy and to involve speed and guessing.

e.  Questions on higher order strategies difficult to construct and artificial, and success in them depends mainly on familiarity with the medium.

f.  Writing may be educationally valuable but doing objective tests cannot be.

3.  Procedure for making objective tests.

a.  A team should be collected: subject experts and psychologists trained in testing techniques.

b.  Content of exam. should be mapped out and weighting decided.

c.  Preliminary pool of draft items for consideration > leads to item bank.

d.  Selection of 1.5 as many questions as required  for validation.

e.  Validation on 300-500 similar pupils to test discriminability, frequency of choice and length of paper.

f.  Elimination of unsatisfactory items and construction of final exam.

g.  Statistical assessment of each exam. and adjustments of item bank.

4.  Conclusions.

a.  Objective MCOQ tests not completely objective or reliable.

b.  But more objective and systematically prepared than conventional ones.

c.  Most natural tests should be applied. Facts better tested by objective tests than by essays.

d.  Objective tests most appropriate where large numbers (100+) are to be examined.

e.  Objectives tests useful where candidates are not fluent writers and where short items can be used.

f.  Useful for lower order strategies and higher order ones in science and technology.

g.  Essay form superior at advanced level where higher order strategies are to be tested, fluency in writing can be assumed, examiner ration is high, and students are unpractised at objective level.



15.  THE REQUIREMENTS OF TESTS

1.  Standardisation.

a.  The task - the problems, the instructions used and the time - should be the same for all candidates.

b. Test must be capable of numerical assessment, and the method of scoring must be the same for all candidates.

2.  Reliability.

a.  The test should measure accurately whatever it is measuring, and scores should not rely on chance.

b.  An estimate of reliability can be produced by:

1) repeating the exam.

2)  using parallel exams.

3)  comparing two halves of the same test.

c.  Tests can be made more reliable by:

1)  increasing length.

2)  designing them to produce a wider range of scores.

3)  several examiners marking same papers.

3.  Validity.

a.  The test should measure what it is supposed to be measuring.

b.  Validity must be assessed by comparing test with some external  measure, and that must depend on the purpose of the test. An attainment test can be compared with the results of another method. A predictive test must be compared with future results (Technical validity).

c.  The test must measure whether the candidate can carry out the activities which were the objectives of the course (Content validity).

d.  Estimates of validity can be assessed by:

1)  comparing results with those of another exam. of proven validity.

2)  comparing results with an exam. of another type.

3)  comparing results with own assessments of students' abilities.

e.  Exams. can be made more valid by:

1)  adequate identification of objectives to be assessed by exam.

2)  a sound selection of objectives to be assessed in particular exam.

3)  the selection of a suitable method of examining.

4)  clear relationship between course objectives and exam. items.

VALIDITY IS CRUCIAL. A TEST CAN BE RELIABLE AND NOT VALID. IF IT IS UNRELIABLE IT CANNOT BE VALID.



16.  SOURCES OF UNRELIABILITY IN TESTS OF CONVENTIONAL TYPE.

1.  Scorer unreliability:

a.  Different examiners have different standards.

b.  Different examiners look for different qualities.

c.  An examiner differs in mood, and accuracy of interpretation from time to time.

2.  Content unreliability (or invalidity):

a.  Poor sampling of content in conventional exam. leads to question spotting and a chance factor is involved. Large variation in papers results.

b.  Poor question format and ambiguous instructions lead to unreliability.

3.  Temporal unreliability:

a.  Sickness and fatigue affect results on different occasions.

b.  Degree of application varies.

c.  Recency of revision affects results.



17.  TEACHERS' ASSESSMENTS: PROS AND CONS

1.  Advantages:

a.  Based on pupils' work over a period of time, not on one day.

b.  Reflect normal work under everyday conditions.

c.  Teacher can teach what he considers appropriate without reference to mind of examiner.

d.  Can take into account student performance over a number of dimensions, including enthusiasm, creativity and team-work.

e.  Avoid the need for revision and for the overload of facts beyond what is required for understanding.

2.  Disadvantages:

a.  A teacher cannot set aside his own reactions to pupil, although these may be irrelevant.

b.  Assessments can be variable, particularly where they concern middle range pupils.

c.  Teachers rarely teach pupils over the whole range of a field of study at one time. In consequence memory has to be relied upon.

3.  Methods of overcoming defects:

a.  Required scholastic performance should be itemised, and assessments should be given for items independently.

b.  Assessments should be made by ranking in order of merit.

c.  Assessments should be made by more than one teacher and totalled.

d.  To avoid halo effect, assessments should be made on separate occasions and without reference to previous assessments.



18.  THE FUNCTIONS OF TESTS

1.  Criterion-based tests: certify whether or not a student has obtained knowledge, and skills required for a certain level of proficiency. (RAF trade tests are of this type.)

2.  Predictive tests:  assess capacity for a future job. Aptitude and intelligence tests measure capacity not attainment. (RAF recruitment tests are of this type.)

3.  Normative tests:  a device to grade students in an order of merit. Such tests will be more concerned to discriminate than to test. Useful for selection purposes.

4.  Diagnostic tests:  identify particular weaknesses of individual students at the beginning of a course. Progress tests at intermediate stages of courses can give feedback as to effectiveness of instruction as well as
on students' progress.

5.  Learning functions:  tests can assist in the learning function, not only by providing a spur to student, but also by providing a spur to student, but also by providing knowledge of results to assist student to see what he is supposed to be learning, and providing both him and instructor with a monitoring device.