UNIVERSITY OF

ILLINOIS LIBRARY

AT URBANA-CHAMPAIGN

BOOKSTACKS

Digitized by the Internet Archive

in 2012 with funding from

University of Illinois Urbana-Champaign

http://www.archive.org/details/computerizedtest267burt

Faculty Working Papers

COMPUTERIZED TESTING IN THE BEGINNING ACCOUNTING COURSE

Eric James Burton, James C. McKeown Jeffrey Shlosberg

#267

College of Commerce and Business Administration

University of Illinois at Urbana-Champaign

FACULTY WORKING PAPERS College of Commerce and Business Administration University of Illinois at Urbana-Champaign September 10, 1975

COMPUTERIZED TESTING IN THE BEGINNING ACCOUNTING COURSE

Eric James Burton, James C. McKeown Jeffrey Shlosberg

#267

Computerised Testing in the Beginning Accounting Course

Eric James Burton Ph.D. Candidate, Accountancy University of Illinois Urbana-Champaign

James C. McKeowi, Ph.D.

Associate Professor oj. Accountancy

University of Illinois

Urbana-Champaign

Jeffrey Shloeberg Arthur Andersen & Co« Minneapolis, Minnesota

August 4, 1975 Draft 3

COMPUTERIZED TESTING IN THE BEGINNING ACCOUNTING COURSE

The purpose of this paper is to report on a unique educational tool being used by the Department of Accountancy of the University of Illinois at Urbana- Champaign (U1UC). The PLATO IV (Programmed Logic for Automated Teaching Operations) computer system is the tool. This note will describe the use of

PLATO for examination purposes. A full discussion of the implementation of

1 PLATO IV as a teaching device is contained in a paper by James C. McKeowa.

How PLATO Works The PLATO system Is a remote terminal access system. The remote terminals

consist of a keyset and a plasma display panel which are linked to Central Processing Unit via direct or dial-access telephone lines. Programs are written in a system- specific languaged called TUTOR.

When he wants to work on PLATO he addresses a terminal by typing in his name, his course and his password. If the machine accepts these items, the student is allowed to select lessons from the course in which he is listed. The student user needs absolutely no programming experience or knowledge.

The student sees, on his plasma panel, a series of frames (analogous to the pages of a book), Each frame may contain textual material, graphics, journal entries, questions, or combinations of these. When questions are asked in a lesson, the student must type in his response. If his response is correct, it is acknowledged and the student may proceed. If it is incorrect, a wrong answer response is shown and the student must try again. The objective of this procedure is a continuous check on the student's understanding. In many instances, if the student can not obtain the correct answer alone there is a programmed in HELP sequence to assist him in understanding the material and obtaining a correct answer.

i

Accy 101 Experience Since the Summer Semester of 1973 the UIUC Department of Accountancy has experimented with the PLATO system in the introductory course (Accy 101).

During the Fall Semester 1973, a Latin-squares design experiment was conducted to determine the effectiveness of the system and the 26 programmed lessons as a teaching tool (see McKeown), Simply said, the results encouraged expansion and further experimentation.

During the Spring, 1974, the Department continued informal experimentation on a student self-paced basis. Through that Spring Semester and the following Summer Semester an examination procedure was developed. In the Fall, 1974, the examina- tions were revised and utilized in four (4) sections of Accountancy 101.

PLATO _ Examinations Three PLATO examinations were given during the Fall, 1974 semester. The first examin- ation consisted of multiple choice questions s other objective types of questions, and

journal entries. The second exam Included multiple choice questions, journal entries, and computational questions. The final exam was approximately a 60-40

mix of material covered after the second exam and material covered on the first

two exams. It was comprised of multiple choice questions, journal entries,

and computational questions.

Each student received a unique examination. However, all students received the same types of questions and the same proportion of questions on the different material being tested.

Multiple Choice Questions

When multiple choice questions are used* the question pool will normally contain at least two and one-half times the number of questions any student will receive on his exam. (It is, of course, possible to increase this ratio.) For example, if twenty multiple choice questions will appear on each individual student f s exam, at least fifty questions will be written and programmed . If the multiple choice questions cover five basic topics, then the 50 questions may be

stratified into blocks of 10 questions per topic. Each student will then receive four questions from each block. The four questions to be received are selected by a pre-programmed random number generator.

To further insure the integrity of the multiple choice section each ques- tion is programmed with a correct answer and four incorrect answers. Once a question has been randomly selected for inclusion on a student's examinations the answer set that the student will receive is built internally. From the five programmed answers one is randomly selected to be discarded. The remaining four answers are randomly arranged and shown as A, B, C, or D. Answer E is always shown as ''None of the above." If the correct answer was the discarded answer, then "E" becomes correct. If an incorrect answer was discarded then the correct answer remains in A, B, C, or D and may be selected. (See Exhibit 1.)

If a multiple choice question contains a number as an intergral part of the question, that number is randomly generated. When the question requires a numeric answer every answer In the set (except for "None of the Above") is generated through pre-programmed formulae utilising random number inputs.

Another example of objective type questions which are used is classification of accounts. From a pool of perhaps thirty (30) account titles, the machine randomly selects ten (or any other desired number) accounts for each student. The student may then be asked to classify the account as to type (asset, revenue, etc.), normal balance, or both.

Journal Entries When using journal entries the examiner ha© the option of giving all stu- dents the same questions with the numbers randomly generated for each student, having the machine randomly select a given number of entries for the programmed

A

pool, or a combination of questions randomly selected from a pool and numbers randomly generated for each selected question.

To respond to the journal entry questions, the student instructs the machine as to whether the account and amount ab&ut to be entered should be debited or credited. Secondly the student types in the account title he wishes. (This title, and a machine accepted abbreviations is contained in a long list of pos- sible account titles furnished tfaji student aithar on paper, on line on the com- puter, or both*) The student must also supply the appropriate amount, which may be in the form of an arithmetic expression. The machine accepts debits and credits in any order and will, in the scoring procedures, collapse two simple entries into one compound entry if necessary. (See Exhibits 2 and 3.)

If an account title is not recognized as a possibility, the student is so told. The non-recognition may be from misspelling, an unaccepted abbreviation , transposed words, superfluous words, or unknown words. Spelling, transposed words and superfluous word errors are indicated for what they are. The other errors are simply not accepted and the student must submit an acceptable answer from the previously mentioned list.

Computational Questions

Any ?time a student is required to compute an, answer, he has available two computational methodologies. First, when inserting an answer the student may input an expression such as [(1350 + 750) * 7] instead of 300. (See Exhibit 2.) The answer **II1 be scored >the >same in either case.

If the computation is more complex and the student wants to do it in parts, he may choose to request an on-line calculator. The calculator will retain and display the last calculation made. (Calculations requiring any power or root as well as trigonometric functions -etc., .-.jaay be entered.) (See calculator in Exhibit 1.)

- .

i

c

U

<y

*- —

<0 <~

o

0

X 0

c

*+•> 4J

(0

<0

•—«

•P X

»—•

<0 3

5

«0 <0 0 <U

0 -*■

$- <tf

<u

• •

Cl w

,-r

JU r-i

+»

%z

-*

<0 <53

"O

3

<U <U

c

X $

> >

flj

<o •**

-P $

JZ <u

0.

0

Q

Si

<** r-«

x <u

£5

N4

1J u

*»

OJ

X

M

• X

*~ 0

V

Ds+»

4-*

*

1—4

a

.— t «•*

10 ft

<m

«G 0

£- ••"•

0

*5

<U

5" <u

c c

>v

V u

4-* 0

u

"0 J

u £

«3

A3 <S

»■« <

<u &

ClO

<tf

(

W

"■'. 10

<U >

X -

<d

(0 c 0

U

rj

H |

tf) V

«

V «g

<d jz

*•-<

Q

•-« 3

0

X

+>

4-* »

-M 4->

IX

&t

: >. C

a.

$ ess

Q. <U

«* u%

£

"0

«-•

T5 a;

c

u ft

10

C V

<«

£

iQ U

0 (0

z

&0 ^

g-

o

C «3

ts

n

0

*

••** ^*}

H-

£ 50

•t.

in.

(J)

« C

>■£>

<»»$ o^

Q •■-*

m

0 «*•

o

0

-p

a

0

0

IT*

O Q

4J

a

0

s. u

S3

■P

o

«3

4J

-«-«

0

LA

O

4J Q 3 O Q Q

CL 0

■

"D

t0

Q

0

50

+»

0

c

0 2

y

n n n

o

in

3>

y

«

©

O

y

41

'•.;

H

"1

•Hi

•SI

(0

"0

i

c

c

c

2,

0

- to

«J

3

q "o

V

75

r £

jQ

!~* 0

V

» A

sO

$~

4<

75

«0

— <v

C

H JO

0

to

3 H-

jQ

75

0

C

i-

V

0

H- u>

JO

JC!

i C\3

h-

C la

V

1

•

4-'

JO

v (8

C

i-

£

- L

£

*

*

3 0

4->

IX£

o *••«

tO

•«—-«

IA

>

U

•—•

•— «

t&

>

*S

«s

iO ^

■e

3

10

c

41 erv

-«-*

c

• r*

j-

e

3

<?$ •>

e

<0

<v

0

X —

t

i

+J

»-»

u

V

•»•«

<0

J- c

4-»

c

*~

»-~«

3 *TJ

i

«

«0

a»->

so

to

4-»

$-

c

u

V

» c

0

V

^i

£

0 0

»— 1

•—«

t-

V

u

&

<0

CP

<u

<o

«

£

|l 3

!X

M — *

to

to

«U

«5 >tJ

<d

Cl

X

-H >

H

to

"0

*.

~# <u

.—«

^

»

E 0

<u

CO

IA

j «

.£

r^

0 '--<

&>

1

V

-H

»-»

V <Sj

&

<c

75 *t

n

i aj

0

+*

»-<

z: *■

-p

to

<»

4>

•

J!-

V

J-

c

t.

4»

<C

F- 0

«

4>

»~>

75

V i. u

Q

*

X LA CM IA f

3

O U

u cr

Q

c 41

>

m v t

41

c

»

-p

s>

0

e

»

!t}

r>-.

T?

1T>

j:

»—

<a

^

4-*

Si

10

<o

V

u

4)

V

C

M

3

C

»->

• «-<

c

«M

0

0

4->

■

c

c

*J

0

d

■ (-»

•s-K

0

75

-p

0

41

0

to

fc.

V

.."■«

0

»— <

X5

»— «

o>

0

J-

jc

0

0

4-»

*U

e

«M

JO

3

0

4J

•pH

e

4J

t.

4»

c

0

5»

3

<«

ii

0 £

>s

4»

55

5_

X

^",

P

4)

c

X

<u

0

4j

4$

u

X

c

V

4->

0

4^

■ fi

c

41

4^

LJ

^4

«e

<n

N

r

*-*

h z

3

o

0

o

y

a >

UJ

a

£ O

z

D O £ kl UJ

a y > •

u SI

H

J

y

z

>

>

Z

a

a

h

§

Q

z y

y

h

H

H

a

y

y

h

h

S3

y

5

>

m

u

a

0

G

y

«i

a

0

z

I

z

j

Q

•

0

«

3

•

•*

i

S3 O

5

o

H

td u

trf

CO

c o

-H

to

"0

i

c

c

c

0

-!-»

0

Si

JL

V

* g)

<d

3

o -x>

<U

T?

c c

x«

>-» o

V

- jQ

!f)

i~

V

•g

to

»~» <U

c

JS X

0

fl

3 h-

J2

T5

0

C

i~ •

V

0

t- UN

x

Si

J CM

i—

C UN

<u

• »-i "^*

*

r~

II

4J

p

D (0

C

I.

V

- J-

€

9 9

3 0

-tJ

y<a

0 *»-•

is)

«— i UN

>

It

#■»■-*

— •

S5

>

<d

«d

g) r*

c

3 8

c

V O

•--.

C —»

u

| -•*

c

3

<J •*

s

a3 0*

0

.k. «-•

t-

i +»

*~>

V

01

■•-» «0

u c

4-1

1 L

.— <

3 *

i

fl

CL*~i

D5

10 4J

J-

C

0J

«u

• C

0

V .-'

c

C 0

«— «

-- i.

V

u

J> <0

(J?

V

*0

«j e

V 3

x

Ai »-«

ti

tt u

§ id

*

Ql X

+* >

K

ti

"0

•t

— V

i— e

;? .

r o

V

CD UN

i <g

X

px

<a <~

C»

i

V

-P «-4

U fi)

X?

•0

"0 &>

in &

0

4-* »-•

2T ^

4^

S)

^

(U

X «**

L C

P 0

(0

4J >->

+>«

cb j cnT "

j^*

!

01 +* 3 10

t»

>

+»

*-

0

*=

■

5

53

"0

ON

c

«•"•

*

4^

^

«

oo

v

u

tl

v

e

4J

3

c

H")

• t"«

£»<*

rt

0

'»/

«jj

B

c

C

4J

0

13

• r4

"«■««

0

13

4J

0

W

0

83

S-

4>

- i»^

Q

(p««<

T3

f*»*

V

0

5-

X

0

0

-H

0?

£

*»-t

X

3

0

4-»

•H

£

-H

L

V

c

0

U

5

<M

a.

s

A

<y

<Q

u

X

■+*

4J

V

r~

f

U

0

4}

S!

t-

X

c:

V

~p

0

+>

"•»•

c

01

4.^

LJ

pfJ^i

hj

«d

N

o

m u

j 1-

H H

h J

y Z

y

z

3 O

h

z

faj o: a

D

0

a

»-

z

llj

y y

<l m z h

<r h x

ay

z bJ

y > a

h z

!

d j y y o m a

p 0 . h

§1 f*V

Id

y

y y h h

> x in

b y «

1 z J

Q 0 3 X + i

c o

•H

j— ( 3

y

S3

-J

w iJ

a w

c

w 3

Certain questions are entirely computational in nature. For example, most depreciation entries or Inventory costing entries are rather rote once the cal- culation has been made. Therefore, only the calculation may be desired for testing purposes.

Again;, the examiner has the option of giving all students the same com- putational questions with randomly generated numbers in the question, selecting questions from a pool, or a combination of random questions with random numbers..

Essay Question

At this time, if essay questions are desired,, they are given separately as hand-written exercises. It is possible to allow students to type essay answers onto the machine. However, it would be unnecessarily time consuming as most students do not type as fast as they write.

It should be pointed out that essay questions are not generally used in the introductory accounting course. However, it is felt that* if desired, PLATO could be programmed to grade Accy 101 essay questions (based on key words, phrases, and concepts) with an acceptable degree of consistency.

Student Movement Within the Exam

Students way answer questions in any order desired just as in a written exam. They may look back at questions previously answered or skip a question and coiae back to it later. For example , in the multiple choice sections a student may mark a question for later review whether he originally answered the question or not.

By requesting an index frame the student may see questions answered, ques- tions left blank, or questions marked for later review. With this information in mind the student then has the option of automatically cycling back through all questions, only those questions left blank, or only those questions marked

for review. Additionally, the student may choose to return to any particular question he chooses. (See Exhibits 4 and 5.)

Conclusion of the Exam When the student has concluded the examination (either because he has com- pleted it or because time has been called^ he goes through a procedure which terminates his exam. The final step in this procedure is the typing of "finish." Within 10 seconds of the termination of the exam* the student is shown the number of points he received and the number of points possible. Generally, the student is not shown which questions he missed at this moment because of time constraints. He is, however, given this information later. It is anticipated that in the future students will be allowed to review their graded exam immediately after completing it.

Scoring

Each question of each type (multiple choice, journal, entry* etc.) can be weighted as the examiner desires. In many instances partial credit is availa- ble and, because of the programmed nature, is probably more consistently applied than on hand graded exams. With multiple choice questions the answer is judged correct or incorrect and partial credit is not granted. This is generally the case with all objective questions.

The grading routine for journal entry and computational questions is some- what more complex. Journal entries are divided into at least four (4) parts (debited account, debited amount, credited account, and credited amount) and each part receives a weight. If the entry is fully correct then nothing is deducted. It any part is incorrect then the appropriate weight is deducted. If the entry is correct but the debit and credit are reversed, a stipulated amount Is deducted from the possible score for the question.

id

•0

3

•w*

>

o*

u

0

o <**

• r-4

•>• 4-» s

3 i0 T*

O <U V

v — * £

«— 4 S

0 03

<U M iO

s/ -1 -i

:^0 0

4-> 0

1.0 3

cr

,c o

X 4->

(U

<U

x o

10

f-

0

■p-t

01

3 cr

-j

<E

m -i

o

4-' 0

:\

u

4J 10

0

a

4-5

10

o»

4J i- 0

-P

10

•ft

0

t- a.

o

<♦-* »-» u v a,

10

*

3

V

*#*

>■

01

u.

0

CO CO

!

01

u

V

jC

OJ

o o u

3

o

0«

a.

>

u £ -o

0* id 0*

3 ^4

10 -P £-

£ 0 «

(0 W h

c c c

0 0 0

in jo i.o

0) OJ OJ

15 3 3

cr cr rr

Ul 50 tO

V V V

4_' 4-1 4^

Sid tf

0 0

TJ T« "0

p c e

>

— "J

a

i

?

;

!

1

1

1

!

1

8

1

1

5

1

, J

1

«

1

!

1

i.

v<

•

&

«

ft

■

ft

*

•

•

*

•

1

(r

ft

«

*

■

9

ft

^

iS>

w

CM

CO

1-

IA

«x>

h*.

CD

fjN

^

»-«

CM

CO

'*•

IA

VD

f **•

00

CM

CM

CM

CM

CM

CM

CM

CM

CM

CM

CO

CO

CO

CO

CO

CO

CO

CO

CO

z

5

+

+

+

1

1

■•-

f

j

4

+

i

1

+

■f

+•

4.

8

»

1

H

1 .

n

*

•

»

■

a

ft

»

■

•

0

»

0

■

*

9

ft

ft

ft

fl

•»-«*

CM

CO

■<r

IT.

vO

r-*

CO

O

St

»-«

<M

CO

f

ir»

vD

r>-

00

°\

Id

tt

*

■ *<■«

*-•

•-^

*>4

v«-4

^-i

v-«

¥H

»-i

8

ft* X

e

4J

•H

<U

•»-«

0

X

u

+»

^

«

ID

£

tO v

OJ

OJ

c —

4->

r—»l m m

0 Cl

JZ *>

•l-« •«-!

U

0 u

o

-»-» 4-»

3

i. V

3

tO —

o

cls:

0

0» 3

:x

c

3 z:

51

O i-

0

■<r*

<♦•« ja

TJ

_J 0

— £

«o

_J -M

£

c

o 3

•H

0

<E

<0

5-

v c

c

-*->

0

?<

3

a

V

X &0

<l>

■f

ID £

£

<u

W

<u

tO 4-»

,*

3 0

<*-•

0

fltf «— i

4-> tO

•r-l

V +>

o

-P

-Q

51 <U

t— «

fc.

0 0

«U >

1

X ^

c

=

+* J-

£ C

3

-U [3J

0

X

a

4J H

0

hr

• f«

m

0

tO

:\

V y

+>

•i-«

m a>

a> £ to

r— « gS

0

c

a

> *- <y

75

o n

<u

• rH

0 X

c m x

•—»

:\ ©

to

<i-.

4J 4J

\r-i \— itf

3

u

=

| J—

0

ITk

£ Dfl

3

fix)

CO

Uil

v-«

5- C "D

fU 0 <u

-4-»

vS.

,-J.

»V-J

«->«

h- J 5-

fll

s £-

X

ftfl i~ §}

3

•

>

C 0 <~ 0 <m <U

to

z

c

U <U t-

0

h-4

0 '-« 0

H

M

t*~. JL <•-«

h

+>

£

4J

0.

en

S-

*

tO 51 tO

H

<U

= s

<U U <U

<r

•

-H

■ft •!-♦

0

M

10

.*:

!

to

^4

$. <u $-

«n

u

<0

0

D0

<D

0

-P S~ -P

iLJ

0

<y

o = =

C

-p s

0 *

C 0 c

0

-M

!.-

4-»

0

0

4-»

y eu

in

i-

if)

_J

2

in

• a •

<u

60

«

P4

s

T5 \

C

H

m

«t

O

«

•H

^j

■

0

fLI

J3

•H

O

•S

TJ

0) r-4

W

<U

(X

1

•H

10

s

c

£

fd u

o

V

2

£ 3

«U 10

— c

,

_o us

0

5- 0

a c

ID tO

V <u

4-» 4-»

<d <d

U 0

'ft -t-4

"0 TJ

C 53

»-• »-» »-♦

i i i i t i i i i > i i

(O t Ift

When a question depends upon a preceding question, full credit may be given for the second question even if the answer is technically wrong but is still correct based upon the first answer. For example, assume the correct amount of goods available for sale was $1,000 and (say) question 8 asked for the calculation of cost of goods sold while question 9 called for ending inventory. If the response to 8 was wrong but 9 was computed as $1,000. minus the student's answer to 8, then 9 would re- ceive full credit.

In addition, the machine can be programmed to give partial credit in the event that students give certain anticipated wrong answers. For example, if the ques- tion asks for the calculation of ending inventory and the student gives the correct figure for cost of goods sold, he might receive partial credit.

Recording

Each student's examination is filed in its entirety on disk so that the student can review it later and raise any appropriate questions. Also, the students' scores are listed for the instructor. The instructor may choose to have the list in alphabetical order or numeric rank order and may change the form at will.

For each examination the machine computes and displays the number of per- sons taking the exam, the mean, and the standard deviation. It can also be programmed to give a cumulative weighted average if so desired.

At the option of the instructor the machine will display a score frequency graph, segment students according to pre-set grade break points or prepare and present various types of statistical analysis by section and/or by instructor. (See Exhibits 6 and 7.)

Problems There are certain problems associated with administering examinations on the PLATO system currently. The most pressing problem is the lack of sufficient

a:

'■O

Ln

N (O iA

03} »-« *«*

u o

CO

f CT> CO

N IA N

i CO

cr» co r*-

«-f cvj

(O N Ift

00

a

« M

o

>>

J}

so

(0

Of

w

u

•H •H

3

CM \o CSI

<D vO T

• ■ •

vo in <o

2

72.25 73.50 72.71

z

in cm is. in co oo

31

3

0

U

accypa accybe Tota 1 s

00

4J

C

«H

o

S^

'-•■

* ~

s~«»

5^>

4J

^3

CO *T U "

•s

fcl

-R

«

Oh

g »

3 —-

M

W

JS}

US

«s>

US

tej

UN

«s»

IA

fca

IA

is?

LA

S5

SO

US

ITl

T

<T

CO

CO

(SI

CSf

•*~«

»-«

»)

IS}

8

contiguous terminals to allow everyone to take the examination simultaneously.

This problem is being remedied as more terminals become available. Still the built-in randomisation features protect the Integrity of the examina- tion even though some students take it later than other students.

As a test of the integrity of the examination, a statistical analysis was run on the results of the Spring, 1975, Plato experience. During this term three examinations were administered to 375 students, The first examination was administered on-line; the second was administered on paper, and the comprehensive final examination was divided between on-line and on paper. Due to the limited number of terminals available for testing purposes, the first examination was given at 13 different times over a six (6) consecutive day period. The on-line portion of the final exam was administered at 11 different times over three (3) consecutive day periods* The on paper exams were administered to all students simultaneously.

The data collected for each student included scores for all examinations s the students' weighted-average score for the course, and the hour and day at which the student took the first examination and the hour and day at which the student took the °lato portion of the finl examination*

The relevant correlation matrix is displayed In Table 1. As can be seen8 there is only minor deviation in the correlation numbers between any two examin- ations. Examination 2 (written) and the Plato portion of the final correlate most strongly (.77599) while the first examination (Plato) and the written portion of the final show the smallest correlation number (.65159).

*** Insert Table 1 about here ***

The results shown in Table 2 and Table 3 indicate, as were the authors* priors, that the testing techniques did not produce significantly different results. The within media correlation numbers are included in the range of correlation numbers for across media.

Table 1

Relevant Correlation Matrix

" — '

Exam 1

Exam

Final Exam

Exam 1

Plato

Written

Plato

Time

Exam 1~P

1,00000

Exam 2-Written

.66828

1,00000

Fianl Exam - P

. 70902

.77599

1.00000

Final Exam - W

.65159

.69112

.73777

NA

Exam 1 Time

-0.04711

NA

NA

1.0000

Exam 3 Time

NA

NA

-0.15147

.19732

i

j L-

NA « Not Applicable

*** Insert Tables 2 & 3 about hare ***

Table 1 alsu shows negative correlation between the time a student took

a Plato examination and his score on the examination. These negative correla- tions might indicate that the integrity of the Plato exams was, indeed, intact and that information about the exams was either not passed on to students taking later exams or information which was passed was not of value. However since students were allowed to select the time at which they took the Plato examinations (no attempt was made to randomise the students by ability) * these negative correlations could be caused by better students choosing to take the exams at the earlier times. As shown in Jj|bj-@JL, the correlation between the times individual students took the two Plato examinations is quite low (Exam 1 Time and Exam 3 Time » .19732). This would indicate that good students did not tend to take both Plato exams early while poorer students took them later. We chose s however, to further test the hypothesis that the time at which a student took a Plato examination did not significantly affect his performance on the examination as indicated by his r 'ore on that exam.

*** Insert Table 4 about here ***

An analysis of covar lance, with one. covariate, was run for the pairings shown below:

•"N^ependent Covariate

Exam 1 Paper Final Plato Final

t

Ave_._~Exam X

Exam I

X

Plato Final

JL

Table 2 Across Technique Correlations

Exam 1 - Plato

Exam 3 - Plato

Exam 2 - Written

Exam 4 - Written

.65159

.73777

Table 3 Within Medium Correlations

Exam 1 - Plato

Exam 2 - Written

Exam 3 - Plato

Exam 4 - Written

,70902

KA

NA

j 1, ,

.69112

NA « Not Applicable

10

In each case the predictable portion of the dependent variable was eliminated and the residual (actual score - predicted score) was tested to see if the time when the examination was taken might have caused trend in the residuals. The null hypothesis was that there was no trend, We could not reject this hypothesis at the .10 level. (See Table^A . )

The trend over time of the Plato final (using paper final as covaria£e was only marginally non-significant (p*.13) . An analysis revealed that the trend* if anys in this case would actually be negative. That is, rather than improving one' s score by taking the exam later (see Gra£h_ I) t the student's score actually decreased (see Graph 2). Having already allowed for the student's ability (as estimated by the ccvariate), one would have expected time to have been a positive factor on the unpredicted portion of a student's score if information of value was being passed. Contrary to that expectation, the trend over time of the Plato final (using the paper final as covariate) more closely fits Graph 2 than Graph 1.

We would conclude that, in general, no information of value relative to the content of the Plato examinations was passed from those taking the examina- tion earlier to those taking later examinations. In general, the time at which each student chose to take his Plato examinations had no significant effect upon his score.

Another potential problem occurs if the PLATY system "crashes" while an examination is being given, However the crash amounts to a minor inconvenience since there is an automatic storage routine which updates each student's file every eight (8) minutes. When a crash occurs, it is necessary to allow the students time to re-do a maximum of eight (8) minutes of work which may have been lost in the system malfunction. Such crashes have occurred infrequently and not at all during the last semester use of the testing system. Students are informed, in advance, that there is a possibility of a system malfunction and that, If it occurs, they will be given sufficient time to re-do any lost work and to make up for the system down time.

Table 4

Dependent Variable

s.

u >

8

Plato Final

Exam I

Aug - Exam 1

Paper Final

Exam 1

F - 0.1806 « 0,6712

F * 0.2083

J]u64a4_

NA

4-

-L-

Pl&to Final

NA

F » -0.2369

p * 0.6269

F » 2.2482

p ■ 0.1346

D.F » 1 and 363

NA =» Not Applicable

Time is the independent variable.

Graph 1

u

•a

•H

4! U O

o w

o

•H

I

!

Earlier

Later

Time Period when Exam Taken

If time were a positive factor

Graph 2

Earlier

Later

Time period when exam taken

If time were a negative factor

(This graph greatly exagerates the magnitude of any actual trend and is used only to Illustrate the direction of trend.)

11

Advantages

Obviously the advantages are multiple. Once an examination is constructed s it can be used for several semesters with only minor corrections or changes. Make-up examinations need not be constructed since each exam is unique anyway.

The grading routine provides welcome relief from the usual grading monotony. This is particularly valuable for final examinations when examinations must be graded and final grades computed in very limited time. The instantaneous grading, cumulative weighted average feature, and central tendency and other statistics features makes the job much easier.

Because of the nature of the routine, grading of examinations is both core consistent and more accurate than hand grading methods. Each time PLATO encounters a reversed debit and credit on a certain problem it deducts the same amount. Anyone who has hand graded examinations will recall instances where this was not true with hand grading.

The machine is also quite accurate. It does not look at its answer key, see "B" to be correct, and then not mark a "C" student response wrong because it is tired or its attention was nomentar-' ly diverted.

When numeric answers are required, the machine is programmed to accept answers within a specified range. This allows for rounding too early in the problem or simply rounding incorrectly. However, the acceptable range is kept narrow enough to avoid accidentally close answers*

Because of the computational aids available on-line while the student is taking an examination, it is possible to ask more complex questions than are normal on hand written exams. For example, the student could be asked to solve present value problems and be given a calculator which would provide the appro- priate factor when supplied with rate, periods, and single-value or annuity components. An internal rate of return calculator could be programmed as well.

12

Calculations requiring any power, including fractional powers, are possible. Trigonometric functions are built-in as well. If one wishes to test accounting concepts and methodologies rather than mathematical ability, the PLATO system is an excellent tool.

13

Future Expansion

The Department of Accountancy at UIUC is continuing its experimentation and analysis of the PLATO system as a tool in the accounting curriculum. Ex- periments are planned with the second, accounting course and it is anticipated that more advanced courses will begin to use PLATO as a tool for teaching certain suitable subject areas. Currently, approximately 1200 introductory accounting students are utilizing the PLATO system.

Although this paper has dealt specifically with the Plato system, many of the concepts and achievements are felt to be useful in other CAI systems. It is thought that the detailed analysis presented here may create additional interest in CAI in general as well as provide ideas for those currently using another CAI system.

Footnotes

^McKeown, James C, "Computer-Assisted Instruction for Elementary Accounting," Faculty Working Papers #lr0* College of Comsr«2ree and Business Administration, University of Illinois-Urbana, 1974, also Accounting Reviev January, 1976.

~The covariate Ave. -Exam 1 was a transformation which computed a weighted average comprised only of the last two examinations. This was done in order that the analysis would not be distorted by having the dependent variable included in the covariate.

"eOUNo^

'\Vi r-94