Computerized testing in the beginning accounting course

UNIVERSITY OF

ILLINOIS LIBRARY

AT URBANA-CHAMPAIGN

BOOKSTACKS

Digitized by the Internet Archive

in 2012 with funding from

University of Illinois Urbana-Champaign

http://www.archive.org/details/computerizedtest267burt

Faculty Working Papers

COMPUTERIZED TESTING IN THE
BEGINNING ACCOUNTING COURSE

Eric James Burton, James C. McKeown
Jeffrey Shlosberg

#267

College of Commerce and Business Administration

University of Illinois at Urbana-Champaign

FACULTY WORKING PAPERS
College of Commerce and Business Administration
University of Illinois at Urbana-Champaign
September 10, 1975

COMPUTERIZED TESTING IN THE
BEGINNING ACCOUNTING COURSE

Eric James Burton, James C. McKeown
Jeffrey Shlosberg

#267

Computerised Testing in the
Beginning Accounting Course

Eric James Burton
Ph.D. Candidate, Accountancy
University of Illinois
Urbana-Champaign

James C. McKeowi, Ph.D.

Associate Professor oj. Accountancy

University of Illinois

Urbana-Champaign

Jeffrey Shloeberg
Arthur Andersen & Co«
Minneapolis, Minnesota

August 4, 1975
Draft 3

COMPUTERIZED TESTING IN THE BEGINNING ACCOUNTING COURSE

The purpose of this paper is to report on a unique educational tool being
used by the Department of Accountancy of the University of Illinois at Urbana-
Champaign (U1UC). The PLATO IV (Programmed Logic for Automated Teaching
Operations) computer system is the tool. This note will describe the use of

PLATO for examination purposes. A full discussion of the implementation of

1
PLATO IV as a teaching device is contained in a paper by James C. McKeowa.

How PLATO Works
The PLATO system Is a remote terminal access system. The remote terminals

consist of a keyset and a plasma display panel which are linked to Central Processing
Unit via direct or dial-access telephone lines. Programs are written in a system-
specific languaged called TUTOR.

When he wants to work on PLATO he addresses a terminal by typing in
his name, his course and his password. If the machine accepts these items, the
student is allowed to select lessons from the course in which he is listed.
The student user needs absolutely no programming experience or knowledge.

The student sees, on his plasma panel, a series of frames (analogous to
the pages of a book), Each frame may contain textual material, graphics,
journal entries, questions, or combinations of these. When questions are
asked in a lesson, the student must type in his response. If his response is
correct, it is acknowledged and the student may proceed. If it is incorrect,
a wrong answer response is shown and the student must try again. The objective
of this procedure is a continuous check on the student's understanding. In
many instances, if the student can not obtain the correct answer alone there is
a programmed in HELP sequence to assist him in understanding the material and
obtaining a correct answer.

i

Accy 101 Experience
Since the Summer Semester of 1973 the UIUC Department of Accountancy has
experimented with the PLATO system in the introductory course (Accy 101).

During the Fall Semester 1973, a Latin-squares design experiment was conducted
to determine the effectiveness of the system and the 26 programmed lessons as
a teaching tool (see McKeown), Simply said, the results encouraged expansion
and further experimentation.

During the Spring, 1974, the Department continued informal experimentation
on a student self-paced basis. Through that Spring Semester and the following Summer
Semester an examination procedure was developed. In the Fall, 1974, the examina-
tions were revised and utilized in four (4) sections of Accountancy 101.

PLATO _ Examinations
Three PLATO examinations were given during the Fall, 1974 semester. The first examin-
ation consisted of multiple choice questions s other objective types of questions, and

journal entries. The second exam Included multiple choice questions, journal
entries, and computational questions. The final exam was approximately a 60-40

mix of material covered after the second exam and material covered on the first

two exams. It was comprised of multiple choice questions, journal entries,

and computational questions.

Each student received a unique examination. However, all students
received the same types of questions and the same proportion of questions on
the different material being tested.

Multiple Choice Questions

When multiple choice questions are used* the question pool will normally
contain at least two and one-half times the number of questions any student will
receive on his exam. (It is, of course, possible to increase this
ratio.) For example, if twenty multiple choice questions will appear on each
individual student f s exam, at least fifty questions will be written and programmed . If
the multiple choice questions cover five basic topics, then the 50 questions may be

stratified into blocks of 10 questions per topic. Each student will then receive
four questions from each block. The four questions to be received are selected
by a pre-programmed random number generator.

To further insure the integrity of the multiple choice section each ques-
tion is programmed with a correct answer and four incorrect answers. Once a
question has been randomly selected for inclusion on a student's examinations
the answer set that the student will receive is built internally. From the
five programmed answers one is randomly selected to be discarded. The remaining
four answers are randomly arranged and shown as A, B, C, or D. Answer E is
always shown as ''None of the above." If the correct answer was the discarded
answer, then "E" becomes correct. If an incorrect answer was discarded then the
correct answer remains in A, B, C, or D and may be selected. (See Exhibit 1.)

If a multiple choice question contains a number as an intergral part of the
question, that number is randomly generated. When the question requires a
numeric answer every answer In the set (except for "None of the Above") is
generated through pre-programmed formulae utilising random number inputs.

Another example of objective type questions which are used is classification
of accounts. From a pool of perhaps thirty (30) account titles, the machine
randomly selects ten (or any other desired number) accounts for each student.
The student may then be asked to classify the account as to type (asset, revenue,
etc.), normal balance, or both.

Journal Entries
When using journal entries the examiner ha© the option of giving all stu-
dents the same questions with the numbers randomly generated for each student,
having the machine randomly select a given number of entries for the programmed

A

pool, or a combination of questions randomly selected from a pool and numbers
randomly generated for each selected question.

To respond to the journal entry questions, the student instructs the machine
as to whether the account and amount ab&ut to be entered should be debited or
credited. Secondly the student types in the account title he wishes. (This
title, and a machine accepted abbreviations is contained in a long list of pos-
sible account titles furnished tfaji student aithar on paper, on line on the com-
puter, or both*) The student must also supply the appropriate amount, which
may be in the form of an arithmetic expression. The machine accepts debits
and credits in any order and will, in the scoring procedures, collapse two
simple entries into one compound entry if necessary. (See Exhibits 2 and 3.)

If an account title is not recognized as a possibility, the student is so
told. The non-recognition may be from misspelling, an unaccepted abbreviation ,
transposed words, superfluous words, or unknown words. Spelling, transposed
words and superfluous word errors are indicated for what they are. The other
errors are simply not accepted and the student must submit an acceptable answer
from the previously mentioned list.

Computational Questions

Any ?time a student is required to compute an, answer, he has available two
computational methodologies. First, when inserting an answer the student may
input an expression such as [(1350 + 750) * 7] instead of 300. (See Exhibit 2.)
The answer **II1 be scored >the >same in either case.

If the computation is more complex and the student wants to do it in parts,
he may choose to request an on-line calculator. The calculator will retain and
display the last calculation made. (Calculations requiring any power or root
as well as trigonometric functions -etc., .-.jaay be entered.) (See calculator in
Exhibit 1.)

- .

i

c

U

<y

*- —

<0 <~

o

0

X 0

c

*+•> 4J

(0

<0

•—«

•P X

»—•

<0 3

5

«0 <0
0 <U

0 -*■

$- <tf

<u

• •

Cl w

,-r

JU r-i

+»

%z

-*

<0 <53

"O

3

<U <U

c

X $

> >

flj

<o •**

-P $

JZ <u

0.

0

Q

Si

<** r-«

x <u

£5

N4

1J u

*»

OJ

X

M

• X

*~ 0

V

Ds+»

4-*

*

1—4

a

.— t «•*

10 ft

<m

«G 0

£- ••"•

0

*5

<U

5" <u

c c

>v

V u

4-* 0

u

"0 J

u £

«3

A3 <S

»■« <

<u &

ClO

<tf

(

W

"■'. 10

<U >

X -

<d

(0 c
0

U

rj

H |

tf) V

«

V «g

<d jz

*•-<

Q

•-« 3

0

X

+>

4-* »

-M 4->

IX

&t

: >. C

a.

$ ess

Q. <U

«* u%

£

"0

«-•

T5 a;

c

u ft

10

C V

<«

£

iQ U

0 (0

z

&0 ^

g-

o

C «3

ts

n

0

*

••** ^*}

H-

£ 50

•t.

in.

(J)

« C

>■£>

<»»$ o^

Q •■-*

m

0 «*•

o

0

-p

a

0

0

IT*

O
Q

4J

a

0

s.
u

S3

■P

o

«3

4J

-«-«

0

LA

O

4J
Q
3
O
Q
Q

CL
0

■

"D

t0

Q

0

50

+»

0

c

0
2

y

n n n

o

in

3>

y

«

©

O

y

41

'•.;

H

"1

•Hi

•SI

(0

"0

i

c

c

c

2,

0

- to

«J

3

q "o

V

75

r £

jQ

!~* 0

V

» A

sO

$~

4<

75

«0

— <v

C

H JO

0

to

3 H-

jQ

75

0

C

i-

V

0

H- u>

JO

JC!

i C\3

h-

C la

V

1

•

4-'

JO

v (8

C

i-

£

- L

£

*

*

3 0

4->

IX£

o *••«

tO

•«—-«

IA

>

U

•—•

•— «

t&

>

*S

«s

iO ^

■e

3

10

c

41 erv

-«-*

c

• r*

j-

e

3

<?$ •>

e

<0

<v

0

X —

t

i

+J

»-»

u

V

•»•«

<0

J- c

4-»

c

*~

»-~«

3 *TJ

i

«

«0

a»->

so

to

4-»

$-

c

u

V

» c

0

V

^i

£

0 0

»— 1

•—«

t-

V

u

&

<0

CP

<u

<o

«

£

|l 3

!X

M — *

to

to

«U

«5 >tJ

<d

Cl

X

-H >

H

to

"0

*.

~# <u

.—«

^

»

E 0

<u

CO

IA

j «

.£

r^

0 '--<

&>

1

V

-H

»-»

V <Sj

&

<c

75 *t

n

i aj

0

+*

»-<

z: *■

-p

to

<»

4>

•

J!-

V

J-

c

t.

4»

<C

F- 0

«

4>

»~>

75

V
i.
u

Q

*

X
LA
CM
IA
f

3

O
U

u
cr

Q

c
41

>

m
v
t

41

c

»

-p

s>

0

e

»

!t}

r>-.

T?

1T>

j:

»—

<a

^

4-*

Si

10

<o

V

u

4)

V

C

M

3

C

»->

• «-<

c

«M

0

0

4->

■

c

c

*J

0

d

■ (-»

•s-K

0

75

-p

0

41

0

to

fc.

V

.."■«

0

»— <

X5

»— «

o>

0

J-

jc

0

0

4-»

*U

e

«M

JO

3

0

4J

•pH

e

4J

t.

4»

c

0

5»

3

<«

ii

0
£

>s

4»

55

5_

X

^",

P

4)

c

X

<u

0

4j

4$

u

X

c

V

4->

0

4^

■ fi

c

41

4^

LJ

^4

«e

<n

N

r

*-*

h
z

3

o

0

o

y

a
>

UJ

a

£
O

z

D
O
£ kl UJ

a y
> •

u
SI

H

J

y

z

>

>

Z

a

a

h

§

Q

z
y

y

h

H

H

a

y

y

h

h

S3

y

5

>

m

u

a

0

G

y

«i

a

0

z

I

z

j

Q

•

0

«

3

•

•*

i

S3
O

5

o

H

td
u

trf

CO

c
o

-H

to

"0

i

c

c

c

0

-!-»

0

Si

JL

V

* g)

<d

3

o -x>

<U

T?

c c

x«

>-» o

V

- jQ

!f)

i~

V

•g

to

»~» <U

c

JS X

0

fl

3 h-

J2

T5

0

C

i~ •

V

0

t- UN

x

Si

J CM

i—

C UN

<u

• »-i "^*

*

r~

II

4J

p

D (0

C

I.

V

- J-

€

9 9

3 0

-tJ

y<a

0 *»-•

is)

«— i UN

>

It

#■»■-*

— •

S5

>

<d

«d

g) r*

c

3 8

c

V O

•--.

C —»

u

| -•*

c

3

<J •*

s

a3 0*

0

.k. «-•

t-

i +»

*~>

V

01

■•-» «0

u c

4-1

1 L

.— <

3 *

i

fl

CL*~i

D5

10 4J

J-

C

0J

«u

• C

0

V .-'

c

C 0

«— «

-- i.

V

u

J> <0

(J?

V

*0

«j e

V 3

x

Ai »-«

ti

tt u

§ id

*

Ql X

+* >

K

ti

"0

•t

— V

i— e

;? .

r o

V

CD UN

i <g

X

px

<a <~

C»

i

V

-P «-4

U fi)

X?

•0

"0 &>

in &

0

4-* »-•

2T ^

4^

S)

^

(U

X «**

L C

P 0

(0

4J >->

+>«

cb j cnT "

j^*

!

01 +*
3 10

t»

>

+»

*-

0

*=

■

5

53

"0

ON

c

«•"•

*

4^

^

«

oo

v

u

tl

v

e

4J

3

c

H")

• t"«

£»<*

rt

0

'»/

«jj

B

c

C

4J

0

13

• r4

"«■««

0

13

4J

0

W

0

83

S-

4>

- i»^

Q

(p««<

T3

f*»*

V

0

5-

X

0

0

-H

0?

£

*»-t

X

3

0

4-»

•H

£

-H

L

V

c

0

U

5

<M

a.

s

A

<y

<Q

u

X

■+*

4J

V

r~

f

U

0

4}

S!

t-

X

c:

V

~p

0

+>

"•»•

c

01

4.^

LJ

pfJ^i

hj

«d

N

o

m
u

j 1-

H H

h J

y
Z

y

z

3
O

h

z

faj
o:
a

D

0

a

»-

z

llj

y y

<l m z h

<r h x

ay

z bJ

y > a

h
z

!

d j y y
o m a

p 0 . h

§1 f*V

Id

y

y y h h

> x in

b y «

1 z J

Q 0 3 X + i

c
o

•H

j— (
3

y

S3

-J

w
iJ

a
w

c

w
3

Certain questions are entirely computational in nature. For example, most
depreciation entries or Inventory costing entries are rather rote once the cal-
culation has been made. Therefore, only the calculation may be desired for
testing purposes.

Again;, the examiner has the option of giving all students the same com-
putational questions with randomly generated numbers in the question, selecting
questions from a pool, or a combination of random questions with random numbers..

Essay Question

At this time, if essay questions are desired,, they are given separately as
hand-written exercises. It is possible to allow students to type essay answers
onto the machine. However, it would be unnecessarily time consuming as most
students do not type as fast as they write.

It should be pointed out that essay questions are not generally used in
the introductory accounting course. However, it is felt that* if desired, PLATO
could be programmed to grade Accy 101 essay questions (based on key words,
phrases, and concepts) with an acceptable degree of consistency.

Student Movement Within the Exam

Students way answer questions in any order desired just as in a written
exam. They may look back at questions previously answered or skip a question
and coiae back to it later. For example , in the multiple choice sections a
student may mark a question for later review whether he originally answered
the question or not.

By requesting an index frame the student may see questions answered, ques-
tions left blank, or questions marked for later review. With this information
in mind the student then has the option of automatically cycling back through
all questions, only those questions left blank, or only those questions marked

for review. Additionally, the student may choose to return to any particular
question he chooses. (See Exhibits 4 and 5.)

Conclusion of the Exam
When the student has concluded the examination (either because he has com-
pleted it or because time has been called^ he goes through a procedure which
terminates his exam. The final step in this procedure is the typing of
"finish." Within 10 seconds of the termination of the exam* the student is
shown the number of points he received and the number of points possible.
Generally, the student is not shown which questions he missed at this moment
because of time constraints. He is, however, given this information later.
It is anticipated that in the future students will be allowed to review their
graded exam immediately after completing it.

Scoring

Each question of each type (multiple choice, journal, entry* etc.) can be
weighted as the examiner desires. In many instances partial credit is availa-
ble and, because of the programmed nature, is probably more consistently applied
than on hand graded exams. With multiple choice questions the answer is judged
correct or incorrect and partial credit is not granted. This is generally the
case with all objective questions.

The grading routine for journal entry and computational questions is some-
what more complex. Journal entries are divided into at least four (4) parts
(debited account, debited amount, credited account, and credited amount) and
each part receives a weight. If the entry is fully correct then nothing is
deducted. It any part is incorrect then the appropriate weight is deducted.
If the entry is correct but the debit and credit are reversed, a stipulated
amount Is deducted from the possible score for the question.

id

•0

3

•w*

>

o*

u

0

o <**

• r-4

•>• 4-» s

3 i0 T*

O <U V

v — * £

«— 4 S

0 03

<U M iO

s/ -1 -i

:^0 0

4->
0

1.0
3

cr

,c o

X 4->

(U

<U

x o

10

f-

0

■p-t

01

3
cr

-j

<E

m
-i

o

4-'
0

:\

u

4J
10

0

a

4-5

10

o»

4J
i-
0

-P

10

•ft

0

t-
a.

o

<♦-*
»-»
u
v
a,

10

*

3

V

*#*

>■

01

u.

0

CO
CO

!

01

u

V

jC

OJ

o
o
u

3

o

0«

a.

>

u £ -o

0* id 0*

3 ^4

10 -P £-

£ 0 «

(0 W h

c c c

0 0 0

in jo i.o

0) OJ OJ

15 3 3

cr cr rr

Ul 50 tO

V V V

4_' 4-1 4^

Sid tf

0 0

TJ T« "0

p c e

>

— "J

a

i

?

;

!

1

1

1

!

1

8

1

1

5

1

, J

1

«

1

!

1

i.

v<

•

&

«

ft

■

ft

*

•

•

*

•

1

(r

ft

«

*

■

9

ft

^

iS>

w

CM

CO

1-

IA

«x>

h*.

CD

fjN

^

»-«

CM

CO

'*•

IA

VD

f **•

00

CM

CM

CM

CM

CM

CM

CM

CM

CM

CM

CO

CO

CO

CO

CO

CO

CO

CO

CO

z

5

+

+

+

1

1

■•-

f

j

4

+

i

1

+

■f

+•

4.

8

»

1

H

1 .

n

*

•

»

■

a

ft

»

■

•

0

»

0

■

*

9

ft

ft

ft

fl

•»-«*

CM

CO

■<r

IT.

vO

r-*

CO

O

St

»-«

<M

CO

f

ir»

vD

r>-

00

°\

Id

tt

*

■ *<■«

*-•

•-^

*>4

v«-4

^-i

v-«

¥H

»-i

8

ft*
X

e

4J

•H

<U

•»-«

0

X

u

+»

^

«

ID

£

tO v

OJ

OJ

c —

4->

r—»l m m

0 Cl

JZ *>

•l-« •«-!

U

0 u

o

-»-» 4-»

3

i. V

3

tO —

o

cls:

0

0» 3

:x

c

3 z:

51

O i-

0

■<r*

<♦•« ja

TJ

_J 0

— £

«o

_J -M

£

c

o 3

•H

0

<E

<0

5-

v c

c

-*->

0

?<

3

a

V

X &0

<l>

■f

ID £

£

<u

W

<u

tO 4-»

,*

3 0

<*-•

0

fltf «— i

4-> tO

•r-l

V +>

o

-P

-Q

51 <U

t— «

fc.

0 0

«U >

1

X ^

c

=

+* J-

£ C

3

-U [3J

0

X

a

4J H

0

hr

• f«

m

0

tO

:\

V y

+>

•i-«

m a>

a> £ to

r— « gS

0

c

a

> *- <y

75

o n

<u

• rH

0 X

c m x

•—»

:\ ©

to

<i-.

4J 4J

\r-i \— itf

3

u

=

| J—

0

ITk

£ Dfl

3

fix)

CO

Uil

v-«

5- C "D

fU 0 <u

-4-»

vS.

,-J.

»V-J

«->«

h- J 5-

fll

s £-

X

ftfl i~ §}

3

•

>

C 0 <~
0 <m <U

to

z

c

U <U t-

0

h-4

0 '-« 0

H

M

t*~. JL <•-«

h

+>

£

4J

0.

en

S-

*

tO 51 tO

H

<U

= s

<U U <U

<r

•

-H

■ft •!-♦

0

M

10

.*:

!

to

^4

$. <u $-

«n

u

<0

0

D0

<D

0

-P S~ -P

iLJ

0

<y

o = =

C

-p s

0 *

C 0 c

0

-M

!.-

4-»

0

0

4-»

y eu

in

i-

if)

_J

2

in

• a •

<u

60

«

P4

s

T5 \

C

H

m

«t

O

«

•H

^j

■

0

fLI

J3

•H

O

•S

TJ

0)
r-4

W

<U

(X

1

•H

10

s

c

£

fd u

o

V

2

£ 3

«U 10

— c

,

_o us

0

5- 0

a c

ID tO

V <u

4-» 4-»

<d <d

U 0

'ft -t-4

"0 TJ

C 53

»-• »-» »-♦

i i i i t i i i i > i i

(O t Ift

When a question depends upon a preceding question, full credit may be
given for the second question even if the answer is technically wrong but is
still correct based upon the first answer. For example, assume the correct amount of
goods available for sale was $1,000 and (say) question 8 asked for the calculation of cost
of goods sold while question 9 called for ending inventory. If the response to 8 was wrong
but 9 was computed as $1,000. minus the student's answer to 8, then 9 would re-
ceive full credit.

In addition, the machine can be programmed to give partial credit in the event
that students give certain anticipated wrong answers. For example, if the ques-
tion asks for the calculation of ending inventory and the student gives the
correct figure for cost of goods sold, he might receive partial credit.

Recording

Each student's examination is filed in its entirety on disk so that the
student can review it later and raise any appropriate questions. Also, the
students' scores are listed for the instructor. The instructor may choose to
have the list in alphabetical order or numeric rank order and may change the
form at will.

For each examination the machine computes and displays the number of per-
sons taking the exam, the mean, and the standard deviation. It can also be
programmed to give a cumulative weighted average if so desired.

At the option of the instructor the machine will display a score frequency
graph, segment students according to pre-set grade break points or prepare and
present various types of statistical analysis by section and/or by instructor. (See
Exhibits 6 and 7.)

Problems
There are certain problems associated with administering examinations on
the PLATO system currently. The most pressing problem is the lack of sufficient

a:

'■O

Ln

N (O iA

03} »-« *«*

u
o

CO

f CT> CO

N IA N

i CO

cr» co r*-

«-f cvj

(O N Ift

00

a

«
M

o

>>

J}

so

(0

Of

w

u

•H
•H

3

CM \o CSI

<D vO T

• ■ •

vo in <o

2

72.25
73.50
72.71

z

in cm is.
in co oo

31

3

0

U

accypa
accybe
Tota 1 s

00

4J

C

«H

o

S^

'-•■

* ~

s~«»

5^>

4J

^3

CO *T
U "

•s

fcl

-R

«

Oh

g »

3 —-

M

W

JS}

US

«s>

US

tej

UN

«s»

IA

fca

IA

is?

LA

S5

SO

US

ITl

T

<T

CO

CO

(SI

CSf

•*~«

»-«

»)

IS}

8

contiguous terminals to allow everyone to take the examination simultaneously.

This problem is being remedied as more terminals become available.
Still the built-in randomisation features protect the Integrity of the examina-
tion even though some students take it later than other students.

As a test of the integrity of the examination, a statistical analysis was run on the results
of the Spring, 1975, Plato experience. During this term three examinations were administered
to 375 students, The first examination was administered on-line; the second was
administered on paper, and the comprehensive final examination was divided
between on-line and on paper. Due to the limited number of terminals available
for testing purposes, the first examination was given at 13 different times
over a six (6) consecutive day period. The on-line portion of the final exam
was administered at 11 different times over three (3) consecutive day periods*
The on paper exams were administered to all students simultaneously.

The data collected for each student included scores for all examinations s
the students' weighted-average score for the course, and the hour and day at
which the student took the first examination and the hour and day at which the
student took the °lato portion of the finl examination*

The relevant correlation matrix is displayed In Table 1. As can be seen8
there is only minor deviation in the correlation numbers between any two examin-
ations. Examination 2 (written) and the Plato portion of the final correlate
most strongly (.77599) while the first examination (Plato) and the written
portion of the final show the smallest correlation number (.65159).

*** Insert Table 1 about here ***

The results shown in Table 2 and Table 3 indicate, as were the authors*
priors, that the testing techniques did not produce significantly different
results. The within media correlation numbers are included in the range of
correlation numbers for across media.

Table 1

Relevant Correlation Matrix

" — '

Exam 1

Exam

Final Exam

Exam 1

Plato

Written

Plato

Time

Exam 1~P

1,00000

Exam 2-Written

.66828

1,00000

Fianl Exam - P

. 70902

.77599

1.00000

Final Exam - W

.65159

.69112

.73777

NA

Exam 1 Time

-0.04711

NA

NA

1.0000

Exam 3 Time

NA

NA

-0.15147

.19732

i

j L-

NA « Not Applicable

*** Insert Tables 2 & 3 about hare ***

Table 1 alsu shows negative correlation between the time a student took

a Plato examination and his score on the examination. These negative correla-
tions might indicate that the integrity of the Plato exams was, indeed, intact
and that information about the exams was either not passed on to students
taking later exams or information which was passed was not of value. However
since students were allowed to select the time at which they took the Plato
examinations (no attempt was made to randomise the students by ability) * these
negative correlations could be caused by better students choosing to take the
exams at the earlier times. As shown in Jj|bj-@JL, the correlation between the
times individual students took the two Plato examinations is quite low (Exam 1
Time and Exam 3 Time » .19732). This would indicate that good students did
not tend to take both Plato exams early while poorer students took them later.
We chose s however, to further test the hypothesis that the time at which a
student took a Plato examination did not significantly affect his performance
on the examination as indicated by his r 'ore on that exam.

*** Insert Table 4 about here ***

An analysis of covar lance, with one. covariate, was run for the pairings
shown below:

•"N^ependent
Covariate

Exam 1
Paper Final
Plato Final

t

Ave_._~Exam X

Exam I

X

Plato Final

JL

Table 2
Across Technique Correlations

Exam 1 - Plato

Exam 3 - Plato

Exam 2 - Written

Exam 4 - Written

.65159

.73777

Table 3
Within Medium Correlations

Exam 1 - Plato

Exam 2 - Written

Exam 3 - Plato

Exam 4 - Written

,70902

KA

NA

j
1, ,

.69112

NA « Not Applicable

10

In each case the predictable portion of the dependent variable was eliminated and
the residual (actual score - predicted score) was tested to see if the time
when the examination was taken might have caused trend in the residuals. The
null hypothesis was that there was no trend, We could not reject this hypothesis
at the .10 level. (See Table^A . )

The trend over time of the Plato final (using paper final as covaria£e was
only marginally non-significant (p*.13) . An analysis revealed that the trend* if anys
in this case would actually be negative. That is, rather than improving one' s score
by taking the exam later (see Gra£h_ I) t the student's score actually decreased
(see Graph 2). Having already allowed for the student's ability (as estimated by
the ccvariate), one would have expected time to have been a positive factor on
the unpredicted portion of a student's score if information of value was being
passed. Contrary to that expectation, the trend over time of the Plato final
(using the paper final as covariate) more closely fits Graph 2 than Graph 1.

We would conclude that, in general, no information of value relative to
the content of the Plato examinations was passed from those taking the examina-
tion earlier to those taking later examinations. In general, the time at
which each student chose to take his Plato examinations had no significant
effect upon his score.

Another potential problem occurs if the PLATY system "crashes" while an examination
is being given, However the crash amounts to a minor inconvenience since there is an
automatic storage routine which updates each student's file every eight (8) minutes.
When a crash occurs, it is necessary to allow the students time to re-do a maximum of
eight (8) minutes of work which may have been lost in the system malfunction. Such
crashes have occurred infrequently and not at all during the last semester use of the
testing system. Students are informed, in advance, that there is a possibility of a
system malfunction and that, If it occurs, they will be given sufficient time to
re-do any lost work and to make up for the system down time.

Table 4

Dependent Variable

s.

u
>

8

Plato Final

Exam I

Aug - Exam 1

Paper Final

Exam 1

F - 0.1806
« 0,6712

F * 0.2083

J]u64a4_

NA

4-

-L-

Pl&to Final

NA

F » -0.2369

p * 0.6269

F » 2.2482

p ■ 0.1346

D.F » 1 and 363

NA =» Not Applicable

Time is the independent variable.

Graph 1

u

•a

•H

4!
U
O

o
w

o

•H

I

!

Earlier

Later

Time Period when Exam Taken

If time were a positive factor

Graph 2

Earlier

Later

Time period when exam taken

If time were a negative factor

(This graph greatly exagerates the magnitude of any actual trend and
is used only to Illustrate the direction of trend.)

11

Advantages

Obviously the advantages are multiple. Once an examination is constructed s
it can be used for several semesters with only minor corrections or changes.
Make-up examinations need not be constructed since each exam is unique anyway.

The grading routine provides welcome relief from the usual grading monotony.
This is particularly valuable for final examinations when examinations must be
graded and final grades computed in very limited time. The instantaneous grading,
cumulative weighted average feature, and central tendency and other statistics
features makes the job much easier.

Because of the nature of the routine, grading of examinations is both
core consistent and more accurate than hand grading methods. Each time PLATO
encounters a reversed debit and credit on a certain problem it deducts the same
amount. Anyone who has hand graded examinations will recall instances where
this was not true with hand grading.

The machine is also quite accurate. It does not look at its answer key,
see "B" to be correct, and then not mark a "C" student response wrong because
it is tired or its attention was nomentar-' ly diverted.

When numeric answers are required, the machine is programmed to accept
answers within a specified range. This allows for rounding too early in the
problem or simply rounding incorrectly. However, the acceptable range is kept
narrow enough to avoid accidentally close answers*

Because of the computational aids available on-line while the student is
taking an examination, it is possible to ask more complex questions than are
normal on hand written exams. For example, the student could be asked to solve
present value problems and be given a calculator which would provide the appro-
priate factor when supplied with rate, periods, and single-value or annuity
components. An internal rate of return calculator could be programmed as well.

12

Calculations requiring any power, including fractional powers, are possible.
Trigonometric functions are built-in as well. If one wishes to test accounting
concepts and methodologies rather than mathematical ability, the PLATO system
is an excellent tool.

13

Future Expansion

The Department of Accountancy at UIUC is continuing its experimentation
and analysis of the PLATO system as a tool in the accounting curriculum. Ex-
periments are planned with the second, accounting course and it is anticipated
that more advanced courses will begin to use PLATO as a tool for teaching certain
suitable subject areas. Currently, approximately 1200 introductory accounting
students are utilizing the PLATO system.

Although this paper has dealt specifically with the Plato system, many
of the concepts and achievements are felt to be useful in other CAI systems.
It is thought that the detailed analysis presented here may create additional
interest in CAI in general as well as provide ideas for those currently using
another CAI system.

Footnotes

^McKeown, James C, "Computer-Assisted Instruction for Elementary
Accounting," Faculty Working Papers #lr0* College of Comsr«2ree and Business
Administration, University of Illinois-Urbana, 1974, also Accounting Reviev
January, 1976.

~The covariate Ave. -Exam 1 was a transformation which computed a weighted
average comprised only of the last two examinations. This was done in order
that the analysis would not be distorted by having the dependent variable
included in the covariate.

"eOUNo^

'\Vi
r-94