Electronic Homework

Fong-lok Lee & Rex M. Heyworth

The Chinese University of Hong Kong

Electronic Homework

Fong-lok Lee & Rex M. Heyworth

The Chinese University of Hong Kong

Homework, traditionally done with pencil and paper, can be an effective means of helping students to consolidate what they have learned.. When students have problems, some fortunate ones may have immediate assistance from their parents, siblings or others who act as human tutors. However, most of them have to wait until the next day before they can ask for their teachers’ help, so while they are doing their homework, they must resort to other means of overcoming any difficulty which, if unresolved, may become sources of later errors [1]. It would be desirable if a personal tutor could be made available to help each student when they encounter homework problems.

A Personal Tutor for Students

There are good reasons why a personal tutor should be able to help students of varied abilities. For better students, immediate feedback should be provided to strengthen the learning effect. For students of lower ability, misconceptions or errors should be corrected immediately to prevent them becoming stable errors [1]. If and when students do not know how to continue, immediate help should be provided so that students' correct behavior will be reinforced and incorrect behavior avoided. Ideally, this should be done by human tutors, but if human tutors are not available, computer tutors that can act like human tutors may be an alternative solution.

To provide a computer tutor to each student is easier said than done. The necessary condition is that each student should have a personal computer at home, a condition which is not fully met at the present time. However, with the increasing prosperity of society and the lowering of the price of personal computers, it is possible that in the coming few years, this condition will be satisfied.

The biggest handicap is the lack of suitable software, particularly in the area of mathematics. Most currently available computer-assisted instructional systems in mathematics focus on tutorial and drill-and-practice functions [2]. Students are given exercises to practice, either in the form of multiple choice, or short questions for which only short answers are expected. The purpose of this kind of software is to drill the students until they reach a certain degree of competence which is measured by a test similar to the exercises. Those who pass the test are allowed to go on to the next part of the system whilest students who cannot pass the test will be asked to either revise certain parts of the material or be given additional materials to read until they master the subject. There is no attempt to understand individual students' errors and all remedial measures are prespecified.

What a Homework System Should Do

Recently, computer systems have begun to attempt to understand students’ errors, and then provide assistance to help students overcome them. Examples can be found in the area of linear algebraic equations in one variable [3][4][5], geometry [6] and calculus [7]. However, to be a true homework assistant system, the following conditions have to be satisfied in addition to providing tutoring to students:

1. Teachers or even students should be allowed to enter problems that they would like to have as homework. To achieve this, the computer software has to be able to find the solution to any problem entered as well as detect any errors made by the student during the problem solving process.
2. Besides helping students, the system should help teachers by scoring students’ work. Furthermore, it should help teachers to better understand their students by collecting and summarizing their errors and weaknesses shown in the students’ work. This frees the teacher from much tedious routine work, but still allows them to quickly ascertain the students’ progress.

Our Electronic Homework designed to achieve this purpose is composed of two components: the Computer Tutor and the Homework Administrator. The Computer Tutor is an intelligent tutoring system that can provide personal assistance like supplying hints, checking errors and providing remediation. The Homework Administrator is a teacher’s assistant which assigns and marks the homework, summarizing errors for the teacher’s reference and prioritizing problems. When using Electronic Homework, teachers would simply assign homework by distributing floppy disks containing the assignment for students to do at home. Students can work at their own pace under the guidance of the computer tutor, and next day when they return the disks to school, teachers do not have to mark or correct the homework because it has already been done by Electronic Homework. However, teachers will, after collecting the disks, have a clear picture of how the work was done by having the computer summarize students’ progress and displaying it on the screen. Teachers therefore have more time to focus attention on improving their teaching.

System Design

In order that Electronic Homework can achieve the above purposes, both the components -- the Computer Tutor and the Homework Administrator – are themselves composed of several modules. The following sections describe the content and function of these modules.

Knowledge Stored in the Computer Tutor

So that the Computer Tutor can diagnose students’ errors as well as describe them, different types of knowledge, mainly obtained from human experts, has to be incorporated. To allow for future expansion, the pieces of knowledge, in the form of rules, are stored in separate modules. Each module is described in more details in the following paragraphs.

The Expert Module

This contains the knowledge that the system imparts to the student. It is called an expert module since it includes what an expert in the subject area concerned should know. As the present system is intended to teach logarithmic knowledge, it comprises mainly knowledge required to solve logarithm problems. However, solving logarithm problems may require other mathematics knowledge such as solving algebraic equations, simplifying algebraic expressions and factorizing numbers or algebraic expressions. The knowledge base therefore includes a large number of rules.

In Electronic Homework, there are two types of rules: the strategic rules and the axiomatic rules, as suggested by Lewis, Milson, & Anderson [5]. Strategic rules state what strategies would be used whenever certain patterns are observed, while axiomatic rules correspond to behaviors according to mathematics axioms. The following examples found in The Teacher's Apprentice [5] serve to illustrate this difference:

(Words in italics are variables.)

In the above examples, the rule [R1] recognizes that distribution is applicable to the equation and sets the subgoal to distribute num over term1 and term2. It is a strategic rule. The other three are axiomatic rules since they show the actions according to distributive law, multiplication and addition facts respectively.

Strategic rules and axiomatic rules, together, form the domain knowledge base of the system. However, according to Roberts & Park [8], besides this domain knowledge base, there should be one more element which they call the criterion-performance model. The domain knowledge base includes both the knowledge of the contents to be taught and the knowledge on how to use the content knowledge to solve related problems. The criterion-performance model is a computer-based expert that solves the same problem which the student is working on so that the system can evaluate the student’s performance.

Criterion-performance model Versus Model-tracing

There has been some argument as to whether the criterion-performance model is necessary for a computer tutor. An example of systems employing this model is PIXIE [3][4]. In this system, the solutions to a problem, whether correct or incorrect, are generated before the problem is presented to the students. Students' answers are then compared with these generated solutions as models and instructions will be presented in response to those answers found identical to an incorrect model.

In systems like The Teacher’s Apprentice [5] and LISPITS [9], there is no specific criterion-performance model. Instead, they use a "model tracing" method of tutoring. At each state (step) of the process, the system infers the learner’s internal state by matching the learner’s output with the problem state generated by using ideal and incorrect rules (referred to as buggy rules). Instructions will be given according to this inference.

Both the model-tracing and the criterion-performance model have their advantages and disadvantages. The criterion-performance model approach usually requires a lot of space to store the models and also a heavy investment of time to generate all the possible models while the model-tracing approach might prevent the student from learning by making errors because it disallows the students from exploring other possible solutions. A detailed discussion on whether the criterion-performance model or model tracing method should be employed in the present study is presented later in the sections on tutoring module.

The Student Module

The function of a student module is to store knowledge which might be possessed by students, both correct or incorrect, in the form of rules. Traditionally, student modeling falls into two broad categories: the quantitative method [10], which is mostly used in conventional computer-based instruction (CBI) which will not be elaborated further here, and the qualitative method. Clancey [11], in defining qualitative models, says “The qualitative model is neither numeric nor physical analogues. Rather, it describes objects and processes in terms of spatial, temporal, and causal relations.” There have been mainly two types of qualitative methods used to model students as follows:

1. Overlay model: the student’s performance is compared to that of the computer expert. "The expert’s competence is assumed to be broken into a set of skills so small that the pupil either has them or doesn’t" [12]. In other words, the student has some part of the expert’s knowledge.

2. Bug identification method: the student model contains both domain knowledge as rules and misconceptions, and errors (bugs) as variants of rules. In this case, the student model includes something that the expert does not have but the student does. It is thought to be more realistic than the first type.

Figure 1 shows the relationship of the student model to the expert’s behavior and the bugs [12].

-- insert Figure 1 here --

This system is designed to help students to correct their errors and because the overlay model does not contain such knowledge, it is reasonable to say that the bug-identification model would be more appropriate for the present use. The student module in the present system therefore incorporates mal-rules, in addition to the correct rules. The mal-rules simulate students’ errors obtained from a set of tests, called mal-rule collection tests, administered to 125 secondary 4 (grade 9) students in Hong Kong.

The Tutoring Module

This module mainly contains the knowledge on how and when to help students to correct their errors. This is based firstly on evidence has been given by Lee [13] that producing conceptual dissonance (i.e., showing the students that their errors are contradictory to their previous knowledge) in the students’ mind, and having students practising the correct rules, can help them to correct their errors more effectively than methods like reteaching and model-based remediation [14]. The prescriptive rules used in Electronic Homework are based on this principle.

The second basis for when to remedy students’ errors, relates to the argument about which of the above models, the criterion-performance model or the model-tracing method, should be used. The main differences between the two methods, is the amount of freedom to explore, given to students during the interaction process, and the possibility of forming mal-rules due to incomplete learning. In the first mentioned, instructions are only given at the end of each problem, thus allowing students to flounder freely. In the process of floundering, students may make errors but it is also possible that they would discover their errors and correct them. Floundering may be good experience for those students who are able to correct errors since it helps them remember the correct rule. However, for those who cannot discover their own errors, floundering can become a source or future errors since the errors made are not corrected immediately [1]. The Model-tracing method keeps the students away from making errors, but at the same time, may force them only to memorize the correct rules without really understanding them. Clearly, both methods have their advantages and disadvantages.

For practical considerations, however, the model-tracing method has the advantage that there is no need to store a large number of problem solving models, which makes it easier for a computer system to handle. The model-tracing method is therefore employed in the present system.

The Communication Module

This module deals with the interaction between the human and the computer tutor. Normally, the module’s work consists of translating the human language into computer language and translating computer language into that which humans can understand, the input and output components of the system respectively. The translating of human language is not an easy task since human language is not well defined and is sometimes even illogical. To tackle this problem, some intelligent systems such as Meno Tutor [15] use a kind of restricted language in which the vocabulary consists only of a limited number of terms and the grammar used is strictly defined. With this kind of language, the computer can understand what the human user enters and react suitably.

Another type of system solves the language problem by displaying icons on the screen so that the users can choose their actions by simply clicking the appropriate icon by using the mouse. An example is The Teacher's Apprentice [5]. As only a limited number of icons can be displayed on the screen, the latter method is only capable of handling simple interactions. However, even the former limited-language method cannot allow complex human-computer interactions. It only works well in restricted domains such as mathematics or computer programming.

As it is believed that the displaying of icons would remind students on how to solve the problem, the iconic approach is not adopted since such iconic display is not normally found in students’ exercisess. Also, Electronic Homework is intended to be used by school students who may not be good at typing. These students will find it easier to use a mouse to input information. All the symbols required in a logarithmic expression are therefore displayed on the screen so that students can simply use the mouse to click those they wish to put in their own expression, thus reducing the troublesome task of typing in the expressions in full. In addition, this also reduces their chances of making low-level errors such as missing brackets.

To reduce a student’s working memory load, the required formula as well as other given values are displayed on the screen for easy access. Further, the screen is divided into three parts: one for displaying each step the student takes in solving the problem, another for displaying feedback to the student and the third for displaying formula and constants. In this way, the student can immediately know where to concentrate during the problem solving process.

The Homework Administrator

The Homework Administrator is responsible for the handling of routines that are normally done by teachers which include the sequencing of problems, collecting students’ errors and the marking of students’ work. The following paragraphs describe how these tasks are done.

How to Order Problems According to Their Difficulty

Evidence has shown that arranging problems in terms of their difficulty benefits students [16]. Although item difficulty is usually measured by the item difficulty ratio which is the ratio of the number of students who answer the item correctly to the total number of students making the attempt. As this can only be determined after the item is administered, it is impossible to use it for the present purpose. The present system uses a measure called ‘Problem Complexity’ [17] to sequence the problems. The formula for calculating Problem Complexity is as follows:

Problem Complexity = 0.11 Machstep + 0.19 Notmfac +

.17 Familar + 1.17

where Machstep is the number of steps required by the computer system to solve the problem, Notmfac is the number of operators (+, -, *, /, etc.) in the problem expression and Familiar is the degree of familiarity of the problem to the students (a value of “1” will be given to problems that require students to simplify expressions involving logarithms of numbers, “2” for problems require simplifying expressions involving logarithms of variables and “3” for solving logarithmic equations).

The difficulty of each problem can be calculated by the system using this formula, and problems entered by teachers can be arranged accordingly. Teachers therefore do not have to take care of the ordering of problems.

How Students’ Errors Are Collected

As students are doing their homework on the computer, every error made is recorded, whether the error is finally corrected or not. The list of errors is then stored on the floppy disk to be submitted to the human teacher. When all the disks are collected, both the list of errors admitted by an individual student and a summary of errors admitted by all the students can be displayed on the computer monitor and the teacher can then very quickly tell how well the students have learned.

How Students’ Work Is Scored

Whether a student can correctly solve a problem or not can be easily checked by comparing the student’s answer with the machine generated one. Hence a student’s score in doing an exercise can be obtained by just counting the number of problems he or she correctly finished. This score is again stored on the floppy disk submitted to the teacher. When the disks are collected, the teacher can quickly know the performance of the whole class by having the computer display a summary of their scores.

The Tutoring Process – An Example

The tutoring process in Electronic Homework by one of the student records as follows: A problem was given and the student (a girl) was asked to solve the problem step by step. Figure 2 depicts what the screen shows when she was solving the problem.