An Evaluation of Intelligent Learning

Environment for Problem Posing

Akira Nakano, Tsukasa Hirashima, and Akira Takeuchi

Kyushu Institute of Technology Department of Artificial Intelligence 680-4 Kawazu,Iizuka 820-8502, Japan

nakano@minnie.ai.kyutech.ac.jp

Abstract. In this paper, we describe an Intelligent Learning Environ-ment which realizes the learning by problem posing. In the learning byproblem posing, a learner poses problems through the interface providedby the ILE. The ILE has a function to diagnose the problems posedby the learner. By using the results of the diagnosis, the ILE helps thelearner to correct wrong problems, or leads her/him in the next step ofproblem posing. We have developed the ILE. In this paper, we also re-port an experimental evaluation in an elementary school. By using theresult of the experiment, we examined (1) usability of the ILE and (2)effectiveness of the ILE. In the ILE, the interface was implemented inJava, and the diagnosis module was implemented in Prolog. The currentenvironment deals with simple arithmetical word problems solved by anaddition or a subtraction.

1 IntroductionSeveral researchers about problem posing of arithmetical word problems suggested thatproblem posing is important to learn arithmetic. For example there are task analysisof problem posing [1], [2], examination of learning effects of problem posing [3], andcase studies in classroom [4], [5]. Besides, “the Curriculum and Evaluation Standardsfor School Mathematics (in USA, 1989)”, and “Professional Standards for Teachingmathematics (in USA, 1991)” also indicated that it was important for learners toexperience to pose problems. However, despite the importance of problem posing, it isnot popular as a learning method in reality. In Learning by Problem Posing, in orderto judge whether the problem is correct or not, a teacher has to examine each problemwhich learners posed. Therefore, it is difficult for teachers to use problem posing as alearning method.

Based on this consideration, we are developing an Intelligent Learning Environmentwhich realizes the learning by problem posing. The domain of the ILE is arithmeticalword problems solved by an addition or a subtraction. In the ILE, when a learner posesa problem, the ILE diagnoses the problem and helps her/him to correct the wrongproblem, or leads her/him in the next step of problem posing [6]. We have alreadydeveloped a prototype of the ILE and used it in the lecture of the class experimentally[7]. However, in the experiment because we could not keep enough computers to provideeach student, two or three students used one ILE. Therefore although we confirmedthat learners could pose problems on the ILE, we could not evaluate the ILE in detail.

In this paper, we report an evaluation of the ILE based on individual use by 4thgrade elementary school students in class. In this experiment, we evaluate the ILE fromthe following two viewpoints, (1) whether the students can pose problems by using theILE or not, and (2) whether the use of the ILE is effective to improve the student’sability of problem posing or not. We call the former viewpoint “usability” and thelatter viewpoint “effectiveness”. To evaluate the two points, we carried out pre-test,post-test, questionary for both students and teachers, and collected log data of problem

posing in the ILE. So, in order to consider about the usability, at first, we analyzedif subjects used the ILE enough. We then checked if the subjects whose results in thepre-test were low could pose solvable problems in the ILE. Moreover, we put severalquestions to several teachers who observed this class of problem posing on the ILE.

We compared the results of the pre-test and the post-test in order to evaluate theeffectiveness. If the result of the post-test is significantly higher than the result of thepre-test, we can judge the ILE is effective in problem posing, not only the quantity ofposed problems but also the quality of them are important to evaluate the ability ofproblem posing. Because there is no standard method to evaluate the ability of problemposing, we also prepare a scoring method of posed problems in this paper.

2 ILE for problem posing

2.1 Solution-Based Problem Posing

To develop an ILE for learning by problem posing, categorization of problem posingis important. We have categorized problem posing in the following three types: (1)Problem-Based Problem Posing, (2) Story-Based Problem Posing, and (3) Solution-Based Problem Posing [7].

The ability to pose problems is necessary to solve problems of which solution arenot known. However, the ability often cannot be acquired only to solve problems.Therefore, the practice to pose problems is a promising way to acquire the ability. Inthis paper, we deal with “Solution-Based Problem Posing”. Solution-Based ProblemPosing is the most popular type of problem posing, because it is effective to masterthe solution method directly [8], [9]. To deal with other types of problem posing oncomputer is our future work.

Solution-Based Problem Posing is an effective practice to learn arithmetic. InSolution-Based Problem Posing, first, a solution method is given beforehand, then,learners pose problems which can be solved by the solution method. We call the so-lution method “stated solution”. The stated solution is defined by a learner, or givenby a teacher or a system. Based on a stated solution, the learner has to pose problemswhich can be solved by the stated solution. Finally, the solution method of the problemposed by the learner has to be equal to the stated solution. We call the solution methodof the posed problem “derived solution”. That is, when the derived solution is equal tothe stated solution or when the derived solution is judged the stated solution, the resultof the problem posing is correct in Solution-Based Problem Posing. So, we think thatSolution-Based Problem Posing is learning method to learn relation between problemand solution.

2.2 Framework of ILE for Problem Posing

We developed an ILE for Solution-Based Problem Posing. We describe the ILE ofproblem posing based on the solutions in this section. The domain is arithmetical wordproblems which can be solved by an addition or a subtraction. Fig. 1 shows the frame-work of the ILE. In the ILE, when a learner poses a problem, the ILE diagnoses theproblem in Problem Diagnosis Module. Moreover, by using result in Advice Generator,the ILE helps her/him to correct the wrong problem, or leads her/him in the nextstep of problem posing. The Problem Diagnosis Module and the Advice Generator andknowledge used in these modules are implemented by Prolog. Client and other moduleand information of Server are implemented by Java.

Fig. 1. The framework of the ILE. Fig. 2. The first step to specify prob-lem formula.

Fig. 3. The second step to pose problems.

Next, we explain graphical user interface in the ILE. The interface of the ILEprovides learners with two steps to promote problem posing. In first step, a learnerspecifies a stated solution before s/he poses a problem, e.g., “5-X=3”. We call thissolution method like “5-X=3” “problem formula”. So, in Fig. 2, the learner specifiesa stated problem formula “5-X=3”. In second step, s/he poses problems which canbe solved by the stated problem formula. In order that the ILE judges whether theresult of the learner’s problem posing is correct or not, the ILE checks whether thederived problem formula is equal to the stated problem formula or not. Fig. 3 is theinterface to pose a problem. This interface is explained in more details. Currently,the ILE can deal with only Change-Problem[10]. In Change-Problem, the quantity

in “the initial situation” is changed to the quantity in “the final situation” by the“change action”. The Change-Problem usually consists of three sentences: the firstsentence describes the initial situation, the second sentence describes the change action,and the third sentence describes the final situation. Therefore, we prepare a “problemtemplate” that composed of three “single sentence templates”. In Fig. 3, the problemtemplate is shown in the left side panel. By filling in the blanks of three single sentencetemplates, the problem is completed. Moreover, in order to filling in the blanks of thetemplates, the ILE provide learners with several concepts in concept panel beforehandand ten-key. In Fig. 3, these are shown in the right side panel. In issue panel, theILE provide learners with information generated based on the result of first step. Incomment panel, the ILE provide learners with advice which is feedback about result oftheir operation in the interface and is generated in Advice Generator. Moreover, weexplain the problem template in detail. The initial situation has the four information:“owner”, “object”, “number”, and “unit”. This means that “owner” has “object” andthe number of “object” is “number”, then, the unit of the number is “unit”. The changeaction has the five information: “actor”, “object”, “number”, “unit”, and “action”. TheILE deals with the type of action where object moves between the object’s owner andother people, i.e., between two peoples. For example, the action where A receives B’sobject from B is dealt with. However, the ILE does not deal with the type of actionwhere object moves among three people. For example, the action where A receives B’sobject from C is not dealt with. The final situation has the four information: “owner”,“object”, “number”, and “unit”. In learning by problem posing, the problem templateand the sentence templates restrict the expression of problems. However, we thinkthat using these templates don’t lose the main effect of learning by problem posing,because, at learning by arithmetical word problem, we think that it is most importantto consider relation between concepts and numerical relations.

3 Experimental use of the ILEA prototype of the ILE has been already developed. So two elementary school teacherspermitted us to use the ILE in their arithmetic classes after they had used the ILEby themselves. Subjects were 55 students of elementary school in 4th grade. In thisexperiment, we could keep enough computers to provide each student.

We evaluate the ILE from the following two viewpoints, (1) whether the studentscan pose problem by using the ILE or not, and (2) whether the use of the ILE iseffective to improve the student’s ability of problem posing or not. We call the formerviewpoint “usability” and the latter viewpoint “effectiveness”.

The process of the experiment is as follows.

(p1): We took 20 minutes to teach how to pose Change-Problems with several exam-ples.

(p2): In the pre-test, the subjects posed problems on paper in 10 minutes. We providedthe subjects with problem formulas, and then, the subjects posed problems thatcan be solved by the problem formulas. In this pre-test, the following problemsformulas were provided: “6+9=?”, “12-5=?”, “7+?=15”, “11-?=4”, “?+9=13” and“?-7=6”.

(p3): The subjects posed problems with the ILE. 10 minutes were taken to explain theway to use the ILE, and 30 minutes were used to pose problems. Then, 4 teachingassistants were supporting how to use the ILE always.

(p4): As the post-test, the subjects posed problems on paper again in the same methodof (p2).

(p5): We asked teachers and the subjects several questions.

Through the experiment, we gathered the following data: (d1) logs of problemposing on the ILE, (d2) scores of problem posing on paper before using the ILE andafter using the ILE, (d3) answers that teachers and students filled questionnaires.

In this section, we explain the evaluation of the ILE based on these data.

3.1 Usability of the ILEIn lesson of problem posing on paper, it is difficult for a teacher to support eachlearner and it is difficult for the learner to write sentences on paper. Therefore, somestudents maybe fail to pose problems, because they are not good at write sentences.Therefore, the ILE provides learners with feedback to correct wrong problem and tolead them next problem posing. Moreover, the ILE provides learners with a tool tomake sentences with combination between concepts. Therefore, learning by using theILE is not practice to write sentences, but it is practice to make relations betweenconcepts and numerical relation. From the results, we expected that the ILE would beusable for learners.

We evaluate the usability of the ILE from the following four viewpoints. At first,(1) we analyzed the number of posed problems with the ILE. Then, (2) we checkedif the subjects whose results in the pre-test were low could pose solvable problems inthe ILE. Next, (3) we put several questions to teachers who participated at this classof problem posing on the ILE. By using the questions their impressions for the lessonof problem posing with the ILE are examined. Finally, (4) we put several questions tosubjects. By using the question, their impressions for the learning of problem posingwith the ILE are examined.

Fig. 4. The relation among amount of the posed problems with the ILE, a distributionof the subjects and the contents of the use of the ILE.

Fig. 4 shows two graphs at the same time. One graph is line graph which showsrelation between amount of the posed problems with the ILE and number of the sub-jects. The other one is bar graph which shows relation between amount of the posedproblems with the ILE and the contents of the posed problem with the ILE. At first,we explain each graph as follow.

In the line graph of Fig. 4, the horizontal axis shows amount of the posed problemswith the ILE, and the vertical axis shows number of subjects (the vertical axis is shownin the right side of Fig. 4). For example, when the value in the horizontal axis is 7, thevalue in the vertical axis is 9. It means that the number of the subjects who posed 7problems with the ILE is 9.

In the bar graph of Fig. 4, both the horizontal axis and the vertical axis showsamount of the posed problems with the ILE (the vertical axis is shown in the left side

of Fig. 4). That is, the value in the horizontal axis is the same value in the verticalaxis. There are two cases in the contents of the posed problems with the ILE; one isthat a subject posed a problem which is judged correct by the ILE, the other is thatthe subject posed the problem which is judged wrong by the ILE. For that reason, wedivided one bar into two colors. Therefore, at each bar in the graph, the length of thepart which is dark color shows “mean of the amount of the posed wrong problems”(The “wrong problem” means that the posed problems is judged wrong by the ILE)and the length of the part which is light color shows “mean of the amount of the posedcorrect problems” (The “correct problem” means that the posed problems is judgedcorrect by the ILE). For example, when the value in the horizontal axis is 7, the valuein the vertical axis is 7, and then, in the bar, the length of the part which is dark coloris 2 and the rest, i.e.; the part which is light color is 5. It means that the subjectswho posed 7 problems with the ILE posed wrong problems and correct problems atthe rate of 2 to 5. That is, their correct answer rate is 71 %. By using both the linegraph and the bar graph, for example, we can know at the same time that the numberof the subjects who posed 7 problems with the ILE is 9 and their correct answer rateis 71 %.

In the experiment, the mean of the amount of the problems posed by total subjectsis 7.5. Forty-seven % subjects posed problems more than the mean of the amount ofthe problems posed by total subjects (i.e.; forty-seven % subjects posed more than 8problems). We guess they posed problems actively with the ILE, although they posedmany wrong problems (the correct answer rate of the subjects is 57 %). Fifty-three% subjects posed problems less than 7.5 with the ILE. However, they did not pose somany wrong problems (the correct answer rate of the subjects is 79 %). So, we guessthat the subjects used the ILE carefully not to pose wrong problems. Four studentsposed less than two problems although the all problems were correct. Their log dataof all operation of the ILE show that they did not operate the ILE at all in the secondhalf of the class. So, we guess that the subjects did not use the ILE to pose problems.

Fig. 5. The relation between ability of problem posing and amount of the posedproblems with the ILE.

Next, we check if the subjects whose results in the pre-test were low could usethe ILE and if the subjects could pose solvable problems in the ILE. Fig. 5 shows two

graphs at the same time. One graph is line graph which shows relation between amountof the correct problems in the pre-test and number of the subjects. The other one isbar graph which shows relation between amount of the correct problems in the pre-testand amount of the posed problems with the ILE. At first, we explain each graph asfollow.

In the line graph of Fig. 5, the horizontal axis shows amount of the correct problemsin the pre-test, and the vertical axis shows number of the subjects (the vertical axis isshown in the right side of Fig. 5). For example, when the value in the horizontal axisis 3, the value in the vertical axis is 9. It means that the number of the subjects whoposed 3 correct problems in the pre-test (in the pre-test, we judged correct problemwhen the posed problem can be solved).

In the bar graph of Fig. 5, the horizontal axis shows amount of the correct problemsin the pre-test, and the vertical axis shows mean of the amount of the posed problemswith the ILE (the vertical axis is shown in the left side of Fig. 5). In the same wayas the bar of Fig. 4, we divided one bar into two colors. Therefore, at each bar in thegraph, the length of the part which is dark color shows “mean of the amount of theposed wrong problems” and the length of the part which is light color shows “meanof the amount of the posed correct problems”. For example, when the value in thehorizontal axis is 3, the value in the vertical axis is 6.9, and then, in the bar, the lengthof the part which is dark color is 2.7 and the rest, i.e.; the part which is light coloris 4.2. It means that the subjects who posed 3 correct problems in the pre-test posedwrong problems and correct problems at the rate of 2.7 to 4.2. That is, their correctanswer rate is 61 %. By using both graphs, for example, we can know at the same timethat the number of the subject who posed 3 correct problems in the pre-test is 9 andtheir correct answer rate is 61 %.

By using Fig. 5, at first, we classify the subjects into several types based on theresult of the pre-test. Subjects who could pose six correct problems or five correctproblems in the pre-test are 22. We consider that they have the high level ability ofproblem posing. So, we call them “high level subjects”. Subjects who could pose fourcorrect problems or three correct problems in the pre-test are 22. We call them “middlelevel subjects”. Subjects who could pose two correct problems or a correct problem inthe pre-test are 11. We call them “low level subjects”.

Next, we examine how each type subject used the ILE. The correct answer rate ofthe low level subjects is 48 %, i.e., the worst in these level subjects. The correct answerrate of the high level subjects is 78 %, i.e., the best in these level subjects. From thisresult, we consider that it reflects the difference of ability of problem posing.

The mean of the amount of the correct problems with the ILE of the low levelsubjects is 4.4, i.e., the worst in these level subjects. The mean of the amount of thecorrect problems with the ILE of the high level subjects is 5.7, i.e., the best in theselevel subjects. From this result, we consider that it reflects the difference of abilityof problem posing, too. However, the low level subjects could pose several correctproblems on the ILE. Therefore, we judge that the ILE was not hard for the low levelsubjects to use. Moreover, we consider that the result shows the function in order tocorrect wrong problem is a factor of the result. We consider that the reason in whichthe high and middle level subjects used the ILE carefully in order not to pose wrongproblem.

Moreover, in order to judge whether amount of the use of the ILE is enough or not,we asked teachers and the subjects several questions. Table 1 shows the result in whichwe asked teachers who used the ILE and presented 4 questions. These are as follows:(t-1) Did you judge that subjects using the ILE were earnest at the class?, (t-2) Didyou judge that the subjects can use the ILE smoothly?, (t-3) Were feedback of theILE helpful for subjects when they posed wrong problems? and (t-4) Were feedbackof the ILE helpful for subjects when they posed correct problems? And, in order toinvestigate usability, we asked user of the ILE (i.e., they are this subject). Table 2shows the result in which we asked subjects 3 question. These are as follows: (s-1) Didyou enjoy problem posing on the ILE?, (s-2) Were feedback of the ILE helpful for you

when you posed wrong problems? and (s-3) Were feedback of the ILE helpful for youwhen you posed correct problems?

Table 1. An evaluation of theteachers.

Question Yes No So-so(t-1) 6 0 1(t-2) 7 0 0(t-3) 6 0 1(t-4) 5 0 2

Table 2. An evaluation of the4th grade students.

Question Yes No So-so(s-1) 47 8 0(s-2) 48 6 1(s-3) 41 10 4

Subjects’ behavior at class (t-1, t-2) and subjects’ interest (s-1) and the functionof the ILE (t-3, t-4, s-2, s-3) are evidence of the above our interpretation which is thatthe subjects posed enough correct problems on the ILE. In the result of the Table 1and the Table 2, these are evaluated high. That is, we think that the teacher thoughtthis class using the ILE succeeded as class of learning by problem posing and subjectsare interested in this class.

In order to judge whether the ILE is learning material for learning by problemposing that elementary school students can use easily or not, we showed two Figuresand two Tables and our interpretation of these data. In the result based on two Figures(Fig. 4 and Fig. 5), almost subjects, even the subjects who posed few problems, couldpose correct problems with the ILE. Moreover, in the result of two Tables (Table 1 andTable 2), the teacher answered that the subjects could use the ILE and the function ofthe ILE was good, and the subjects answered that the class of the ILE was interesting.Therefore, in these results, we judge that the usability of the ILE is high.

3.2 Effectiveness of the ILEIn problem posing on the ILE, a learner has to find the proper combination amongconcepts and numerical relations. Because to find the proper combination is an im-portant task of problem posing, the practice should be effective to improve problemposing ability.

Therefore, we evaluate the effectiveness of the ILE from the comparison of thescores of the pre-test and the post-test. First, the method to give a score to each posedproblem, and then the result of the comparison of the scores of the pre-test and thepost-test is explained.

Scoring method of posed problems.

Difficulty in problem formula. Problem formula has three types: (α) A± B=X, (β)A ± X=C, (γ) X ± B=C (A,B,C are numerical values. X is a variable). “A” isthe number in the initial situation, “B” is the number of the change action, and“C” is the number in the final situation. “X” is the number that is derived bythe solution. In the α type, the answer is in the final situation. So this type ofproblem is the easiest one. In the β type, the answer is in the change action. Inthe γ type, the answer is in the initial situation. In learning by problem solving,generally, problems which can be solved by α type equation are the problems ofthe easiest type in these three types. And, problems which can be solved by γ typeequation are the problems of the most difficult type. We think that there is thesame difficulty of arithmetically thinking in problem solving and problem posing.So, we use the difference of formula type to score the posed problems.

Difficulty of operation. We think that each verb lets us remind each operation. Forexample, “give” usually reminds us of “subtraction”, and “receive” usually remindsus of “addition”. This is the simplest pattern of the combination of a verb and anoperation. For example, if we posed a problem by using “give” simply, the problemis like this: “Tom has 5 pieces of apple pies. Tom gives Nancy some pieces of hisapple pies. Tom has 3 pieces of apple pies. How many pies of apple pies doesTom give Nancy?” We call the operation reminds by the verb “simple remindedoperation”.Meanwhile, in the problem posing, not only the verb in the change action but alsothe combination of other concepts in the change action should be considered inorder to decide the operation. So, all problems are classified in two cases. One isthat an operation of derived problem formula is the same with the simple remindedoperation. And then, the other is that an operation of derived problem formula isdifferent from the simple reminded operation. We call this case “operation with thegap”. The gap means that there is difference between operation of derived problemformula and the simple reminded operation. For example, the case in which thereis the gap is “Tom has 5 pieces of apple pies. Nancy gives Tom some pieces ofher apple pies. Tom has 8 pieces of apple pies. How many pies of apple pies doesNancy give Tom?” The operator of derived problem formula is “addition”, but thesimple reminded operation of “give” is “subtraction”. Therefore, this problem hasthe gap. So, we judge that a problem which has the gap is more difficult than aproblem which does not has it.

Result of comparison between pre-test and post-test. In evaluation of theeffectiveness of the ILE, we investigate whether the ability of problem posing of allsubjects is improved or not, by using the ILE.

We explained the method to evaluate each posed problem. The scoring methodis as follows: if the problem type is the α type problem, the score of the problem isone. If the problem type is the β type problem, the score of the problem is two. If theproblem type is the γ type problem, the score of the problem is three. Moreover, onlyif the problem has gap between an operation of derived problem formula and a simplereminded operation, the score of the problem is incremented one.

So, by using the scoring method of the posed problems, we can get each learner’sscores of the pre-test and the post-test. We evaluate the effectiveness by using thedifference of the scores between the pre-test and the post-test. In the pre-test and thepost-test, the highest score is 18.

Table 3. A comparison between the pre-test and the post-test.test subjects mean standard deviation

pre-test 55 7.4 3.7post-test 55 9.6 4.0

So, we tried t-test in order to investigate a change between the mean of scores ofthe pre-test and the post-test. Table 3 shows the relation of both tests. In the result,the difference of the change is significant. (Mean of the pre-test=7.4, Mean of the post-test=9.6; t-test, t(55)=2.00, p<.001). Therefore, we think that the ability of problemposing of the 4th grade subjects is improved by using the ILE.

4 ConclusionsWe evaluated (1) the usability of the ILE and (2) the effectiveness of the ILE throughthe use of the ILE in the class of elementary school.

In the evaluation of the usability of the ILE, we presented that almost subjects,even subjects who posed few problems in pre-test, could pose correct problems with

the ILE. Moreover, in order to judge whether amount of the use of the ILE is enough ornot, we asked teachers and the subjects several questions. In the questions, the teacheranswered that the subjects could use the ILE and the function of the ILE was good,and the subjects answered that the class of the ILE was interesting. Therefore, in theresult, we judge that the ILE has enough usability for use in the class.

In the evaluation of the effectiveness of the ILE, at first, we proposed the scoringmethod of problem posing. And then, we reported the effectiveness of the ILE basedon result of t-test about the difference of the change between mean of the scores of thepre-test and mean of the scores of the post-test. In the result, we judge the subjectswho used the ILE improved in the ability of problem posing, i.e., the ILE also hasenough effectiveness for learning of problem posing.

The problem posing in the ILE (problem posing based on stated problem formula)is one of Solution-Based Problem Posing. Of course, we think that it is necessary todevelop other ILE based on other type of Solution-Based Problem Posing. At present,an evaluated environment in the ILE is an environment to pose problems based on astated problem formula. However, we are developing other environment to pose prob-lems based on information which is specified only calculation procedure to solve theposed problem, e.g., the information is “5-3”.

Moreover, we will develop the other types of problem posing, i.e., Problem-BasedProblem Posing and Story-Based Problem Posing. To realize the learning environmentof the two types of problem posing is also our future work.

AcknowledgementsThis research is supported in part by Grant-in-Aid for Scientific Research from the Min-istry of Education, Culture, Sports, Science and Technology of Japan, The Telecom-munications Advancement Foundation, and Artificial Intelligence Research PromotionFoundation.

References1. Silver，E.A.，Mamona,J., Posing Mathematical Problems:An Exploratory Study，

Journal of Research in Mathematics Education，vol.27，No.3, pp293-309, 1996.2. Lyn D. English, Children’s Problem Posing Within Formal and Informal Contexts，

Journal of Research in Mathematics Education，vol.29，No.1, pp83-106，1998.3. Ellerton, N. F., Children’s made up mathematics problems: A new perspective on

talented mathematicians，Educational Studies in Mathematics, 17, pp261-271, 1986.4. Silverman, F. L., Winograd, K., Strohauer, D., Student-generated story problems.

Arithmetic Teacher，39, pp6-12, 1992.5. Silver，E.A.，Cai,J., An Analysis of Arithmetic Problem Posing by Middle School

Students，Journal of Research in Mathematics Education，vol.27，No.5, pp521-539，1996.

6. Nakano，A.，T.Hirashima，A.Takeuchi, “A Learning Environment for Problem pos-ing in Simple Arithmetical Word Problem”，Proc of ICCE2000，pp91-98，2000.

7. Nakano，A.，T.Hirashima，A.Takeuchi, “An Intelligent Learning Environment forProblem Posing and Its Evaluation -In the case of Arithmetical Word ProblemsSolved by an Addition or a Subtraction”，Proc of ICCE2001，pp1242-1249，2001.

8. Nakano，A.，T.Hirashima，A.Takeuchi, “Problem-Making Practice to MasterSolution-Methods in Intelligent Learning Environment”，Proc of ICCE’99，pp891-898，1999.

9. Hirashima,T.，A.Nakano，A.Takeuchi， “A Diagnosis Function of ArithmeticalWord Problems for Learning by Problem Posing”，Proc of PRICAI2000，pp745-755，2000.

10. Silver, E.A., “On mathematical problem posing. For the Learning of Mathematics”，For the Learning of Mathematics 14(1), pp19-28, 1994.