Electronic Portfolios: Enhancing Student-Teacher  Interactions
"Yes" to internet interaction; "No" to automatic courses sent out on the web

Funding Source: Retention of Students using Electronic Portfolios, Sloan Foundation Michael Fried, Prof. of Math., UCI, June 1994
mfried@math.uci.edu

Introduction: WHAT WE DID IN A COURSE WITH 65 STUDENTS

Minority students show less confidence in using the instructor as a resource. To bring equity to the achievement of all our students here is what we did using technology.
These daily interactions brought more contact with students in one course than I had in 20 years of teaching. Without shell program technology, these interactions and their associate evaluations would have overwhelmed this instructor beyond measure. These portfolios started the process of documenting the value added by the instructor and the value of retaining these students. Many of these are minority students who were borderline for dropping out.

A. How Interactive Portfolios Work:

The background technology in this project is available to any university running UNIX in its e-mail accounts. Flexible shell programs can be used on many platforms, thus enhancing the portability of the project. The first step was to get students e-mail accounts and familiarize them with e-mail. Processing their responses and collating them into individual formatted portfolios would have been impossible without a system of shell programs.

A.1. Items in Individual Portfolios: Each student's portfolio isa text file; it has no formatting. It consists of items of three types:
  1. Those common to all student portfolios, usually responses to I(nteractive)Q(uestionnaire)s and P(roblems)O(f the)D(ay)s;
  2. Those common to all students working on a related class project;
  3. Individual student e-mail interactions, often a question/response sequence (initiated by either student or instructor)

Each student-portfolio's items are surrounded by a tag – it looks like an html tag. For example, assume students have responded to an IQ on the topic of Lines in 3-Space. I don't cover here the process by which the IQ results – with their associated tags – are placed into individual portfolios.

A.2. Tags and IQs: IQs tend to be mostly curricular, though sometimes they are surveys about students' backgrounds, or their worry about the course, etc. I stick with the curricular case. That is the nub of the problem: Using them to improve the delivery of curriculum and the assessment that tests the quality of that delivery.

An IQ would have its tags based on a taghead like line3space. Typically an IQ is a short series of questions, maybe just one or two. Yet, each has crucial parts, three to five to involve students in step-thinking.

Their respective tags might be line3space1 and line3space2. More likely, however, the tags would be suggestive of the questions: line3space-IsntALine, line3space-IsALine. Here I'm posing questions about objects, defined by expressions or equations, and asking the students about them.

In this case, I would do an IQ with just the question with the tag line3space-IsntALine.  (The published paper gold02-08-98.pdf shows what IQs look like to students.) That question might be about an object whose points satisfy an equation. The question related to line3space-IsntALine might have 4-parts  in which the student is taken through steps. The final conclusion is that the set described by the equation is NOT a line. There are a number of curricular points here.

Upshot: Students much trust their regurgitation. Yet, none of their memorized expressions for 2-space, without striking conceptual changes, work in 3-space. There are no exceptions in any classes I've ever taught in which lines in 3-space have arisen to this: The expressions for what students initially think should be lines in 3-space are actually planes. If you don't address this early, it becomes a persistent misconception – because lines (of sight) are so important in  picturing objects – to everything in the course.

The four-part IQ line3space-IsntALine addresses this. Each question part has an associated tag. These might be line3space-IsntALine-a ... line3space-IsntALine-d, or again, they might have more expressive ends. The craft of building IQ questions is in making the steps precede by small bites. The point is to separate curricular elements of computation, and definition remembering, from misconceptions.

The IQ four parts could have been phrased as one short question, as typically in text books. Yet, that would have left their misconception about the expression intact. Yes, they would have noted they were marked wrong. Yes, they would momentarily be bothered by that, but only because they lost points.

A.3. Using the tags to recall the Portfolio responses: An artful IQ, however, will catch their misconception at a critical time. Yet, few students – in practice, none in a size 65 class – would find that sufficient to straighten out the full curricular difficulty. At a later time, maybe the 3rd week of class, students would respond to an IQ with tag line3space-IsALine, also four part, paralleling line3space-IsntALine.

This IQ would have as its base a different expression. Its upshot would be a demonstration that the points satisfied by this expression are everything students would have thought a line should satisfy.

For example, high school geometry has an indirect definition of line – uniquely defined by properties saying it "goes through two given points." In the high school situation there is no actual direct definition in 3-space. Here the question uses the text's expression to emphasize it satisfies the high school line.

Suppose, as happens often after  line3space-IsALine, students suggest they know how to do this IQ more easily! What they might be saying is they would use the expression from line3space-IsntALine. At that time I would have to convince them that they already showed this wouldn't work. You need to know that for each IQ there is an IQ reader, a small program. For these IQs, they would be IQR-line3space-IsntALine and IQR-line3space-IsALine.

Technically,  these are the steps for how I would address the student misconception.

IQR-line3space-IsntALine thereby allows me to send a personalized e-mail message to each student. The personalization was their answer to an IQ extracted from their personal interaction portfolio. I can type into the  IQ reader  supporting explanation. Each student would receive a message that looks as if I addressed them individually, starting Dear Student_name, and containing their individual IQ response to line3space-IsntALine-c.

The graphic Polling Portfolios illustrates an overview of exactly this process. In that graphic, you see a topic cone over each "portfolio" represented by a student  desk. That is, the cone is for just one topic, evidenced in each student, by their responses to various items in IQs or PODs.  Literally, students – supported by instructor phrasing – can manage to answer related topic questions at one time. Then, later – lacking instructor support – find they have no way mentally back into what they learned about the topic. The IQ-Portfolio technology allows efficiently, though interactively, returning to the topic. That is, the varying width of that cone is an evidence of a classroom dynamic. The DynLearnGph07-14-04.html (and its accompanying graphic DynLearnGph07-14-04.gif) illustrates what happens without IQ intercession on series topics.

One summary of that: When the final is over you realize students held dearly to a misconception, that pretty much wiped the value of the class. Serious topics in mathematics classes can't be learned just once. You must revisit them, in an efficient timely manner. 

You create IQs with a program. A curricular expert need only use their expertise to craft the progression of IQ items, but it helps to have previous IQ examples. Why? Because most instructors are accustomed to letting students do problem solving on their own, following whole-cloth questions. Few students ever develop independent problem-solving ability from this. The program automatically creates the IQ reader.

A.4. Using IQ Results in Class and in Overall understanding of students: After students have received the e-mail message on line3space-IsntALine-c, I might go to class with a handout of all the responses tagged by line3space-IsntALine-c. The gist of this would be to show the class that they had the experience to overcome their misconception about how 3-space works.

The lesson I've used here is major in having students realize the standards for really understanding material.

C. WHY WE DID IT

Classroom time pressure cuts into the amount of feedback in two ways. Not enough time for more than a few students to interact. There's not enough time for students to think. In a practical way, classrooms are synchronous channels. The Internet provides an asynchronous communication that uses individual freedom to replace time constraints. Vector Calculus is a bottleneck prerequisite for completion of Physical Sciences and Engineering careers. This has its greatest impact on minority students. Their failure rate shows how unprepared they are for the spatial combination of algebra and geometry. Even those who do pass a second or third time through the course, suffer a retarded schedule toward graduation. The failure lies in the coordination of Algebra and Geometry in the early high school years. The problem: 9th and 10th grade algebra texts (excluding the Chicago Math project) have yet to show teachers how put the algebra and geometry modes of thinking together into one classroom. On our first questionnaire 85% of our students report having no recollection of a problem solving experience outside exercises from their textbook. Only two found it an enticing experience. Rather than being a teacher who showed them their first problem solving experience, I preferred raising their consciousness about their previous experiences. I stated two goals to the project repeatedly during our classes:
B. How Class Projects Work with Portfolios:

B.1. Intensely Interacting with Students Without Killing Yourself: Many students, however, do have special background problems.

If you see only an occasional student in office hours, and even then, one-on-one, you might never guess at particular student confusions. The portfolio of interaction also contains (tagged) e-mail interactions in which students express difficult confusions. Here is a portfolio extract from one of 65 students in the first course in which I used IQs.

Date: Tue, 19 Oct 93 03:43 From: ____ ____ <_____@orion.oac.uci.edu> Professor Fried,

Here is what I have learned from purchasing a modem. I did not know how a modem worked or that there is a big computer and that my computer is just a terminal contacting this big main frame. There are only 36 lines available and you have access to so many things like Melvyl and AntPac. Also I haven't really figured out how Emacs works because someone told me that you could compile a Fortran Program in Emacs since it is hooked to a main frame or something---I don't really understand.

Also, there are so many different commands to use. I get frustrated easily because I think I cannot figure out how to do something though I may have the handouts in front of me. I always felt it is something I don't know and completely over my head. ... It took many explanations to get me to see the picture. That is what I have the most trouble with: seeing it, visualizing it, and understanding. I don't know what I would do without my study Partner. I don't see how other people have no trouble while I struggle. I always wonder if I missed something all these years I have been in school or I have learned the wrong way because everything has to be systematic plug and chug for me to understand.

[MY REPLY] Dear _____, Your description of struggling with math is touching. Yesterday, one of my themes suggested we would use these projects to raise the esteem of students. ... Since we must try to do away with plug and chug, I've had to analyze carefully what could help us do that. Projects seem crucial for it. You have already gone further than you expected because of CAMP and your own growing initiative. ... Sincerely, Mike

A few things to notice:
The point of having students do projects is clearly to have them raise their standards on what learning is about. By engaging with a team partner (sometimes I used teams of three), students could interact with someone appropriate for their skills. Yes, I allowed them a say in their choice of partners. Yet, I also used my own intuitions to team people up. Again, to keep me the instructor, from being inundated I needed technology whose ingredients I list. B.2. The 3-stage project: With the programs in place, I gathered student IQ responses to a survey on how much initiative they'd ever been given in their classes for doing independent work. The first shock, the first time I did this, was that – despite their sincere responses – they couldn't imagine what doing a project was about. Most of my colleagues, by the way, couldn't either (imagine, for their students).

E. GRADING HIGH LEVEL EXAMS WITH 1/3 RD THE EFFORT

When the user executes an interactive e-mail message, it poses questions to the screen to which the user responds. The program places these responses in key fields in eval. When the program has completed, and the user has finished her responses, the last line mails a copy of the program and responses to me. Finally, when these return, I use mva to place them altogether in a folder with the grading program key_eval for eval. I've gained systematic and timely responses to questionnaires and evaluation instruments. Even more, however, the user hasn't tampered with the internal structure of eval.

The instructor types key_eval, which puts a list of question titles on the screen in a menu. From it the instructor can choose any question. Then, key_eval will place the complete question at the top of the screen followed by a menu showing student responses to the question. This is the fundamental simplification in grading: having all answers to a given question appear together on the screen in a menu. The instructor can grade one question in a displayed batch. Improved efficiency comes from being able to develop key_eval to add responses and a grade to selected subsets having similar responses. The grader's responses and marks come from a command line; there is no need to move around files seeking the right place to place these. On completion of grading the question, key_eval fits each student's answer, with graded responses, back into the whole evaluative instrument.

key_eval can post several statistics on the grades: total grades or grade on any one question for each of the students; and class average on all or any specific question.

C. Electronic Grading:

C.1. A primitive IQ: Here is the beginning of an interactive questionnaire after a student saves a message from e-mail. The student's e-mail name here is byerly. 
/home/users/student/byerly:113=>sh exper
Extracting exp_ques1
Extracting exp_ques2
   ...
Extracting exp_ques6
Extracting pre_exp_ques
   ...
Extracting prog_exp_ques
Dear Robert,
Try to answer the questions that appear in front of you briefly. I'm trying to find a way to automate more feedback from you, while creating no extra work for you, and not too much for me. If you need more than one line to answer a question, just keep typing before you press return. Once you press return you will have another chance to answer, but then this will move onto the next question.
Sincerely, Mike
Here is question 1.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
What days are OK for an extra class from 4-5? Type your answer and press <return>.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Respond to Question 1 by typing. When done, press <return>.
...
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
How can I help you improve with the guess and check problem solving strategy? Possible answer: Give you more time in class to try problems which we grade in class.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Respond to Question 4 by typing. When done, press <return>.
...
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
My feedback said some didn't really want to do a presentation, but since your grade last quarter was given for it, you felt you had to. Others clearly learned a lot from giving the presentation. Give some feedback where you stand on this. Honest, I won't get mad. Rather, I will try to find other ways to help involve you that don't necessarily put you in the position of making presentations.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Respond to Question 6 by typing. When done, press <return>.
Six questions with chances to respond if the student doesn't like her first answers. When the questions are done, the program mails them back to me. Back at my machine: 
New mail for mfried@math.uci.edu has arrived:
----
Date: Wed, 18 May 94 02:03:06 PDT
From: byerly (Robert Byerly)
To: mfried@math.uci.edu
Subject: exp_ques_response
C.2. A primitive IQ reader: A shell program mva automates moving this file to the directory of the student portfolios.  for viewing along with others that have come in. I go to that directory with one command and then type the name of the reader IQR-gno in the unix command line. You see this: 
Do you want to look at responses to a specific question? y(es)/n(o)
y
Choose a number.
1. Show exp_ques <number>
2. Show responses to exp_ques <number>
3. Put output of responses to exp_ques <number> in a file.
4. See list of questions again.
I ask to see the answers to Survey Question number 4, which was ''What do you think of the Guess and Check method I suggest often in class." Here are a few responses to this early IQ: 
\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%\%
akoines: I would like a regular written exam that will be like the qualifying exam.

alee: I realize any guess is better than no guess but I would feel more comfortable coming up with a guess that was somewhat on the mark. I think the in-class problems are very helpful, but I think they are more focused on checking rather than guessing. I have a hard time coming up with an educated guess when you ask a guess and check problem.

byerly: assign homework with hints on how to do the problems with guess and check

hly: There is no need to change because of the way the question is asked is at exactly the the level we are concentrate on and we are given sufficient time.

nunez: The guess and check strategy is something we all need to practice on our own, but I think a little more class time to practice this would be great.

rcollatz: Indeed, more guess and check opportunities in class would be welcome. Also, perhaps we might specifically devote some of the afternoon sessions to a few "practice" tests in which we would tackle problems of appropriate difficulty under pressure.

rmuir: give small problems in some organized manner, then give the solutions and explain how you got them. this could be done in class, or as take home problems. the book is full of topics but some are more important than others.

wprophet: I once had a basketball coach that told us over and over that practice does not make perfect; perfect practice makes perfect. I feel that we should practice this technique in simulated "game situations" to develop it.

I can now respond to each of these or to them in groups. It is as if I pulled all their answers together from a bluebook and could respond to them collectively. Survey questions, as you see above, tend to be all over the place. The students aren't all alike. Survey questions are useful. Yet, true curricular problems tend to come to grips with the effectiveness of rubrics like guess and check more pointedly.

G. WHAT HAS BEEN DONE BY OTHERS