The Effective Use of Undergraduates to Staff
Large Introductory CS Courses

Stuart Reges, Assistant Chairman for Education
John McGrory & Jeff Smith, CS198 Coordinators

Department of Computer Science
Stanford University
Stanford, CA 94305

Abstract
In the past few years many schools have tried to simultaneously achieve the following goals in their introductory CS courses:

Everyone has discovered that the first two goals are difficult to achieve in the presence of the third. This paper presents a model that has evolved over the last five years at Stanford University where all three goals have been accomplished by replacing graduate student TAs with undergraduate section leaders.

Background
The Department of Computer Science at Stanford University has only recently offered an undergraduate major in computer science, but has always taught large introductory courses for undergraduates. Interest in these courses has increased at a tremendous pace, but the administration has been either unable or unwilling to reallocate TA money to our department, even when faced with statistics like the following comparison of undergraduate units in various science departments.

Department 74-75 84-85 Change
CS 3,907 11,140 +185%
CS 3,907 11,140 +185%
Physics 10,897 10,247 -6%
Chemistry 14,554 12,150 -17%
Biology 14,130 10,067 -29%
Psychology 21,424 14,855 -31%

Ten Year Change in Student Units

As a result, TA salaries have been paid mostly from department gift funds for many years. Thus, the department has had to find creative ways to teach courses for less money. In the case of introductory courses, the department embarked on a number of experiments to find out how to best make use of undergraduates as TAs.

The Experiments
Before any experiments began, the basic course structure was an instructor with a number of graduate students all helping out with both the grading and the administration of the course:

The first experiment was to have undergraduates taking on the same responsibilities as graduate students, but receiving units rather than pay. This experiment worked fairly well, and saved a considerable amount of money. We were actually surprised by the number of undergraduates who were not only willing, but actually eager, to volunteer as units TAs.

Then some members of the administration expressed concern that undergraduates should not get units for work that grad students get paid for. Strong feelings were voiced on both sides, but a compromise was reached. We created CS198, The Teaching of Computer Science. Students in CS198 ran discussion sections for the courses and helped students at the computing center. Thus, we gave units for those parts of the job that seemed the most educational and fun. We continued to employ both undergrads and grads as graders, but with so many different kinds of staff, we decided to designate one grad student as "head TA" to oversee the rest of the staff.

This structure had significant problems, the most apparent of which was the grading. In the previous structures where the graders participated more fully in the course, they seemed to be more in touch with the course's philosophy. In this new structure the graders often didn't know what to stress in grading. This was particularly true of the graduate students who had not taken a programming course at Stanford.

The grad student graders were obviously not working out, and the undergrad graders were cheaper, so our next experiment was to eliminate the grad student graders. We retained the single grad student serving as head TA.

This structure was an improvement over the previous one, but there were many complaints that what was taught in section was not what was stressed in grading.

After much deliberation, we decided to put the grading and section leading back together. We still had a strong feeling that CS198 was a significant learning experience, so we decided that first-time students would get credit and experienced people would be paid.

Section Leaders
Discussion Sections
The weekly discussion section is the most formal aspect of the section leaders' responsibilities. Intended as a supplement to the regular lecture material, these weekly sessions can be used primarily to review and/or supplement lecture material, or as a more informal problem-oriented discussion based largely on relevant examples. The determination of the format taken in any individual section depends on the guidelines set by the instructor, the section leader's preferences, and, most importantly, the students in the actual section.

Typically, the section leader will attend a meeting with the instructor, grad TA and other section leaders early in the week, prior to his own section, during which the instructors discuss the goals of the current lectures and discussion section. Ideally, the section leader is given a list of topics that should be covered and even a few examples that can be used. This is followed by a general discussion among the other section leaders concerning apparent trends in student progress and response to new material and assignments. Based on this information, the section leader can then prepare material for his/her section.

The actual section is focused entirely on the needs of the students; this gives rise to a wide range of section styles, each catering to the particular group of students in the section. The section leader may follow his lesson plan of reviewing lecture topics, presenting additional examples, and addressing students' questions, or he may be forced to abandon all prepared material if a serious misunderstanding of an earlier, more fundamental concept becomes apparent. One essential point regarding material that is covered in the section is that it is not intended to be a "problem session" in the conventional sense (i.e. focusing on current problem sets or programming assignments); rather, the section leader is expected to develop examples that will aid the students in understanding the concepts necessary to complete the assignments.

Consulting
Every section leader holds consulting hours at the main campus computer facility; these hours are similar to conventional TA office hours, but they are directed more towards hands-on assistance. There are two to four helpers on duty at all times, all of whom are available for questions from students in any intro course.

The on-duty helpers assist the students in such a manner that they will not only be able to work through their immediate technical problem, but also that they will develop their problem solving skills and knowledge of other problem solving approaches (e.g. using debuggers). Section leaders are not expected to give direct solutions to students, but instead to guide them to answering their own questions.

Interactive Grading
Under this system, students hand in their assignments, and then set up an appointment with their section leader for an interactive grading session. Sometime before the grading session, the section leader looks over students' programs to make initial observations, but does not assign grades to the work. During the interactive session, the student and section leader discuss the program - the student is expected to explain certain technical aspects of his/her work, as well as the motivation behind the design, and the section leader asks the student about any other questions he might have about the student's work. These sessions then often lead into discussion of new material, and any other questions that the student may have independent of the assignments.

The great benefit of this system is that the student is given the opportunity to explain his thought processes and understanding (or lack thereof) of the material; consequently, the section leader is able to identify the needs of each individual student. Every student participates in one of these grading sessions on a weekly basis, so the amount of individual attention is substantial.

What Makes a Good One?
The qualities that make a "good section leader" are difficult, maybe even impossible, to define; many section leaders with a wide variation in technical and personal backgrounds have been very successful in the program. In fact, there are very few traits that seem to be necessary for being a "good" section leader.

The most essential characteristic is genuine concern for the welfare of his/her students. This can be manifested in many ways, from developing examples for the discussion section that are specially applicable to unique problems of the students to the willingness to be flexible and offer help at virtually any time. When the students see the section leader making an extra effort to help them, they will frequently be more inclined to put a little more effort into their work. The net result is that the students gain more from the experience than they might have otherwise.

Although extensive technical expertise is not necessary, the desire to thoroughly understand all aspects of the students' work is crucial. This understanding may come easier to those section leaders who have a more technical background, but it is by no means dependent on it. Diligence and responsibility are of course important - keeping up with all class details and attending staff meetings is necessary, and constantly looking for possible improvements in the section, the lecture, or the overall program is always beneficial.

The Role of the Coordinator
The glue that holds the program together comes from the work of the coordinators. The coordinators' most basic responsibilities are to:

  1. train new section leaders
  2. provide channels of communication between section leaders, teaching assistants, lecturers, and students
  3. organize and handle administration
Training of New Section Leaders
Most of the actual training of section leaders is experienced on-site. A group of new section leaders will, starting the second week in a quarter, lead a section for a course. As the quarter progresses, their on-site training is complemented by open discussions which the coordinators organize and lead including: Rather than simply lecturing, we try to create environments and opportunities for the new section leaders to look at their own teaching/communicating skills through a mirror, and to give all an incentive to share their skills with others, as well as emulate the techniques of others. We encourage experienced people to also attend and participate in our workshops, to refresh their skills and to share their experience with the group. Probably one of the most valuable resources at the Thursday Workshops is the section leading experience of the coordinators themselves, which new section leaders draw from often.

Communication Channels
The coordinators generally lead an open meeting between all the section leaders for the various classes as well as the teaching assistants and sometimes lecturers for those classes. The purpose of these meetings are threefold.

  1. educate section leaders for all classes about current programming assignments and lecture topics, in order that they all be effective consultants
  2. give section leaders a chance to communicate with teaching assistants from all classes about questions and problems related to programming assignments and course material
  3. give teaching assistants the same opportunity to communicate with section leaders
A typical meeting might give each of the course teaching assistants ten minutes to discuss program assignments and lecture material. At the end of each TA's presentation, the group has the opportunity to ask questions about the presentation, or to present other problems/ideas that the teaching assistant may have over-looked. Following, some administrative details are discussed, after which the group goes over typical problems and events that happened during the week of consulting.

Coordinators are also responsible for establishing the channels of communication between various entities in the undergraduate program.

Organization and Administration
We have found that a good share of coordinating responsibilities are attributed to organizing all the various meetings and providing the necessary frame work for the rest of the quarter to proceed smoothly. Much of this responsibility stems from setting up all of the sections at the beginning of the quarter. Throughout the quarter, the coordinators are the resource lecturers and teaching assistants utilize for problems with sections for their course. Later in the quarter a coordinator might: While the program was administered by a single coordinator in the beginning, it soon became apparent that two coordinators were necessary.

The Issue of Credit
The issue of credit first came up when we decided to give undergraduates units for work that graduate students are paid to do. Some faculty outside of computer science thought that this was inappropriate. We pointed out that all departments routinely do this for research projects, giving undergraduates independent study units for work that graduate RAs are paid to do. There was still a strong sentiment that participating in teaching was not as worthy of credit as participating in research.

But nobody complains about our current structure for CS198. This is partly due to our efforts to emphasize the educational aspect of CS198, requiring students to attend a weekly lecture on teaching techniques and constantly emphasizing the idea that this is a learning experience. But more importantly, everyone has come to realize how truly educational CS198 really is.

At the same time, we do not offer CS198 for a letter grade. When students were receiving credit for TA work, they always received an A, so it seemed that the decision was a binary one. We were initially concerned that the pass/fail nature of the course would encourage laxity, but that hasn't happened.

Summary
The quality of education in our introductory CS courses has increased tremendously in the past five years. Without spending any more money than we used to, we have been able to bring about the following changes.

The keys to the success of our program are:
Stuart Reges
Last modified: Sun Feb 27 15:55:06 MST 2000