Proposal: Introduction | Theoretical Foundations | Proposed Technology | Selection criteria | References | Budget | Schedule

Introduction

When our department recently moved to a new building with computer classroom facilities, we began to infuse our traditional lecture-based introductory Java programming course with computer-based active learning exercises.  This included worked examples, a technique where we demonstrate steps of the programming process while students follow along.   We were delighted when course evaluations showed that students not only believed that the computer-based activities helped them learn, they were even fun!  We’ve come to realize, however, that students have not achieved commensurate improvements in performance. We believe this phenomenon is partly due to the cognitive requirements inherent in the hands-on exercises; students must listen and follow along with us, interact with the operating system to manage their files, cope with an unfamiliar editor, type their code correctly, debug any errors they introduce along way, and somehow manage to catch up if they fall behind.  It’s no wonder the concepts get buried in all that clutter!

Scaffolding is a teaching strategy designed to systematically build on what the learner already knows in order to move him or her to the next level of understanding.  We propose to scaffold in-class programming exercises with a two part framework: First, by priming students with engaging presentations based on multimedia learning theory. These will introduce the underlying concepts before students get bogged down with implementation details.  Their effectiveness will be assessed using a modified version of our web-based electronic classroom assessment system to gauge students’ retention and transfer of the presentation material.   Second, following the hands-on exercises, we will annotate the worked examples using tablet inking to provide students with “beacons” to facilitate schema acquisition. Both the priming presentations and annotated worked examples will be made available to students as screencasts from the class webpage and via RSS feed.

Back to top

Theoretical Foundations

Multimedia learning theory - Our priming presentations will employ Richard Mayer’s [4] research-based principles for designing multimedia messages.  These principles, which are generally lacking in the typical bulleted list style presentations accompanying many textbooks, are particularly relevant for novice learners. Additionally, because they capitalize on people’s dual channel, visual and auditory processing capabilities, the presentations will be less cluttered and therefore visible on small, portable device screens.  Following the priming presentations, students will be directed to a modified version of our current electronic classroom assessment system to answer a retention question, to assess how much of the material they recall, and a transfer question, that asks them to apply what they have learned to a new situation.  These will allow students to self-assess how well they understand the material, enable us to address any misconceptions and make improvements to the presentations, and serve as an archival record of students’ progress throughout the class.

Guided worked examples - Worked examples, where the instructor demonstrates the problem solving process, are “particularly relevant to programs of instruction that seek to promote skill acquisition, e.g., music, chess, and programming”[2]. In designing examples to facilitate schema acquisition, it is important to provide labels and a visual separation of steps to “accentuate subgoals” [3]. We will reinforce labels and subgoals by annotating completed worked examples using Tablet PC inking.  Another technique thought to promote schema acquisition is to provide students with at least a second example.  It’s difficult, given our jam-packed curriculum, to accommodate multiple worked examples during class, thus we will make them available for required viewing over the web via manual download or RSS feed. 

Back to top

Proposed Technology

The technology used to support the above activities will include:

1. Software to produce screencasts (e.g., Camtasia Studio) of the priming presentations and worked examples.
2. Modifications to our current electronic classroom assessment system [5].  This web-accessible, database system was developed as a student research project we mentored with funding from the Computing Research Association’s Committee on the Status of Women (CRA-W).  It is based on the pen-and-paper Minute Paper Classroom Assessment Technique (CAT)[1], which asks students to articulate the most important thing they learned during the class session or lab and if they have any remain questions about the material.  Since 2005, these electronic minute papers have been used successfully in 19 different courses offered by five of the eight instructors in our department, primarily to obtain feedback during lab sessions that can later guide classroom instruction.  Our goal is to enhance the system to both assess this project, and to make it more flexible to support a wider range of pedagogical situations.
3. A Tablet PC for classroom use and video production. Currently the department owns one Tablet PC.  The addition of a second machine will allow both Laurie Murphy and David Wolff to produce videos, set up examples for classroom use, and use the PCs for classroom instruction without the burden of passing one machine back and forth.

Back to top

Selection criteria

Impact: We have realized positive benefits from electronic minute papers and guided worked examples in our classes, and believe multimedia based screencasts and annotated worked examples offer theoretically grounded and technologically relevant ways to further scaffold students’ learning.  The adverse impacts of poor pedagogical practices are more severe for underrepresented groups.  In computer science this includes women, who make up only about 25% of the students in our introductory courses.  Typically women in our courses have less pre-college programming experience than the men, are more likely to drop the course, and are less likely to continue on in the major.  Anything we can do to improve instructional practices will hopefully have a positive impact on this gender imbalance.

Innovation:  We believe the integration of screencasts, tablet PC annotations and electronic feedback based on sound pedagogical theories and applied to the highly complex and difficult task of learning to program makes this project unique.

Feasibility: Our department is housed in a new technology building with excellent computer classroom facilities to support the instructional aspects of the project.  We also recently obtained a new UNIX fileserver that will easily accommodate the enhanced electronic minute paper system requirements. We teach approximately 80-100 students in our introductory Java programming courses each year, which will allow ample opportunity to assess the projects’ effectiveness.  David Wolff and Laurie Murphy have both taught introductory Java programming for many years, have experience using Tablet PCs, and worked to implement and conduct educational research on the current electronic minute paper system [5].  David teaches the department’s database course, implemented the department’s online file submission system, and has conducted several tutorials on database and web technologies at regional CCSC-NW meetings. Laurie has studied educational theories and has participated in several multi-institutional, multinational CS education research projects investigating the knowledge and debugging skills of novice programmers. 

Technology Transfer and Outreach: We have a track record of sharing our classroom innovations and educational research results with colleagues through tutorials, papers, presentations and panels at CCSC-NW and SIGCSE conferences.  We are committed to doing the same should this project be funded.    

Outcomes Assessment: As mentioned above, we will use the modified electronic minute paper system to assess the effectiveness of the priming presentations and screencasts.  Because we offer multiple sections of our introductory class each term, we can easily use the results from experimental and non-experimental groups on traditional summative assessments, such as quizzes or exam questions, to assess the impact of the ink annotations on student learning.  A questionnaire will be given to students at the end of the term to obtain their overall impressions of the screencasts and annotated worked examples.

Accessibility and Security:  Providingmaterials for review on the web should make them more accessible to individuals with learning or attention-related disabilities. The ink annotations would also seem likely to facilitate greater understanding for students with hearing difficulties. One challenge of using multimedia learning theory principles to capitalize on the typical learners’ dual-channel processing capabilities is that it could make the material more difficult to grasp for those with reduced hearing or vision.  Currently we have a hearing impaired student enrolled in our Java course and have found that making slide notes available to her before class has made the material more accessible.  We will ensure that presentations are available not only as videos and screencasts, but also in a form that allows students to also view our notes or scripts.  The updated electronic minute paper system will be password protected to protect students’ privacy and system integrity.

Back to top

References

1. T. Angelo and P. Cross. Classroom Assessment Techniques : A Handbook for College Teachers. Jossey-Bass, 1993.
2. R. Atkinson, S. Derry, A. Renkl and D. Wortham, Learning from Examples: Instructional Principles from the Worked Examples Research, Review of Educational Research, vol. 70, 181-214, 2000.
3. M. Caspersen and J. Bennedsen. Instructional design of a programming course: a learning theoretic approach. In Proceedings of the Third international Workshop on Computing Education Research, 111-122, 2007.
4. R. Mayer. Multimedia Learning. Cambridge University Press, 2001.
5. L. Murphy and D. Wolff. Take a minute to complete the loop: using electronic Classroom Assessment Techniques in computer science labs. Journal of Computing Sciences in Colleges. 21:1, 150-159, 2005.

Back to top

Budget

Back to top

Schedule

Summer 2008:
            set up website
            evaluate options for screencast production software
make revisions to electronic classroom assessment system
            create priming presentations
            develop worked examples
            record and upload screencasts
            develop assessment survey
Fall 2008:
            write progress report
            use priming presentations and worked examples in CSCE 144
            collect assessment data
January 2009:
            evaluate assessment data
            write up results and submit a paper and/or tutorial proposal to a CSEd conference such as CCSC-NW
            update website
Spring 2009:
            write final report

Back to top