JiTT@OS2017

What and Why

A Revelation

• Lectures in module “Computer Structures and Operating Systems” in summer 2016
  • I made intense use of a classroom response system
    • Quizzes testing students’ understanding during classes
• Quiz results continuously worse than anticipated
  • Yet, students did not engage in discussion
• Then I did this poll:

What is the Problem?

• Regardless of my satisfaction with my lectures, learning outcomes are unsatisfactory
  • Yet, students do not seem to care
• Notice: I would not have known without classroom response system…
  • What about your classes?
• Asides
  • Not an issue: Students liked my classes
    • Average grade in end-of-term evaluation was 1.8
  • A separate issue: Class participation
    • At most 50% of students taking exam attend classes; typically about a third

Are Lectures a Waste of Time?

• I think so.

  

  Figure 2:

  Image CC0; converted from pixabay

• Others agree
  • [SR17] on Vermont medical school: “Saying Goodbye to Lectures in Medical School—Paradigm Shift or Passing Fad?”
    • “60 slides in 45 minutes may seem like an efficient way to teach, but it is unlikely to be an effective way to learn”
    • Prefer flipped classroom, “active learning” [FEM+14]

Cognitive Load Theory [DeJ10]

• Capacity of working memory (aka short-term memory) is limited
• Learning material and processes come with different cognitive loads
  • Germane loads, associated with learning processes
  • Intrinsic loads, related to difficulty/complexity of topics
    • Element interactivity is central factor
  • Extraneous loads, related to suboptimal instructional procedures
• If combined loads too high, cognitive overload of working memory hinders learning

Cognitive Load Impact

• Either I need to reduce the intrinsic load

  

  Figure 3:

  Image CC0; converted from pixabay

  • (Lecture about “easier” topics, less topics)
• Or I need to move the locus of learning
  • JiTT
  • Flipped classroom
• (Reading assignments did not work, experiment in Master course just ended)

JiTT@OS as Digi-Fellowship

• Digi-Fellowship
  • Funded by Ministry of Innovation, Science and Research, NRW, Germany and Stifterverband.
• Introduce Just-in-Time Teaching
  • Lectures on Operating Systems (part of CSOS)

JiTT Big Picture

Just-in-Time Teaching

• http://jittdl.physics.iupui.edu/jitt/
• Teaching and learning strategy [NPG+99],[MSN16]
• Core idea: Feedback cycle among students and instructors
  • Students work through educational resources and submit solutions to pre-class assignments
    • They discover challenges throughout the term
  • Instructors correct students’ solutions before class meeting
    • Instructors adjust class topics just-in-time

JiTT Benefits

• Students learn actively and continuously
  • Based on their individual background
  • At their individual pace
• Instructors identify difficulties, misunderstandings, and incorrect prior beliefs
  • And react accordingly just-in-time
• Shared class time is used more efficiently and more effectively
  • Focus on interesting aspects
  • Interaction instead of passive listening

JiTT for OS 2017

• Presentations as OER at https://oer.gitlab.io/OS
• Source material at https://gitlab.com/oer/OS

OS 2017 in Numbers

• 106 – 91 students took mandatory graded tests
  • Typically, 45 – 30 students attended class meetings
• 86 students wrote end-of-term exam, 61 passed
• 79 students worked on at least one JiTT assignment (out of ten)
  • 28 on all
  • 51 on eight or more

On JiTT Assignments

• Lots of work
  • Creation of questions as well as correction of answers
    • In particular, redundant work for JiTT corrections when similar problems required similar feedback
      • (Also, individual students are blocked by similar challenges)
    • Next installment may use online tool for discussions
• Worth the effort
  • Lots of surprising answers/insights
  • Less surprising, yet highly relevant: Knowledge from prior/related classes may be missing
• I improved/extended every presentation after learning about students’ difficulties
  • From wording to sequences of new slides

Particularly Noteworthy Answers

• Making me happy
  • Suggestion of in-class game
  • Various presentations/animations of algorithms
• Shocking

  • 8 out 56 students found it hard or impossible to locate the text for the following task:

    “Submit your solutions to Exercise 2.3 of [Hai17].”

Students’ Understanding (1)

Do you believe that JiTT assignments combined with in-class meetings improve or decrease your understanding of course topics compared to traditional lectures?

Students’ Understanding (2)

Side note: In absolute numbers, 66% in 2017 amounts to 36 students.
37 students scored 75% or more in a related exam question.

Students’ Understanding (3)

Workload

Average increased from 4.1 to 5.2 hours per week (for module with 6 hours per week)

Class Evaluation

Average improved from 1.8 to 1.6; however, two outliers

Conclusions

Findings

• Subjective
  • Much more fun in class
    • (Traditional lectures lost fun for me)
  • Surprises out of class
• Objective
  • Students who participate like JiTT
  • Exams results without significant change from previous year
    • However, with correlation to JiTT activities

Announcement by DBIS Group

• We will flip courses starting in January 2018
  • Data Integration
  • CSOS
• MIS/DWH to be discussed

Future Work

• How to increase student participation?
  • In particular, less successful ones?
• Roll out JiTT
  • In your courses, too?
  • (Broader exposition to JiTT might change students’ mindsets…)
• Term structure in studies?
  • Fewer courses in parallel, each one more intense
  • (Avoid simultaneous workload peaks of independent courses)

Different Topic: OER

• Open educational resources (OER)
  • Teaching material under free licenses such as Creative Commons
    • Five Rs (retain, reuse, revise, remix, redistribute; originally four [HWSJ10])
  • Self-study presentations for Operating Systems as OER
    • https://gitlab.com/oer/OS
    • (Source code for this presentation)
• Created with my own software, emacs-reveal
  • In line with ALMS framework of [HWSJ10]
  • Suitable for Single Sourcing [Roc01]

Discussion

Bibliography

• [SR17] Schwartzstein & Roberts, Saying Goodbye to Lectures in Medical School—Paradigm Shift or Passing Fad?, New England Journal of Medicine 377(7), 605-607 (2017). http://dx.doi.org/10.1056/NEJMp1706474
• [FEM+14] Freeman, Eddy, McDonough, Smith, Okoroafor, Jordt & Wenderoth, Active learning increases student performance in science, engineering, and mathematics, Proceedings of the National Academy of Sciences 111(23), 8410-8415 (2014).
• [DeJ10] De Jong, Cognitive load theory, educational research, and instructional design: some food for thought, Instructional science 38(2), 105-134 (2010).
• [NPG+99] Novak, Patterso, Gavrin & Christian, Just-in-Time Teaching: Blending active Learning and Web Technology, Prentice Hall, 1999.
• [MSN16] McGee, Stokes & Nadolsky, Just-in-Time Teaching in Statistics Classrooms, Journal of Statistics Education 24(1), 16-26 (2016).
• [Hai17] Hailperin, Operating Systems and Middleware – Supporting Controlled Interaction, Revised edition 1.3, 2017. https://gustavus.edu/mcs/max/os-book/
• [HWSJ10] Hilton, Wiley, Stein & Johnson, The four ‘R’s of openness and ALMS analysis: frameworks for open educational resources, Open Learning 25(1), 37-44 (2010).
• [Roc01] Rockley, The Impact of Single Sourcing and Technology, Technical Communication 48(2), 189-193 (2001).

License Information

Except where otherwise noted, this work, “JiTT@OS2017: Why and how not to lecture”, is © 2017 Dr. Jens Lechtenbörger, under the Creative Commons license CC BY-SA 4.0.

No warranties are given. The license may not give you all of the permissions necessary for your intended use.

In particular, trademark rights are not licensed under this license. Thus, rights concerning third party logos (e.g., on the title slide) and other (trade-) marks (e.g., “Creative Commons” itself) remain with their respective holders.