Author Archives: Dr Kristin Stephens-Martinez, Ph.D.

Some academic papers in CER

Leave a reply

Below is a collection of papers on student help-seeking and formative assessment. In conjunction with the reading academic papers assignment, pick 3 papers. For convenience, I have placed a copy of all of these papers in our Box folder. Remember, we can collectively edit that folder, so grab a copy for yourself, don’t edit it directly in Box if you take notes.

Undergrad TAs and Student Help-Seeking

Aaron J. Smith, Kristy Elizabeth Boyer, Jeffrey Forbes, Sarah Heckman, and Ketan Mayer-Patel. 2017. My Digital Hand: A Tool for Scaling Up One-to-One Peer Teaching in Support of Computer Science Learning. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 549–554. DOI:https://doi.org/10.1145/3017680.3017800

Yanyan Ren, Shriram Krishnamurthi, and Kathi Fisler. 2019. What Help Do Students Seek in TA Office Hours? In Proceedings of the 2019 ACM Conference on International Computing Education Research (ICER ’19). Association for Computing Machinery, New York, NY, USA, 41–49. DOI:https://doi.org/10.1145/3291279.3339418

Diba Mirza, Phillip T. Conrad, Christian Lloyd, Ziad Matni, and Arthur Gatin. 2019. Undergraduate Teaching Assistants in Computer Science: A Systematic Literature Review. In Proceedings of the 2019 ACM Conference on International Computing Education Research (ICER ’19). Association for Computing Machinery, New York, NY, USA, 31–40. DOI:https://doi.org/10.1145/3291279.3339422

L. Battestilli, M. Zahn, and S. Heckman, “Academic help seeking patterns in introductory
computer science courses,” in 2022 ASEE Annual Conference & Exposition, (Minneapolis,
MN), ASEE Conferences, August 2022. https://peer.asee.org/41526

Zhikai Gao, Sarah Heckman, and Collin Lynch. 2022. Who Uses Office Hours? A Comparison of In-Person and Virtual Office Hours Utilization. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2022). Association for Computing Machinery, New York, NY, USA, 300–306. https://doi.org/10.1145/3478431.3499334

Formative Assessments

Neil C.C. Brown and Amjad Altadmri. 2014. Investigating novice programming mistakes: educator beliefs vs. student data. In Proceedings of the tenth annual conference on International computing education research (ICER ’14). Association for Computing Machinery, New York, NY, USA, 43–50. DOI:https://doi.org/10.1145/2632320.2632343

Kristin Stephens-Martinez, An Ju, Krishna Parashar, Regina Ongowarsito, Nikunj Jain, Sreesha Venkat, and Armando Fox. 2017. Taking Advantage of Scale by Analyzing Frequent Constructed-Response, Code Tracing Wrong Answers. In Proceedings of the 2017 ACM Conference on International Computing Education Research (ICER ’17). Association for Computing Machinery, New York, NY, USA, 56–64. DOI:https://doi.org/10.1145/3105726.3106188

Shuchi Grover. 2021. Toward A Framework for Formative Assessment of Conceptual Learning in K-12 Computer Science Classrooms. Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, 31–37. DOI:https://doi.org/10.1145/3408877.3432460

Max Fowler, Binglin Chen, Sushmita Azad, Matthew West, and Craig Zilles. 2021. Autograding “Explain in Plain English” questions using NLP. Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21). Association for Computing Machinery, New York, NY, USA, 1163–1169. DOI:https://doi.org/10.1145/3408877.3432539

Assignment: Reading academic papers

Leave a reply

Due: Tuesday 9/20

Given what you learned in class, let’s practice it! For this week’s reading, do the following.

  1. Pick 3 papers to read using pass 1
  2. Of those 3 papers, pick 2 to read at the level of pass 2
  3. Of those 2 papers, pick 1 to read at the level of pass 3.
  4. Update your QQC doc to reflect your decisions and write a QQC entry for each paper.
  5. Add a separate section in your QQC doc with a Header 1 of “Reflection on Reading Academic Papers” and answer the following:
    1. What criteria did you use to choose the papers to move on to the next pass? Note this isn’t asking “why that particular paper,” it’s asking you to go meta and think about your decision process.
    2. How did it feel to read in a way that is different than reading every word?

Grading

  • Exemplary (10 points) – There is an entry for all three papers, noting which paper was which pass, and the reflection questions are answered.
  • Satisfactory (9 points) – All entries are there, but they are not clearly labeled. Reflection is present.
  • Not yet (6 points) – One pass is missing or the reflection is missing.
  • Unassessable (2 points) – There is an entry but it does not fulfill the Not yet criteria.

Regrade

You may redo the work and request a single regrade to update your grade.

How to read an academic paper & Group Project Discussion

Leave a reply

Reading academic papers is hard. In this class, we will discuss reading academic papers, how to apply this process, common pitfalls, and paper reading mindset.

Afterward, I’ll leave time for working on finding and forming groups. To help prepare for that, there is also a reading below on designing research questions.

Reading

Things to note

The above reading was not written by a CER researcher, so here is some interpolating:

  1. Here are some categories for CER papers
    1. Experimental studies
    2. Case studies
    3. Quasi-experimental studies
    4. Literature review
    5. Opinion paper
    6. Experience report
    7. Introduction of or application of an educational/CER theory
  2. CER Theories
    1. We have some, but there are not many
    2. Moreover, they are not widely used. If a paper does use one, this often strengthens the paper
  3. Mathematical proofs are rare in CER, the equivalent in CER is more the nitty gritty details of the method and data. It’s okay not to fully understand those after the second pass.

Handbook: Ch 3 Literature Review and Voices from the Field 3.1-3.3

Leave a reply

Read the following in The Cambridge handbook of computing education research of Chapter 3 “Computing Education: Literature Review and Voices from the Field”

  • 3.1 Introduction (1 page)
  • 3.2 Methods (1.5 pages)
  • 3.3 Findings: Rationales for Justifying CS Education (6.5 pages)

Add an entry about this reading to your QQC Doc.

Optional reading

“Good (and Bad) Reasons to Teach All Students Computer Science” episode from the #CSK8 podcast

Project Proposal

Leave a reply

Due: Thursday 9/29

General Directions

The purpose of this document is to prepare your team for success in the course project. Your proposal should contain at least three parts, which we define below. In terms of length, it should be 2-3 pages using standard margins (1 in.), font (11-12 pt), and line spacing (1-1.5). In addition to these three components, you should provide any additional context or information necessary to understand your vision for your project. You should convert your final document to a pdf and upload it to Gradescope under the assignment “Project Proposal” by the due date. Be sure to use the group submission feature on Gradescope to include all of your group members in a single submission.

Introduction and Research Question

Your proposal should introduce your topic in general and motivate why your research question is relevant. Relevance addresses the importance and interest of your research question to the CSEd community (or at least this class). It should then define one or more research questions. The research question(s) should be substantial and feasible.

  1. Substantial research questions require more than a surface-level analysis (more than just computing basic summary statistics on readily available datasets, for example).
  2. Feasible research questions can actually be addressed by your group over the course of approximately eight weeks, including writing a report and creating a presentation.

Related Work with Status

Your research questions should be informed by related work. This section should summarize the key takeaways from the related works you have read so far with a note on the reading pass level for that work (see the reading academia papers assignment). There should be at least 3 works found by the time of the proposal that have been read at least at a pass 1 level. And at least 4 more that will potentially be useful but only the title and abstract have been read to determine their potential relevance.

The works do not need to be very similar to the research question you are proposing. It is okay if they simply motivate why your research question is relevant (For example, if your research question is about UTAs, a related work could be how novices struggle to learn computer science material, which motivates why UTAs are important).

Overall, for a related work section in any project report, not all of your related work needs to be read at the pass 3 level. In fact, most of them will not need that level of a pass. Only a handful will need to be read at a pass 3, a few more at pass 2, and most of them will be at a pass 1 level. How many fall in each bucket will depend on how similar a given related work is to your research question.

Collaboration Plan

While working in pairs or triples usually does not require a lot of coordination, having a frank conversation on working styles, communication expectations, etc. is important for any team. Therefore create a plan that addresses the following:

  1. How will you divide responsibilities? Will some students be responsible for certain portions of the project, or will you be more integrated and decide on responsibilities every week?
  2. About how much time do you expect every group member to spend on the project each week, on average? It is ok if this number is higher toward the last couple of weeks of the semester.
  3. When and how will you meet? You should plan to meet at least once per week for at least 30 minutes to check in on one another’s progress, get help, and plan for what comes next. Identify a day of the week, a time, and the platform you will use to meet.
  4. What platform(s) will you use to communicate between meetings? Will you primarily use email, text, slack, or other chat apps? If you want a more professional enterprise tool, Duke provides free access to Microsoft Teams.
  5. Where will you store data, code, writing, etc., so that all group members can easily access shared materials? Duke provides free access to Box and GitLab, which could serve these purposes, but you could also use external services like Google Drive or GitHub. Provide a link to the folder/repository in your proposal to demonstrate that it is created and ready. Remember that data should only be on Duke-provided tools (e.g. Box) and never in a version-controlled repository.

Feedback and Grading Rubric

Each section will be graded on a four-step rubric scale as follows.

  • E (Exemplary) – Work that meets all requirements of that section.
  • S (Satisfactory) – Work that meets all requirements with only slight mistakes or missing pieces of information.
  • N (Not yet) – Work that does not meet some requirements and/or displays developing or incomplete work that needs substantial revision to meet satisfactory standards.
  • U (Unassessable) – Work that is missing, does not demonstrate meaningful effort, or does not provide enough evidence to determine a level of mastery.

The entire assignment is worth 100 points.

  • 10 points will be allocated for meeting general directions (length, on-time pdf submission, group submission, etc.). You cannot submit a proposal greater than 3 pages. Learning how to be succinct is an important skill.
  • 30 points are allocated for each section.

The rubric will be converted to points as follows:

  • E = full credit
  • S = E_full_credit – 1
  • N = E_full_credit / 2
  • U = E_full_credit / 5
  • Blank = 0

Anything earning less than an E will receive feedback in Gradescope. If your proposal earns less than an S in any section you will be allowed 2 resubmissions to bring it up to the E or S standards for all sections. If your proposal earns E’s and S’s only, you can have 1 resubmission if your group decides to aim for a higher score. Each resubmission must be done within 1 week, starting from when the feedback is returned. This is to limit the amount of time spent on the proposal for all those involved.

Using Word & How to have a good discussion presentation

Leave a reply

In this class, we will first briefly discuss the one Word feature that I (Prof. Stephens-Martinez) still use and why you should consider using it as part of your writing process. The rest of the class will be an overview on how to have a good discussion presentation and discuss projects if we have time.

Supplemental Material

QQC Doc Check-in

Leave a reply

The main point of this check is to confirm you are doing your due diligence in following the directions about what goes in your QQC Doc. In the real world, no one actually checks your notes to hold you to some standard. The only person that might ever care is the future you that has to go back and use those notes. However, the intention behind this exercise is to provide structure and help you practice reasonable organization and note-taking skills with me (Prof. Stephens-Martinez) to hold you accountable.

There are 10 possible points for each check, and they will be awarded as follows:

  • Exemplary (10 points) – Formatting is correct for all entries (title, citation, link if needed), and there are at least three “things” per entry. A thing can be a question, comment, or quote, but you do not need one of each.
  • Satisfactory (9 points) – Some formatting is incorrect, but otherwise, everything is there.
  • Not yet (6 points) – It is clearly missing one or more entries or frequently has fewer than three “things” per entry.
  • Unassessable (2 points) – The document exists and there is an attempt at at least one entry, but otherwise, there is little there.

Regrade

You may redo/update the work and request a single regrade to update your grade. To ask for a regrade email the instructor with the subject: “CS290 Regrade: QQC Doc Check-in” include a link to your QQC doc.

Checklist for QQC Check-in 1, due Thursday, 9/29

  1. Handbook: Ch 1 History of CER
  2. Handbook: Ch 3 Literature Review and Voices from the Field Part 1
  3. Some academic papers in CER: Student help-seeking and formative assessments
    1. 3 entries, one from each paper in that assignment
  4. BYOPaper: ICER or Handbook

Checklist for QQC Check-in 2, due Thursday, 11/3

  1. Handbook: Ch 3 Literature Review and Voices from the Field Part 2
  2. How to write briskly and well
  3. Chapter in Critically Conscious Computing
  4. BYOPaper: SIGCSE or Handbook
  5. Handbook: Ch 16 Equity and Diversity

Checklist for QQC Check-in 2, due Thursday, 12/8

  1. Handbook: Ch 14 Assessment and Plagiarism
  2. Teaching theoretical computer science using a cognitive apprenticeship approach
  3. BYOPaper: CER Journal

Handbook: Ch 1 History of CER

Leave a reply

Read the following in The Cambridge handbook of computing education research

  • 1.1 The Scope of Computing Education Research (2 pages)
  • 1.2 Early Studies in Computing Education Research (5.5 pages)
  • (optional) 1.3 Redesigning the Learner’s Interface for Computing Education (2 pages)
  • 1.4 Enter the Education Researchers (4 pages)
  • (optional) 1.5 Computing Education Emerges as a Research Discipline (2 pages)
  • 1.6 Research Questions in Computing Education Research (2.5 pages)
  • 1.7 Conclusion: Future Research Questions in a Historical Context (2.5 page)

Add an entry about chapter 1 of the handbook to your QQC Doc. If you are wondering how much to write as the QQC’s for the entry, check out the QQC Doc Check-in post.

QQC Doc

Leave a reply

Due: Thursday, 9/1

QQC stands for Question, Quote, or Comment. The QQC Doc is a Google document shared with the instructor of the class and will serve as a bibliography and notes for the readings. To start, create and share a Google document with ksm@cs.duke.edu

The Google document should follow a particular format to help with navigation. To make things easier here is a Google doc template. Clicking on this link will prompt you to make a copy of the Google doc.

Format

  • Google doc name: “QQC Doc: Given_name Surname” – replace Given_name Surname with your name such that it can be matched to your name in the gradebook.
  • Contents for each reading
    • Title of the reading with the format “Heading 1”, so they appear in the document outline.
    • Citation – Must contain title, author, publication venue, and year (Google Scholar’s Cite feature will help)
    • Link to the reading (not needed for readings from the Handbook, but could be useful to you)
    • Your Question(s), Quote(s), and Comment(s) about the reading. If you are wondering how much to do, check out the QQC Doc Check-in post.

Supplement

You can learn about QQC from 5 Ways College Teachers Can Improve Their Instruction by the Cult of Pedagogy podcast.

Grading

The grading will be based on whether you created the document based on the above instructions. There are 10 possible points for each check and they will be awarded as follows:

  • Exemplary (10 points) – By the due date the document exists, is shared with the instructor, and is named correctly.
  • Not yet (6 points) – The document exists, is shared with the instructor, but is not named correctly.

Regrade

You may redo/update the work and request a single regrade to update your grade. To ask for a regrade email the instructor with the subject: “CS290 Regrade: QQC Doc Creation” include a link to your QQC doc.