Student-Generated Evaluation Criteria

Title

Student-Generated Evaluation Criteria

Description Librarians and instructors can flip an explanation of evaluation criteria by having students in the class generate a list of qualities and characteristics they think are important when selecting sources for their assignments. This will engage students in what they would look for. Librarians and instructors can help facilitate – and add to – the list generated by the students.

Questions (can be adapted):

  • What criteria do you think are important for figuring out how “trustworthy” or “reliable” a source is?
  • How do you know to trust someone or something?
  • What qualities would you look for in a source?
  • What criteria would you use to figure out whether a source is helpful for you for a paper/assignment?
  • What do you think makes a relevant/good article?
Steps
  1. Ask students to help generate a list of criteria they think are relevant for assessing “quality” in a source.
  2. Document their criteria on a whiteboard, or a shared list, and ask students to refer to this list of criteria as they are searching for, and selecting, sources.
Tags evaluating-sources; in-class;
Time 10 minutes (during class)
Attachments None.

Paywalls and Information Costs

Topic

Paywalls and Information Costs

Key Takeaways
  • Scholarly publications are not typically free to access and subscription costs for certain resources are exorbitant. Journals in certain disciplines (especially engineering, business, & medicine) have higher price tags than others.
  • While a lot of scholarly content is discoverable on Google, attempting to access the full text often leads to a page from a publisher asking for money, a paywall.
  • Since individuals cannot afford to buy/subscribe to journals, university libraries pay large sums of money to provide institutional access.
 

Notes for the instructor/librarian

 

The topic of paywalls and information costs would be good for any intro-level student to learn about early on. Paywalls are a jarring reminder of the fact that access to scholarly publications is restricted and costs a lot of money. This topic may naturally inspire discussion of “information privilege” and the impact paywalls may have on researchers without institutional access. The issue of piracy (e.g., Sci-hub) might also come up and librarians and instructors should be prepared to discuss the ethics of this.

Lesson Content Below you will find a variety of ways to briefly address this topic. Pick, choose, and adapt as you like.
Quick Mentions
  • When teaching a class and discussing databases, talk to them about how much the library pays to subscribe and provide access. Also mention that students will lose access to most of our databases upon graduation.
  • Mention alternative Open Access sources of information: public libraries, Google Scholar, unpaywall.org, Open Access Button, etc.
  • When talking to students about researching from off campus, explain paywalls – 1). why they exist (conceptual); 2). what to do when you hit them (practical – go through the library’s website).
  • Factoids by the numbers (based on Duke Libraries research):
    • 59 of 100 of the most highly cited articles ever published are behind a paywall.
    • The average cost of one of these articles for an unaffiliated researcher is $33.41. One Nature article typically costs an unaffiliated researcher $32. 
    • If the 244,133 authors who cited the article “CLEAVAGE OF STRUCTURAL PROTEINS…” (1970) paid out-of-pocket for access to this Nature article ($32), the price tag would total $7,821,256.
Discussion Questions
  • Have you run into paywalls on the internet? What were you trying to access when you hit the paywall? What did you do? Give up? Pay? Go elsewhere?
  • What types of information do you think should be free (ex. News? Entertainment? Medical? Government-funded research?)?
  • React to this statement: “A huge part of research output is suffocating behind paywalls. Sixty-five of the 100 most cited articles in history are behind paywalls… That’s the opposite of what science is supposed to do…We’re not factories producing proprietary knowledge. We’re engaged in debates, and we want the public to learn from those debates.” — Guy Geltner, professor/historian at the University of Amsterdam & open access advocate, Wired magazine
  • React to this statement about The Guardian newspaper: “I appreciate there not being a paywall: it is more democratic for the media to be available for all and not a commodity to be purchased by a few. I’m happy to make a contribution so others with less means still have access to information.” Guardian reader
  • What is the impact of information costs on under-resourced institutions?
Visuals & Media

(See all content in Google Drive)

  • Tenontosaurus example: This is what hitting a paywall looks like in real life and how researchers get around this (#icanhazpdf). This example illustrates a very common phenomenon and introduces the phenomenon of the hashtag #icanhazpdf with the solution (in this example) to retrieve the article from an online piracy site (Sci-Hub).
    • Activity variation: Search for the hashtag #icanhazpdf on Twitter. What are you noticing? 

  • Screenshot of Google Results with $ signs next to the paywalled items.

  • A Sciencedirect “Shopping Cart”:

  • Paywall memes. Could be inserted into presentation slides.

Activities

(See all content in Google Drive)

  • Bibliography Price Tags: Give students a small list of sources and ask them to total up the cost of each source using the web to find prices. Discuss: Were you surprised with the prices? Why do you think different kinds of information cost different amounts of money?
    • Kozlowski Ryszard M., Mackiewicz–Talarczyk Maria, Muzyczek Malgorzata & Barriga–Bedoya Jorge (2012) Future of Natural Fibers, Their Coexistence and Competition with Man-Made Fibers in 21st Century, Molecular Crystals and Liquid Crystals, 556:1, 200-222, DOI: 10.1080/15421406.2011.635962
      $________________
    • Dani Setiawan, Jeffrey Brender & Yang Zhang (2018) Recent advances in automated protein design and its future challenges, Expert Opinion on Drug Discovery, 13:7, 587-604, DOI: 10.1080/17460441.2018.1465922 $________________
    • Pickles Sector: Worldwide Forecast until 2022 (2018). https://www.reportlinker.com/p03859343/Global-Pickles-Market.html $________________
    • Oxford Textbook of Global Public Health, 6th Edition. Available for rent on Google Play. $________________
    • Answer key: $50+$89+$2,500+151.99=$2,790.99
    • Variation – which do you think has the highest price tag? Why?
  • “Which cost more?” / Price is Right-style game: Show students a few different resources and ask them to guess the price.

Student Readings

Journal Prestige

Topic

Journal Prestige

Key Takeaways
  • All scholarly sources or peer-reviewed journals are not held in equally high regard.
  • The most common measures of journal prestige, citation count and the related Impact Factor, have limitations and are not a direct measure of quality of the scholarship therein.
 

Notes for the instructor/librarian

 

This topic could fit well into instruction sessions that include significant treatment of source evaluation, and is one potential approach as you move beyond simple categorization of sources as scholarly/non-scholarly or primary/secondary. It stops short of a critical examination of construction of authority, but could be used to hint at greater subtlety and complexity. This topic has particular relevance for upper level undergraduates engaged in research, who may be starting to think about publication from an author’s perspective.

Lesson Content Below you will find a variety of ways to briefly address this topic. Pick, choose, and adapt as you like.
Quick Mentions
  • Comparing journal prestige to university/college prestige: one way that reputations are built or maintained is through exclusivity and high rejection rate. Is this the same thing as quality?
  • When showing students how to sort by times cited in a database: of course, most highly cited doesn’t mean best. A highly cited source might be controversial, or just published longer ago and had more time to get cited….
  • There are incentives for journals to select manuscripts based on their likelihood to make a big splash and generate a lot of citations. This might actually result in publishing studies that are more likely to be overstated, flawed, or eventually retracted.
  • Different numbers of researchers writing on a given topic and different citation practices in a given field can drive huge differences in citation rate. You can’t make comparisons across disciplines.
Discussion Questions
  • How do you think your professor would evaluate a source in her area of expertise? Are you able to do that kind of evaluation with your current knowledge?
  • What factors do you think would make an article likely to get a high number of citations?
  • Are there reasons an article might get a high number of citations other than the quality of the research?
  • If you only look at the most highly cited sources, what would you be likely to miss?
Visuals & Media

(See all content in Google Drive)

  • Create a course-relevant visualization using one of the Scimago Journal Rank visualization tools. (SJR is based on Scopus data. Some details are available on their About page.) Alternatively, create a prompt for students to explore the visualizations themselves. As an example, you might use the Shape of Science visualization, setting the labels to Subject Areas, perhaps pruning to the top quartile, and zooming in to a given field of study. What questions can be answered, or not, based on these rankings and relationships?

 

Activities
Student Readings

Subscriptions and Open Access

Topic

Subscriptions and Open Access

Key Takeaways
  • Fees for access are not the only possible source of revenue for publishers, and charging publication fees to authors is a common practice.
  • There are implications of subscription vs open access models in terms of access to information for readers and access to publication opportunities for authors
  • (Advanced/Extension) There are nuances to open access models (e.g., hybrid journals) and alternatives to simple author-pays (e.g., “green” open access or library sponsorship initiatives).
 

Notes for the instructor/librarian

 

This topic explores publishing business models, and may only be relevant for instruction sessions where the librarian is able to devote significant time to scholarly communication topics. It could be used as an extension to “sticker shock” topics, or possibly when discussing paywalls or information privilege. It might arise naturally in student discussion of those topics, and so could be useful to have considered in advance.

Lesson Content Below you will find a variety of ways to briefly address this topic. Pick, choose, and adapt as you like.
Quick Mentions
  • “How much do you think authors of scholarly articles are paid?” Briefly mention that, in contrast to commercial writing and to most books (including scholarly ones), they are not paid. Peer reviewers aren’t paid either….
  • Aside from high profit margins made by some for-profit publishers, there are legitimate costs associated with running a journal. If a journal is free to read, these costs aren’t recovered through subscriptions. The most common way to fund these journals is through fees paid by authors….
  • Open access publications are typically not just free to read but are also released under a license that permits some level of sharing and reuse, such as CC-BY. Briefly explain the differences between copyright and open licenses.
  • Open access isn’t just for journal articles. If relevant to the course, you could mention various open textbook initiatives, library-funded projects like Knowledge Unlatched, and/or publicly-funded resources like the NCBI Bookshelf.
Discussion Questions
  • What kinds of information do you pay to access? For the information you access without paying, how is its creation funded?
  • [If following discussion of information privilege or similar topic] We’ve talked about disparities created by charging readers for access to information. What disparities could arise when authors have to pay in order to publish their work?
  • Have you encountered Creative Commons or other open licenses? What do they mean? Would you publish your work under an open license? Why or why not?
  • A great deal of research is funded with taxpayer money, for example by NIH or NSF in the United States. Sometimes these funders require open access versions of the resulting publications to be made available, perhaps after some delay. What are the benefits of these policies? What are the potential problems? For the public, for the researchers, for other stakeholders?
Visuals & Media

(See all content in Google Drive)

  • On the topic of what it costs to run a journal, some OA publishers are quite transparent. See image below from Ubiquity Press, which charges low APCs starting around $550US https://www.ubiquitypress.com/site/publish/

Student Readings

Scale of Scholarly Publishing

Topic

Scale of Scholarly Publishing

Key Takeaways
  • Scholarly publishing is an industry whose products range from books to journal articles to data. Millions of articles are published every year from hundreds of countries around the world.
  • (Advanced/Extension) Only four to five major publishers control the majority of scholarly articles published, which leads to rising prices and less choice for libraries when negotiating contracts for purchasing access to content.
 

Notes for the instructor/librarian

 

Introducing this topic could be as simple as indicating the impressive number of scholarly articles published each year or size of library collections budgets, or be part of a lengthier lesson on how academic publishing works. It could be included in searching or source evaluation exercises, and may set the stage for understanding the fundamentals of scholarly communication.

Lesson Content Below you will find a variety of ways to briefly address this topic. Pick, choose, and adapt as you like.
Quick Mentions
  • In 2016, approximately 2.2 million scholarly articles were published across the globe. The United States produces the most articles, followed closely by China. [AJE Scholarly Publishing Report, 2016]  
  • The first journals were produced in the eighteenth century as a way for members of scientific societies to share their discoveries with one another. In the four centuries since, publishing has become a multibillion-dollar industry that facilitates the dissemination of articles, books, and data.
  • The Duke Libraries pay over $13 million a year on subscriptions (i.e., journals and databases) so students, faculty, and researchers can have access to the latest literature.
  • Authors are not paid to produce and publish articles, but readers have to pay for access; libraries are buying back content their institution’s researchers produce for free.
  • (Advanced/Extension) Publishing has been consolidated into almost a near-oligopoly held by four major publishers: Reed-Elsevier, Wiley-Blackwell, Springer, and Taylor & Francis. These companies reap massive profits every year by charging universities to buy back the content their researchers produce. And they keep buying up smaller journals every year.
  • (Advanced/Extension) Not all journals charge to access their content. Open access journals have a different business model, in which fees are usually charged to authors and articles are free to read online.
Discussion Questions
  • What is the predominant medium of sharing research in your field and how does that affect the way you access current literature?
  • Where in the life cycle of publication are you, as an undergraduate researcher, situated?
  • As many journals shift from print to digital format, and are no longer producing and disseminating physical issues, should subscription costs go down? Why or why not?
  • Buying a physical book is a one-time cost for the library, but a journal subscription is something that has to be renewed annually. Without the subscription, the library may lose access to the journal’s content. Is that really “buying” a subscription or “leasing” the research?
Visuals & Media

(See all content in Google Drive)

  • A table comparing the profit margins of major companies with publishers. Illustrates the high profit margins of scholarly publishers, Taylor & Francis and Elsevier.

Profit Table with Revenues

Student Readings

Information Privilege

Topic

Information Privilege

Key Takeaways
  • Lots of scholarly information is inaccessible to people outside of large, well-funded research universities like Duke.
  • There are barriers to information access that people might encounter due to their geography, access to technology, identity, status, financial situation, etc.
  • The Open Access movement seeks to address these disparities.
 

Notes for the instructor/librarian

 

This topic could potentially be a touchy subject for students who haven’t thought about these aspects of privilege previously and should be framed carefully. It could be a great fit for an undergraduate course that has any social justice component. Students in these classes might be better primed for talking about disparities/inequalities and could apply that to this concept of information access. However, this discussion can be applicable to any level/discipline.

Lesson Content Below you will find a variety of ways to briefly address this topic. Pick, choose, and adapt as you like.
Quick Mentions
  • When teaching a class and discussing databases, talk to students about how they will lose access to most of our databases upon graduation.
  • Mention alternative sources of free and open access information: public libraries, Google Scholar, unpaywall.org, Open Access Button, etc.
  • Mention to students that they will likely rarely encounter an item (book, article) they can’t get through Duke Libraries, but many students at smaller colleges or researchers outside of the United States and western Europe may face significant barriers to getting the academic material they need.
Discussion Questions
  • Who has access to scholarly information and who does not?
  • Should access to scholarly information be free (why or why not)?
  • Should access to information be a human right? (Give an example: study reporting experimental cancer treatments; access to legal information for someone who is incarcerated). If not, are there certain types of information that you think should be free vs. others that should have a fee?
  • What barriers to information access can you think of?
  • “Access to science is going to be a first-world privilege … that’s the opposite of what science is supposed to be about.” — Guy Geltner, Professor of History, University of Amsterdam
Visuals & Media

(See all content in Google Drive)

  • Invisible Knapsack / Information Privilege graphic – 

Information Privilege Backpack

The “invisible knapsack” was introduced by Peggy McIntosh in a 1989 essay she wrote on “white privilege.” The essay describes what privilege (in this case, white privilege) looks like in everyday situations. This image riffs off of this and presents examples of what information privilege might look like. The image could spark a conversation for students about the privileges around info access they have experienced, but had not thought about.

  • Memes

  • Defining Information Privilege

  • Digital Divide – who has access to the internet & who does not. This infographic from the IMF illustrates some of the issues surrounding the global digital divide.

Activities

(See all content in Google Drive)

  • Game / Quiz / Survey
    • Should these be free … (Why, or why not??)
      • Slide provides some examples of source types (popular movie, book, journal, engineering standard, market research report) to serve as a discussion about what kind of information is free/not free. Could adapt to examples from the topic/discipline of the course.
  • Case Studies / Scenarios (similar scenario-based lesson plan from Project CORA). Notes for instructor: Provide background on results of lack-of-access in these scenarios – in journalism, in community colleges, in politics.
      • Example: A student at a community college is taking a Biology class and researching zebrafish. The student finds an interesting article in his Google search results, but hits a page asking for money (a.k.a. a paywall). Instead of paying $45(!) for the article, the student finds some alternative information on the topic in a Wikipedia article.
      • Example: A journalist for a local independent newspaper is doing some investigative research on the growing trend of “white flight” between students attending public schools and charter schools in the community. The journalist finds some interesting articles in scholarly education journals and books listed on Google, but her newspaper does not have subscription access. What should she do?
      • Example: You are graduated and interning on Capitol Hill. Your boss asks you to find the latest information on water safety and public health. You’re able to find some readily available sources, but most of the evidence-based/scientific water studies are behind a paywall. What do you do?
  • Information Privilege Walk: An activity where the facilitator (instructor or librarian) asks students to stand up and line up on one side of the room. The facilitator reads a list of questions related to information privilege. Students take a step forward if they answer yes, and step backward if the answer is no. Note for instructor: This activity would be best done when facilitator has a rapport with the students, as it may make some students uncomfortable. It could be adapted to be an used as an anonymous questionnaire, or a list of questions for discussion, if anonymity makes more sense. See Google Drive folder for sample Information Privilege questions. 
Student Readings

Library Icebreaker

Title

Library Icebreaker

Description Librarians can flip key using-the-library talking points and engage students in discussion with this icebreaker activity. These questions address some of the top/most-frequently-pondered questions that first year students have about using the library.

Questions (can be adapted):

  • What is Google Scholar?
  • What if a book or article I need is not at Duke?
  • Is there an easy way to cite & organize all my sources?
  • Isn’t everything on Google?
Steps
  1. Pass index cards out (1 per row/3 students to a card)
  2. Ask students to discuss with their partner/group what they think the answer is to the question. Give them 1-2 minutes.
  3. Ask each group to state their question and answer to the class. Librarian can chime in with additional info/answers to the question.
Tags beyond-google; in-class;
Time 10 minutes (during class)
Attachments None. Index cards on Lilly podium, in Music Library, Bostock 023, or make your own!

Finding a Book

Title

Finding a Book

Description In post-session feedback, first-year students frequently express anxiety over how to physically navigate the library to find a book on the shelf. This is a simple, pre-session activity to help students try this out before class, so that they can discuss with their librarian any challenges they faced in attempting to complete the task. With the help of the course instructor, students are asked to find a book on their research topic (or course topic) and bring it to class.
Steps
  1. Ask students to use the library’s website (https://library.duke.edu/) to search for a *print/physical* book on their research topic or course topic if they don’t have research topics yet. NOTE: Share the worksheet with students, it lists out all these steps.
  2. Students should then find the book in the library where it is located, check it out, and bring it to class.
  3. Students could write a few sentences about any obstacles they faced, or observations they had about navigating the library, etc.. This will help prime them for a discussion.
Tags beyond-google; in-class; pre-assignment
Time 20 minutes (before class); 10 minute discussion
Attachments Finding a Book (worksheet)

Choosing a Topic Flowchart

Title

Choosing a Topic Flowchart

Description This activity helps students pick or refine a topic that is of personal interest and meets the criteria for their assignment. It is based on the idea that students are more engaged with topics that are of personal interest. It can be used as a standalone worksheet for students to use or can be used as a pre-class activity for them to complete before coming to a library session or other class session.
Steps
  1. Before giving the students the worksheet, model how to use it with your own example
  2. Share the flowchart worksheet
  3. Give students 5-10 minutes to fill out the worksheet if used as an in-class activity; have them complete the worksheet before class if preferred.
Tags refining-topic; in-class; pre-assignment
Time 10-15 minutes
Attachments noun_229116Choosing a Topic Flowchart

Classifying Sources: the BAAM Method

Title

Classifying Sources: the BAAM Method

Description This is an activity that uses the framework of BAAM* as a way of helping the students to understand how they would use different kinds of sources in their research papers. BAAM is a way of classifying sources as: Background; Artifact; Argument; Method. The BAAM method is introduced to students before hands-on searching, and then afterwards, as the students begin classifying their results. *Note: We have modified BAAM from Joseph Bizup’s BEAM Method. Substituting Exhibit for Artifact.
Steps
  1. Provide students with the worksheet.  Explain the different categories, using examples which are relevant to the course content or assignment
  2. Teach the navigation/research portion of the class
  3. At the end of the allotted time, have students volunteer some of their search results and how they will be used according to the BAAM classification
  4. Ask them questions, e.g. “Who has research which is an example of ‘background’,” “Who has research which is an example of ‘artifact’,” etc.
Tags  evaluating-sources; beyond-google; in-class
Time 20-25 minutes total (1o minutes to explain concept; 10-15 minutes to discuss at end of class)
Attachments noun_229116BAAM Worksheet