Lit 80, Fall 2013

Evaluating Digital Humanities Projects: Collaborative Course Assessment

For your Partnered #DH project Critique essay, you were asked to use Shannon Mattern’s criteria for evaluating Multimodal Student Work and the peer review criteria from Galey & Ruecker’s “How a Prototype Argues”, (2010) to evaluate selected digital humanities projects. This is an important exercise, especially to those new to #dh, as it helps you think about what does and doesn’t work, about the usefulness of various genres of projects, and about how current projects might be altered to become more useful, more user-friendly, and/or more academically rigorous. As we critique existing projects, we come to a better grasp of how we might develop and manifest our own projects. An important next gesture, then, is for us to establish assessment (read: grading) criteria for our own final transmedia projects. We’ll start where we’ve already started: we’ll look at Mattern’s comprehensive criteria and chose those that best suit our needs. Then we’ll add more, delete the unnecessaries, and edit those we want to tweak.

Here I’ve posted my annotated copy of her list. Your assignment, to be completed within the next two weeks, is to add a comment or two here expressing your ideas and opinions about these criteria. You can add criteria or vote for a deletion. You can suggest edits or request justification for why I’ve highlighted certain sections. The criteria I feel most important to our project are highlighted and annotated below. I welcome all comments. Go!

Augmenting Realities Final Project Criteria_Page_1

Gould Annotations – Click and Zoom to read

Augmenting Realities Final Project Criteria_Page_2

Gould annotations page 2

Augmenting Realities Final Project Criteria_Page_3

Gould annotations page 3

To these, let’s add:

A consideration of Digital Preservation: Let’s think about how we might best preserve our content as a whole. And let’s consider best practices for saving our own personal work. Remember, it is always best to write your webcontent using a saveable (and backup-able) document source, like Word or Google Docs, that you can save, store, and archive. Should our site go down or become compromised, you’ll want to have a backup copy of your hard work. Make sure you download a copy of your media element if possible and/or store an extra copy in the cloud. Let’s think too about zombie links and dead sites.



  1. Shane Stone

    Maybe I’m the only one confused about this, but for this project do you want us to write an actual essay and just add a form of media to it? Or are we supposed to make an actual DH project? I think once I grasp that I can properly comment on the criteria

  2. Shane, for your final project, you will be creating what I am calling a ‘transmedia’ essay. As outlined in the Assignments section of our website, you will write an 8-12 page essay that includes a digital humanities media element. Just adding a form of media is not enough – I want you to CREATE something or ANALYZE something or MAP something or REMEDIATE something. Adding a video or some pictures to your essay will not be sufficient; creating your own video is getting closer. Creating and annotating or analyzing your video in some way is more what I have in mind. There are examples of the sorts of projects I am expecting on the Assignments page. Your media element should be a mini DH project. Does this help?

  3. Craig Bearison

    One of the criterion our media element will be evaluated on is if it is something that could have easily been done on paper. At the same time, our project is supposed to have multiple interfaces such that those who prefer interactive visualization can do that and those who prefer print can read as they are used to. This seems a little confusing to me. To what degree should our media element stand alone to make its own contribution vs. simply presenting data in a different format?

  4. Craig, good questions. 1. Your media element should be chosen based on its ability to do something that paper cannot do. You’ll be expected then to take full advantage of the affordances of the medium you choose. As such, it will be an element that could not easily be done on paper. Your class project as a whole – the collaborative web journal – will have outward going links, embedded media, interlinking pages, tags, interactive commenting, etc etc and these are not reproducible so easily on paper. 2. As for familiarity vs newness, we can discuss this one more fully in class and I can show you some examples. We do not want our project to look or act exactly like a print artifact, but we do want to use understandable conventions so that a print preferrer – who will not spend a great deal of time trying to figure out a website or #dh project that looks completely foreign – is able to relatively quickly understand how to read the pages and artifacts you create. As for multiple interfaces, I am not expecting your projects to be this sophisticated. That said, a multiple-interface navigation can be achieved simply through tools like Tumblr, Blogger, or any number of text-analysis tools. Once the data is in, there are tools for displaying that data multiply. 3. As a scientist, you might think of your media element as an experiment that extends and/or explores the questions you are asking in your work. You can choose to what degree your media element speaks. Indeed, I’d argue that presenting data in a different format, if done thoughtfully and accompanied by a critical essay exploring what those different format presentations mean (or say, or do, or are commenting on, etc), CAN be an argument or relevant contribution to a written work. I can show an example of this as well if you’d like.

  5. Sheel Patel

    I had a specific question regarding my media element. As the project manager/leader for the website and our class projects, do I also have to create a re-mediated element? Also along with what goes on the syllabus regarding my responsibilities, is there anything else that you would like me to do regarding the website’s formatting, layout, etc?

  6. Kim Arena

    I understand the requirements, the layout of the project, and what our overall goal is but I am a little confused over what our topic should be…from what I understand as of now should it be something regarding how technology affects our lives? Or does it not necessarily have to be technology related concept-wise but utilize technology concepts to portray a message (if that makes any sense). I’m having trouble coming up with specific areas that could be the focus of the project.

  7. Mithun Shetty

    My main question is quite similar to Kim’s; I have a rough idea of what is expected from the final project content-wise, but finding a topic is very difficult. Because I don’t know what areas to look at for a topic, I find it hard to work with or apply these criteria to the project. I know our discussions in class leave us with endless options, but it would make things easier to have a slightly more rigid set of guidelines for what our topic should center around.
    Also, the criteria makes many points on whether or not the media element is useful/worth putting in the project. Other than seeing if it could be just as easily executed on paper, I do not know how this can be assessed, or how to determine whether something is merely a “cool data set” or the most appropriate medium for a topic. What are some other points to consider?
    Lastly, in regards to “self-reflexivity,” do we need to include our analysis of the limitations of the media element in the dh project itself, or is that exclusively in the essay section?

  8. Kim and Mithun,

    Topic is a difficult question with an easy answer. The easy answer: your final project can address any of the topics we cover in class. The difficult part is choosing one.

    1. Our course blurb suggests a full list “This course will interrogate how media technologies and our various layers of ‘reality’ converge to alter (or augment) our conceptions of the body and the brain, of time and space, of art and literature, of data and information, of memory and storage, of cities and networks, of medicine and prostheses, of the digital and (digital) culture. In considering issues of ethics and emergence, we will forecast future civilizations and explore possible ways of archiving our past and present digital expression.”
    2. The topic descriptions for each week can be another source of inspiration.
    3. Any of the texts, contexts, and questions we discuss in class are also fair game.
    4. If you’ve noticed, I try to call out interesting final paper topics in class from time to time to get you thinking about how what we are discussing might be flipped or expanded into a final transmedia essay. Class discussion is a great resource for idea generation.

    The tricky part is also selecting a topic. I suggest you think about what has so far excited you and what might excite you looking ahead at our future readings and developing that into a project. Brainstorm different ideas, see if any of your ideas overlap, and choose the one that most draws you. You’ll be spending a good deal of time with your chosen idea so it should certainly be something you are interested in pursuing.

    See our Assignment and Syllabus sections for more direction and for assignment specifics and expectations.

  9. shane

    Thank you. I think then I have a better understanding (and will for sure after the class discussion). But the criteria seem fine

  10. Xin Zhang

    I think links for sharing the project on Facebook and Twitter should also be counted as one of the criteria as it is important to let more people involve in the project and make the project serve the public. Actually, it surprises me that so many projects about multimedia have been done by scholars in America or Europe. In China, scholars who rarely do research about literature like this and most of them still just read classical literature and study theories. That is necessary. But it is time to use modern tools and creative ideas to develop the new kinds of literature and serve the society better. The project like Texture and the Map of Early Modern London really open my eyes.

  11. Great idea, Xin! One of the emerging tenets of digital humanities research and publication is indeed making that digital scholarship public through our digital channels. The web makes it very easy to share content and to create a network wherein conversation, dialogue, critique, and knowledge can be generated and shared. Thank you for pointing this out for us!

  12. Matt Hebert

    I appreciate the call for balance rather than getting carried away with innovation, but if we incorporate a multi-media component that is pretty intuitive on its own, do we have to shoe-horn a more traditional alternative into the project for people who aren’t comfortable with the new media? Like, if I were to make a video of myself playing the game while presenting my analysis, I could just include a transcript of what I was saying. But if I were to make a prototype of a game, I’m not sure how I would be able to accommodate people who weren’t willing to play the game.

  13. Great thoughts here, Matt. We’ll discuss these ideas this week and can brainstorm possible solutions. It is perhaps an uncomfortable and unfortunate truth that you’ll alienate readers if you present only a game. The trick and artistry of this type of #dh work is to find a middle way – one that will appeal to a broad audience. We want to teach non-gamers about games, right? And so we must speak within their particular vernacular while also challenging them to reach beyond the familiar. And, for present purposes, we must remember this: this course is a writing course so we are ‘shoe-horned’ by our condition. Which I do not think is necessarily a bad thing. Creativity comes from constraint, right?!

  14. Sai Cheemalapati

    I think I understand the requirements and expectations. I’m interested in doing a project on big data, and maybe demonstrating simple data mining by using census data. I could do some visualizations too…

  15. This sounds like a fantastic idea, Sai!

  16. David Hemminger

    These are very helpful, thanks for posting this! How exactly should we document the creation process? Should we just include in our final paper a summary of how our project became what it was in the end?

  17. Zhan Wu

    I have read through the criteria and I think that I can understand them perfectly. I have a question though, we might have talked about his in class but I just wanted to make sure. Are we going to have peer reviews on our final project? I think these reviews are a great tool for improving the overall quality of the paper since others are able to point out the inconsistencies in the paper that are not easily noticed by the author.

  18. Great question, Zhan. If you’ll check our syllabus, you’ll see that we have several peer review sessions!

Leave a Reply

Powered by WordPress & Theme by Anders Norén