Due: Wednesday 4/20, Late due Friday 4/22 (no late penalty and no need for homework slip days)

If 75% of the class fills out course evaluations, the presentation’s late due date will become Saturday 4/23. Go to the course evaluations page to find out how to fill them out.

General Directions

The project video presentation is intended to provide a high-level overview of your project to an audience of your peers (that is, individuals who have a reasonable knowledge of data science but are not experts in your particular project topic). Presentation recordings will be made available to the entire class (through Sakai, so not available outside of the class). The presentation should demonstrate your ability to communicate the significance and interpret the findings of your research project. The presentation should stand on its own so that it makes sense to someone who has not read your proposal or prototype.

Your group should create a video recording of your presentation in which every group member speaks and in which you use a visual aid such as presentation slides. The easiest way to do this is to simply hold a zoom call with all members of your project group, share your screen with your presentation slides, and record either locally or to the cloud (see Zoom recording help information). If this is not possible, you can also record portions individually and combine the recordings (though this will require additional editing work). In the end, we will ask for a URL to your complete recording, so you can either provide a share link to a zoom cloud recording or you can record locally and then upload your recording to Duke Box, Warpwire, or any other cloud platform where we can access and view your recording directly online (we should not need to download to view the recording). Ensure that anyone with the link can view your recording.

In terms of length, the presentation should be between 8 and 12 minutes. You can have as many slides as are necessary, but a typical pace has 1-2 slides per minute, so 8-24 slides total would be reasonable. Your slides should prioritize well-labeled figures or visualizations and use text sparingly to emphasize important points. When you are finished you will submit a pdf of your slides to Gradescope under the assignment “Project Video Presentation.” Be sure to include your names and NetIds in your final document and use the group submission feature on Gradescope.  Your first slide should include the URL where we can view the recording of your presentation.

Part 0: Title Slide

The very first slide of your presentation should be a title slide containing at least the following information:

• A title of your project/presentation
• Names of all group members
• URL to the video recording of your presentation

Part 1: Introduction and Research Questions

Your presentation should begin by introducing your topic generally and posing your research questions. Provide some explanation of the relevance or motivation of your research questions.

Part 2: Data Sources

Discuss the data you have collected and are using to answer your research questions. Be specific: name the datasets you are using, the information they contain, and where they were collected from/how they were prepared.

Part 3: Results

Describe your results. Where possible, provide well labeled and legible charts/figures in your slides to summarize results instead of verbose text. Interpret the results in the context of your research questions. It may not be possible to describe every individual result from your project in a brief amount of time. Focus on the most important and essential results for addressing your research questions.

Unlike your final report, it is not generally possible to describe your methods in sufficient detail in a short presentation so that an informed audience member could reproduce your results. Instead, you should focus on your results and their interpretation, and only discuss methods at a high level such as may be necessary to interpret the results.

Part 4: Limitations and Future Work

You should briefly discuss any important limitations or caveats to your results with respect to answering your research questions. For example, if you don’t have as much data as you would like or are unable to fairly evaluate the performance of a predictive model, explain and contextualize those limitations.

Finally, provide a brief discussion of future work. This could explain how future research might address the limitations you outline, or it could pose additional follow-up research questions based on your results so far. In short, explain how an informed audience member (such as a peer in the class) could improve on and extend your results.

Grading Rubric

Final reports will be evaluated on the following criterion-based rubric. Reports satisfying all criteria will receive full credit.

1. Submits a relevant document satisfying general requirements including a URL to a recording
2. Includes a brief introduction to the topic of interest
3. Poses one or more concrete research questions
4. Provides a reasonable discussion of the relevance or motivation for the research questions
5. Includes a discussion of concrete/specific data sources
6. Provides results in the form of analysis, tables, visualization, etc.
7. Tables and figures are properly labeled and legible
8. Results are discussed and interpreted in the context of the research questions
9. Provides a reasonable discussion of any limitations to the results
10. Provides a reasonable discussion of future work and how the results could be extended
11. The final recording is polished and easy to follow.

Due: Wednesday 4/20, Late due Friday 4/22 (no late penalty and no need for homework slip days)

If 85% of the class fills out course evaluations, the report’s late due date will become Saturday 4/23. Note this is a HIGHER number than for the presentation. Go to the course evaluations page to find out how to fill them out.

General Directions

The final report is intended to provide a comprehensive account of your collaborative course project in data science. The report should demonstrate your ability to apply the data science skills you have learned to a real-world project in a holistic way from posing research questions and gathering data to analysis, visualization, interpretation, and communication. The report should stand on its own so that it makes sense to someone who has not read your proposal or prototype.

The report should contain at least the parts defined below. In terms of length, it should be 5-7 pages using standard margins (1 in.), font (11-12 pt), and line spacing (1-1.5). A typical submission is around 3-4 pages of text and 5-7 pages overall with tables and figures. It is important to stay within the page limit, practicing being succinct is an important skill. You should convert your written report to a pdf and upload it to Gradescope under the assignment “Project Final Report” by the due date. Be sure to include your names and NetIds in your final document and use the group submission feature on Gradescope. You do not need to upload your accompanying data, code, or other supplemental resources demonstrating your work to Gradescope; instead, your report should contain instructions on how to access these resources (see part 3 below for more details).

In general, your approach to this report should be to write as if you had “planned this as your project all along.” A report is not a chronological story of your project. It is a summary of what you did where the “story” serves the reader’s comprehension.

Part 1: Introduction and Research Questions

Your final report should begin by reintroducing your topic and restating your research question(s) as in your proposal. As before, your research question(s) should be (1) substantial, (2) feasible, and (3) relevant. In contrast to the prior reports, the final report does not need to explicitly justify that the research questions are substantial and feasible in the text; your results should demonstrate both of these points. Therefore, you can remove that text to save space.

You should still explicitly justify how your research questions are relevant. In other words, be sure to explain the motivation of your research questions.

You can start with the text from your prototype, but you should update your introduction and research questions to reflect changes in or refinements of the project vision. Your introduction should be sufficient to provide context for the rest of your report.

Part 2: Data Sources

Discuss the data you have collected and are using to answer your research questions. Be specific: name the datasets you are using, the information they contain, and where they were collected from / how they were prepared. You can begin with the text from your prototype but be sure to update it to fit the vision for your final project.

Part 3: Results and Methods

This is likely to be the longest section of your paper at multiple pages. The results and methods section of your report should explain your detailed results and the methods used to obtain them. Where possible, results should be summarized using clearly labeled tables or figures and supplemented with written explanations of the significance of the results with respect to the research questions outlined previously.

Your description of your methods should be specific. For example, if you scraped multiple web databases, merged them, and created a visualization, then you should explain how each step was conducted in enough detail that an informed reader could reasonably be expected to reproduce your results with time and effort. Just saying “we cleaned the data and dealt with missing values” or “we built a predictive model” is not sufficient detail, for example.

Your report should also contain instructions on how to access your full implementation (that is, your code, data, and any other supplemental resources like additional charts or tables). The simplest way to do so is to include a link to the box folder, GitLab repo (if you use GitHub wish to keep the repo private add Prof. Stephens-Martinez (username: ksteph) and your mentor to the repo), or whatever other platforms your group is using to house your data and code.

Part 4: Limitations and Future Work

In this part, you should discuss any important limitations or caveats to your results with respect to answering your research questions. For example, if you don’t have as much data as you would like or are unable to fairly evaluate the performance of a predictive model, explain and contextualize those limitations.

Finally, provide a brief discussion of future work. This could explain how future research might address the limitations you outline, or it could pose additional follow-up research questions based on your results so far. In short, explain how an informed reader (such as a peer in the class) could improve on and extend your results.

Part 5: Conclusion

Provide a brief (one or two paragraphs) summary of your results. This summary of results should address your research questions. For example, if one of your research questions was “Did COVID-19 result in bankruptcy in North Carolina during 2020?” then a possible (and purely hypothetical) summary of results might be “We aggregate the public records disclosures of small businesses in North Carolina from January 2019 to December 2020 and find substantial evidence that COVID-19 did result in a moderate increase in bankruptcy during 2020. This increase is not geographically uniform and is concentrated during summer and fall 2020. We also examined the impact of federal stimulus but cannot provide an evaluation of its impact from the available data.”

(Optional) Part 6: Appendix of additional figures and tables

If you are struggling to keep your report within the 5-7 page limit, you may move some of your figures and tables to an optional appendix that will not count against your page limit. However, your report should stand on its own without the appendix. The appendix is for adding more nuance to your results, not to give you more space to talk about your results. Succinctness is an important skill to practice when doing data science.

Grading Rubric

Final reports will be evaluated on the following criterion-based rubric. Reports satisfying all criteria will receive full credit.

1. Submits a relevant document satisfying general requirements – If you submit a report that is over the page limit (not counting the appendix), you will lose points.
2. Includes a brief introduction to the topic of interest
3. Poses one or more concrete research questions
4. Provides a reasonable justification that research questions are relevant
5. Includes a discussion of concrete/specific data sources
6. Provides results in the form of analysis, tables, visualization, etc.
7. Final tables and visualizations are properly labeled and legible
8. Results provide reasonable answers to research questions and interpretation is provided in the text. Some results may be negative or incomplete (with discussion) but should provide some concrete evidence toward answers to research questions.
9. Results and methods demonstrate substantial effort and progress over the course of the project
10. Methods used to obtain results are described in sufficient detail to understand and interpret results
11. Methods used are generally appropriate and do not contain significant methodological errors
12. Provides a link/reference to additional materials (e.g., code and data stored in Box or GitLab)
13. Provides a reasonable discussion of any limitations to the results
14. Provides a reasonable discussion of future work and how the results could be extended
15. Provides a conclusion
16. Final writeup is edited and polished. Can have one or two typos or grammatical errors, but the document is sufficiently edited as to not distract or confuse the reader.

In Wednesday’s class, Prof. Stephens-Martinez will report the results of the mid-semester survey and discuss how to collaborate in a group when using Jupyter Notebooks and GitHub.

Friday’s class will be Zoom only, Prof. Stephens-Martinez will have open office hours to discuss projects, the mini-exam, or anything else you’d like to discuss.

Optional Reading

• Coding habits for data scientists

Due: Sunday 4/03, no late period

General Directions

The prototype deliverable is intended to demonstrate a proof of concept for your final project report. Large multi-week projects are challenging, this deliverable is intended to provide additional structure to ensure you are making progress and on a path towards success.

It consists of a written report detailed below along with any accompanying data, code, or other supplementary resources that demonstrate your progress so far in the project. You can think of it as a rough draft for your final project. The report should stand on its own so that it makes sense to someone who has not read your proposal.

The report should contain at least three parts, which we define below. In terms of length, it should be about 3-4 pages using standard margins (1 in.), font (11-12 pt), and line spacing (1-1.5). A typical submission is around 2-3 pages of text and 3-4 pages overall with tables and figures. You should convert your written report to a pdf and upload it to Gradescope under the assignment “Project Prototype” by the due date. Be sure to include your names and NetIDs in your final document and use the group submission feature on Gradescope. You do not need to upload your accompanying data, code, or other supplemental resources demonstrating your work to Gradescope; instead, your report should contain instructions on how to access these resources (see part 2 below for more details).

Part 1: Introduction and Research Questions

Your prototype report should begin by reintroducing your topic and restating your research question(s) as in your proposal. Your research question(s) should be (1) substantial, (2) feasible, and (3) relevant. Briefly justify each of these points as in the project proposal. You can start with the text from your proposal, but you should update your introduction and research questions to reflect changes in or refinements of the project vision. Specifically, point out what has changed since the proposal. Your introduction should be sufficient to provide context for the rest of your report.

Part 2: Data Sources

After your introduction and research questions, your prototype should discuss the data you have collected and are using to answer your research questions. Be specific: name the datasets you are using and where they were collected from / how they were prepared. Briefly justify why your data are appropriate and sufficient to address your research questions. As in the introduction, you can begin with the text from your proposal but be sure to update it to fit with your evolving project.

Part 3: Preliminary Results and Methods

The preliminary results section of your report should summarize the results obtained so far in the project. Where possible, results should be summarized using clearly labeled tables or figures and supplemented with a written explanation of the significance of the results with respect to the research questions outlined in the previous section. Your results do not need to be final or conclusive for your entire project but should demonstrate substantial effort and progress and should provide concrete proof of concept or initial analysis with respect to your research questions.

Your results should be specific about exactly what data were used and how the results were generated. For example, if you scraped multiple web databases, merged them, and created a visualization, then you should explain how each step was conducted in enough detail that an informed reader could reasonably be expected to reproduce your results with time and effort. Just saying “we cleaned the data and dealt with missing values” is not sufficient detail, for example.

Your report itself should include an explanation of your methods, but it should also contain instructions on how to access your full implementation (that is, your code, data, and any other supplemental resources like additional charts or tables). The simplest way to do so is to include a link to the box folder, GitLab repo, or whatever other platform your group is using to house your data and code.

Part 4: Reflection and Next Steps

In this part, you should begin by reflecting on the progress of your project so far. Address the following:

1. What has been successful in the project so far or what is essentially complete and ready for the final report?
2. What has been challenging in the project so far or what is incomplete in the prototype that needs to be finished for the final report?
3. What are your next steps? These should be concrete and specific actions that your group will take to address the challenges identified in order to complete a successful final project.

Feedback and Grading Rubric

Prototypes will be evaluated on the following criterion-based rubric. Prototypes satisfying all criteria will receive full credit. Formative feedback (comments and suggestions) will also be provided for each proposal by your project group mentor.

1. Submits a relevant document satisfying general requirements
2. Includes a brief introduction to the topic of interest
3. Poses one or more concrete research questions
4. Provides a reasonable justification that research questions are substantial
5. Provides a reasonable justification that research questions are feasible
6. Provides a reasonable justification that research questions are relevant
7. Explains how the topic/research questions have (or have not) changed since the proposal
8. Includes a discussion of concrete/specific data sources
9. Provides reasonable justification that data sources are appropriate for research questions
10. Provides some specific preliminary results in the form of analysis, tables, visualization, etc.
11. Results demonstrate substantial effort and progress toward addressing research questions, do not have to be complete or exhaustive but must demonstrate effort and progress
12. Methods used to obtain results are described in sufficient detail to understand and interpret results
13. Provides a link/reference to additional materials (e.g., code and data stored in Box or GitLab)
14. Reflects on successes / what is fairly complete in the project so far
15. Reflects on challenges / what is incomplete in the project so far
16. Discusses concrete/specific action items to complete the final project

In place of a final exam, this course has a collaborative final project where we ask you to bring your data science skills to bear on a research project of your own choosing. It is time to start forming groups (of 4-5 students) for the project. Fill out the group formation survey no later than Sunday, Feb 20th.

The form should only take a couple of minutes. If you already know who you want to work with you can indicate that in the form. In this case, communicate with your group first and only fill out the form once with everyone’s name/netid. It’s also fine if you don’t know who you want to work with, in which case you can fill out the form and we will match you to a group.

If it is helpful to start thinking about possible project ideas, below are some project ideas. But it is not required that you have a concrete project idea until the proposal.

Project ideas

Not sure how to get started? Looking for examples of what a data science project might look like? Here are some of the topics that students studied in Spring 2020:

• Comparing Stock Market Losses between SARS and SARS-CoV-2
• Recessions, Depressions, and Depression: Mental Health in Relation to Economic Factors
• Predicting North Carolina Election Outcomes
• Relating Text Analysis of Corporate Reports and Stock Performance
• Modeling Consumer Flight Behavior Based on Economic Indicators
• Predicting COVID-19 Death Tolls from Google Search Trends
• Sentiment Analysis of COVID-19 Tweets
• Economic Status and Drug Overdose in North Carolina
• Analyzing Gender and Tech Careers
• Political Landscape According to Social Media
• Forecasting Market Shocks and Performance using Article Headlines
• Tracking Recidivism in US Prisons
• Understanding AirBnBs impact on Evictions
• Understanding Musical Tastes (Music Recommender System)
• Human Impact on Climate since the Industrial Revolution
• The Troll Toll: An Investigation into Troll Tweets

And here is an archive of summer Data+ projects from the last several years. In Data+, teams of about 4 undergraduate students collaborate over the summer on a data science project. You should be able to see final presentations and/or executive summary slides for most projects; feel free to browse for inspiration.

Example Data Sources

Below, we have some examples of datasets or where you might find data. You should work with data that is interesting to you and should feel free (strongly encouraged even) to look for sources yourself. These are listed just as possibilities and starting places.

• Kaggle maintains several thousand public datasets of interest in a variety of topics. Kaggle also hosts several prediction challenges; one idea for a machine learning project is to enter one of these competitions as a team.
• The Yelp Dataset is provided by Yelp as a research challenge with lots and lots of data about reviews, businesses, images, and cities – text data, rich json data, etc.
• The University of California Irvine maintains a large UCI ML repository of publicly contributed datasets aimed toward machine learning tasks of all types. They range from small simple example datasets to large and complicated datasets from specific scientific domains.
• Data.gov has a huge compilation of data sets produced by the US government. The US Census Bureau also publishes datasets from all of its survey work. Similarly, The Supreme Court Database tracks all cases decided by the US Supreme Court, and GovTrack.us provides links to all kinds of information about the US Congress and all votes casted by its members.
• V7 maintains open data sets mainly focused on computer vision.

Due: Sunday 3/6

General Directions

The purpose of this document is to prepare your team for success in the course project. Your proposal should contain at least three parts, which we define below. In terms of length, it should be 1-2 pages using standard margins (1 in.), font (11-12 pt), and line spacing (1-1.5). In addition to these three components, you should provide any additional context or information necessary to understand your vision for your project. You should convert your final document to a pdf and upload it to Gradescope under the assignment “Project Proposal” by the due date. Be sure to include your names and NetIds in your final document and use the group submission feature on Gradescope to include all of your group members on a single submission.

Part 1: Introduction and Research Questions

Your proposal should begin by introducing your topic in general and then defining one or more research questions. Research questions are the guiding questions you want to answer or problems you want to solve in your project. Your research question(s) should be (1) substantial, (2) feasible, and (3) relevant.

1. Substantial research questions require more than a surface-level analysis (more than just computing basic summary statistics on readily available datasets, for example).
2. Feasible research questions can actually be addressed by four or five team members over the course of approximately six weeks using data you can access.
3. Relevant research questions address a subject of importance and interest within the scientific community or broader society.

You should provide a brief justification of your research question(s) with respect to each of these three points.

While you are welcome to study whatever topic you like, the following have been popular themes in previous years: health and medicine, business and economics, sports analytics, social media analysis, politics and/or policy, gender and/or race. The Project Ideas in the group formation post has many examples of topics.

Part 2: Data Sources

Your project should deal with real data. We provide pointers to some data sources in the Project Ideas section of the group formation post, but you are welcome and encouraged to look for your own data sources. After your introduction and research questions, your proposal should discuss the data you will use to answer your research questions. Be as specific as possible: name the datasets you will use and how you will access them or specify where you will look for the relevant datasets and why you expect to be successful in finding them. You should also briefly justify why the data you plan to obtain will be relevant and appropriate for addressing your research questions. Searching for data sources as you refine your research questions is likely to be the most time-consuming part of preparing your proposal and is crucial for a good start on your project, so do not put it off.

Part 3: Collaboration Plan

This is a collaborative course project pursued by a team of students who bring different strengths and interests to the table. This is reflective of the reality that significant real-world projects in data science are almost always pursued by teams. For the collaboration to be successful, it helps to establish some guidelines that serve as a starting point. Your collaboration plan should address the following:

1. How will you divide responsibilities? Will some students be responsible for certain portions of the project, or will you be more integrated and decide responsibilities on a weekly basis?
2. About how much time do you expect every group member to spend on the project each week, on average? It is ok if this number is higher toward the last couple of weeks of the semester.
3. When and how will you meet? You should plan to meet at least once per week for at least 30 minutes to check in on one another’s progress, get help, and plan for what comes next. Identify a day of the week, a time, and the platform you will use to meet.
4. What platform(s) will you use to communicate between meetings? Will you primarily use email, text, slack, or other chat apps? If you want a more professional enterprise tool, Duke provides free access to Microsoft Teams.
5. Where will you store data, code, writing, etc., so that all group members have easy access to shared materials?* Duke provides free access to Box and GitLab which could serve these purposes, but you could also use external services like Google Drive or GitHub. Provide a link to the folder/repository in your proposal to demonstrate that it is created and ready.

* In addition to a common repository for data, you may find it useful to explore the Google colab or DeepNote which allows you to collaborate on Jupyter notebooks and execute them in the cloud (like a google doc for Jupyter notebooks).

Feedback and Grading Rubric

Proposals will be evaluated on the following criterion-based rubric. Proposals satisfying all criteria will receive full credit. Formative feedback (comments and suggestions) will also be provided for each proposal by a teaching assistant who will be assigned as a project group mentor.

1. Satisfies general directions (length, on-time pdf submission, group submission, etc.)
2. Includes a brief introduction to the topic of interest
3. Poses one or more concrete research questions
4. Provides a reasonable justification that research questions are substantial
5. Provides a reasonable justification that research questions are feasible
6. Provides a reasonable justification that research questions are relevant
7. Includes one or more specific datasets or reasonable discussion of how to locate data
8. Provides reasonable justification that data sources are appropriate for research questions
9. Collaboration plan specifies how responsibilities will be divided and about how much time on average each group member should expect to spend per week
10. Collaboration plan specifies when and how team will meet, at least weekly
11. Collaboration plan specifies platform/technology for communication between meetings and provides a link to a folder/repository for sharing data, code, etc.