Table of Contents

Final Project: Human Computation + X

Project Overview

In this graduate seminar, we are mainly focused on human-in-the-loop data management, i.e., human computation + data management. But this is only a small part of human-computation research. There are a lot of other work on Human Computation + X (= Computer Vision, Artificial Intelligence, Natural Language Processing, Mobile Computing, Information Retrieval, Graphics, etc.). In this project, you are free to choose any X you like, even including X = Data Management.

Advice on the choice of X. Since you only have ~2 months for the project, it would be challenging to choose a topic that you are not very familiar with. To be realistic, please discuss with other team members and make sure to choose a topic that the team has the needed expertise.

Due to your diverse backgrounds and skills, I designed three types of projects: (1) Survey, (2) Research, and (3) Programming. Please choose ONE from them, and follow the following steps to do the project:

  1. Form a team of 1-3 persons
  2. Pick up a project type, decide what X is, and submit an initial plan
  3. Present your project in the poster session
  4. Submit your paper

The table below summarized the deadline for each phase:

ID
What
When
Where
1 Form a Team Sunday 10/23 at 11:59pm Create your team in CourSys
2 Initial Plan Friday 11/04 at 11:59pm Submit the filled form to the CourSys activity Initial Plan
3 Poster Session Wednesday 12/07 at 08:00am
Wednesday 12/07 at 03:00pm		 Submit your poster to the CourSys activity Poster
Present your poster at T9204 W&E
4 Paper Monday 12/12 at 11:59pm Submit the paper to the CourSys activity Paper

Types of Projects

Please choose one project type, and follow the corresponding instruction to do the project.

1. Survey

If you choose to do a survey on "Human Computation + X", here are a list of steps you need to do:

  1. Collecting Related Papers. Please follow the instruction in this paper (Sec 3) to find related papers on the topic.
  2. Reading Papers. You should read at least 20-40 papers in order to write a good survey paper. After reading the papers, please make sure that you are completely aware of the main research themes, trends, challenges/issues, and results of the topic.
  3. Designing a Classification Criteria. Determine the classification of the existing literature, and draw a diagram for that. You can get an idea about what the diagram looks like from these examples (Fig 1, Fig 1, Fig 3, Fig 1, Fig 1).
  4. Identifying Pros and Cons. Map the collected papers to their corresponding classes. Compare different classes, and identify the pros/cons of each class.
  5. Predicting Future Trends. Think about what problems on this topic should be paid more attention in the future, and why they are challenging?
  6. Writing the Survey Paper. Here is a suggested outline of the survey paper. You don't have to use it, but the paper has to include the materials mentioned in Steps 3-5.
    • Abstract
    • Introduction
    • Background and Problem Statement
    • Classification Criteria
    • Presentation of Class 1, Class 2, ...
    • Comparisons of Classes (pron/cons)
    • Emerging Trends
    • Conclusion

Submission

  • Initial Plan: Download the Initial Plan form template, and submit the filled form to the CourSys activity Initial Plan.
  • Poster Session: Make a poster to present your paper. The poster session is scheduled at 03:00 pm on Wednesday 12/07. Please upload your poster to the CourSys activity Poster before 08:00 am.
  • Paper: Write the paper using the templates provided at https://www.acm.org/publications/proceedings-template for Word and LaTeX (version 2e). That is, the format of your paper should be like this. The paper has to be at least 8 pages (excluding references).

2. Research

If you choose to do a research project on "Human Computation + X", here are a list of steps you need to do:

  1. Problem. Ask yourself what problems in the field of X cannot be well solved by machine-only techniques. Pick up one such problem, and think about how to leverage human computation to solve it.
  2. Related Work. Get familiar with the state-of-the-art solutions to the problem as well as their limitations.
  3. Key Insight. Figure out the key insight in your solution. The insight has to be original! That is, nobody has applied the same insight to your problem.
  4. Solution. Describe in detail how your solution works and why it works.
  5. Experiments. Do some experiments to evaluate the performance of your solution.
  6. Paper Writing. Here is a suggested outline of the paper. You don't have to use it, but the paper has to include the materials mentioned in Steps 1-5.
    • Abstract
    • Introduction (Background, Existing Solutions, Your Contributions)
    • Problem Statement
    • Your Solution
    • Experiments
    • Conclusion

Submission

  • Initial Plan: Download the Initial Plan form template, and submit the filled form to the CourSys activity Initial Plan.
  • Poster Session: Make a poster to present your paper. The poster session is scheduled at 03:00 pm on Wednesday 12/07. Please upload your poster to the CourSys activity Poster before 08:00 am.
  • Paper: Write the paper using the templates provided at https://www.acm.org/publications/proceedings-template for Word and LaTeX (version 2e). That is, the format of your paper should be like this. The paper has to be at least 4 pages (excluding references).

3. Programming

If you choose to do a programming project on "Human Computation + X", here are a list of steps you need to do.

  1. Project Idea. Come up with an idea for a crowdsourcing system. Note that the idea does not need to be new. For example, you can take the 10000 cents idea, and build a system that allows the user to upload an image and get its crowdsourced drawing.
  2. System Architecture. Think about what components the system needs to have and how to put them together.
  3. Technical Challenges. Identify the technical challenges to implement the system, and discuss how to address them.
  4. Demonstration. Determine how you are going to demonstrate the system to impress others.
  5. Paper Writing. Here is a suggested outline of the paper. You don't have to use it, but the paper has to include the materials mentioned in Steps 2-4.
    • Abstract
    • Introduction
    • Architecture
    • Demonstration
    • Conclusion

Submission

  • Initial Plan: Download the Initial Plan form template, and submit the filled form to the CourSys activity Initial Plan.
  • Poster Session*: You are not only required to make a poster, but also demonstrate the system during the poster session. The poster session is scheduled at 03:00 pm on Wednesday 12/07. Please upload your poster to the CourSys activity Poster before 08:00 am.
  • Paper: Write the paper using the templates provided at https://www.acm.org/publications/proceedings-template for Word and LaTeX (version 2e). That is, the format of your paper should be like this. The paper has to be at least 2 pages (excluding references).

Final Note

As per SFU Policy R20.01, if a project meets one of the following requirements, it is required to submit an ethics application.

"You will need to submit an ethics application if you are conducting a Research Study which involves the collection of data from human participants, including human tissues, fluids, cells or archaeological remains, or if you are accesing secondary human participant data/human biological materials. Secondary use refers to the use in research of information or human biological materials originally collected for a purpose other than the current research purpose. This data/biological material is not available to the general public or considered to be in the public domain and was not collected anonymously. Anonymous information and human biological materials are distinct from those that have been coded, and also from those that have been anonymized. Please review SFU Policy R20.01 for more information. If you are still unsure whether your study require research ethics review, please contact the Associate Director or Director, ORE."

This application process often takes several weeks or even a month, which may delay your final project. My suggestion is that if your project does require to collect data from the crowd, you can write a crowd simulator, which sends HTTP requests to our own crowdsourcing platform, to simulate the behavior of crowd workers.