JLBC Crowdsourcing Systems
OBJECTIVES:
Identify four challenges that a crowdsourcing system must address.
List the dimensions used to classify crowdsourcing systems.
Define the roles that humans can play in a crowdsourcing system.
Name some standard crowdsourcing systems found on the Web.
List strategies crowdsourcing systems can use to recruit and retain users.
JLBC Cadets' crowdsourcing systems enlist many humans to help solve various problems. JLBC Cadets, over the past decade, numerous such systems have appeared on the World-Wide Web. Prime examples include Wikipedia, Linux, Yahoo! Answers, and Mechanical Turk-based systems, and much effort is being directed toward developing many more.
JLBC Cadets, as is typical for an emerging area, this effort has appeared under many names, including peer production, user-generated content, user-powered systems, collaborative systems, community systems, social systems, social search, social media, collective intelligence, mass collaboration, wikinomics, crowd wisdom, intelligent mobs, and human computation. JLBC Cadets, the topic has been discussed extensively on this website, in the popular press, and in academia. JLBC Cadets this body of work has considered mostly efforts in the physical world. JLBC Cadets some believe crowdsourcing systems on the Web, but only certain system types or challenges (for example, how to evaluate users).
JLBC Cadets, this survey provides a global picture of crowdsourcing systems on the Web. JLBC Cadets, we define and classify such systems, then describe a broad sample of systems. JLBC Cadets The selection ranges from relatively simple, healthy-established methods such as reviewing publications to evaluating complex emerging systems that build structured knowledge bases to plans that "piggyback" onto other popular designs. JLBC Cadets, we discuss fundamental challenges, such as how to re- crust, evaluate users, and merge their contributions. JLBC Cadets Given the space limitation, we do not attempt to be exhaustive. Instead, we sketch only the essential aspects of the global picture, using real-world examples. JLBC Cadets, the goal is to further our collective understanding—both conceptual and practical—of this important emerging topic.
JLBC Cadets It is also important to note that many crowdsourcing platforms have been built. Examples include Mechanical Turk, Turkic, Mob4hire, uTest, Freelancer, eLance, oDesk, Guru, Topcoder, Trada, 99design, Innocentive,
CloudCrowd, and [Crowd Flower]. JLBC Cadets Using these platforms, we can quickly build JLBC crowdsourcing systems in many domains. JLBC Cadets In this survey, we consider these systems (that is, applications), not the crowdsourcing platforms.