skip to main content
10.1145/1460563.1460572acmconferencesArticle/Chapter ViewAbstractPublication PagescscwConference Proceedingsconference-collections
research-article

Harnessing the wisdom of crowds in wikipedia: quality through coordination

Published:08 November 2008Publication History

ABSTRACT

Wikipedia's success is often attributed to the large numbers of contributors who improve the accuracy, completeness and clarity of articles while reducing bias. However, because of the coordination needed to write an article collaboratively, adding contributors is costly. We examined how the number of editors in Wikipedia and the coordination methods they use affect article quality. We distinguish between explicit coordination, in which editors plan the article through communication, and implicit coordination, in which a subset of editors structure the work by doing the majority of it. Adding more editors to an article improved article quality only when they used appropriate coordination techniques and was harmful when they did not. Implicit coordination through concentrating the work was more helpful when many editors contributed, but explicit coordination through communication was not. Both types of coordination improved quality more when an article was in a formative stage. These results demonstrate the critical importance of coordination in effectively harnessing the "wisdom of the crowd" in online production environments.

References

  1. Aiken, L. and West, S. Multiple regression: Testing and interpreting interactions. Sage, CA (1991).Google ScholarGoogle Scholar
  2. Atkinson, A. B. On the measurement of inequality. Journal of Economic Theory, 2 (1970), 244--263.Google ScholarGoogle ScholarCross RefCross Ref
  3. Berg, J. E., Forsythe, R., & Rietz, T. A. The Iowa electronic market. In Blackwell Encyclopedic Dictionary of Finance, Blackwell, Oxford UK (1997).Google ScholarGoogle Scholar
  4. Benkler, Y. Coase's penguin, or Linux and the nature of the firm. Yale Law Journal, 112 (2002), 367--445.Google ScholarGoogle ScholarCross RefCross Ref
  5. Brooks, F. P. The mythical man month. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA (1975). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Associates, Mahwah, New Jersey (2003).Google ScholarGoogle Scholar
  7. Galton, F. Vox populi. Nature, 75 (1907), 7.Google ScholarGoogle ScholarCross RefCross Ref
  8. Giles, G. Internet encyclopaedias go head to head. Nature, 438 (2005), 900--901.Google ScholarGoogle ScholarCross RefCross Ref
  9. Hadoop Project. http://lucene.apache.org/hadoop/.Google ScholarGoogle Scholar
  10. Heckman, J. Sample selection bias as a specification error. Econometrica, 47, 1 (1979), 153--162.Google ScholarGoogle ScholarCross RefCross Ref
  11. Hill, G. W. Group versus individual performance. Are n + 1 heads better than one? Psychological Bulletin, 91 (1982), 517--539.Google ScholarGoogle ScholarCross RefCross Ref
  12. Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. The emerging conceptualization of groups as information processors. Psychological Bulletin, 121, 1 (1997), 43--64.Google ScholarGoogle ScholarCross RefCross Ref
  13. http://en.wikipedia.org/wiki/Category:Wikipedia_1.0_assessmentsGoogle ScholarGoogle Scholar
  14. http://en.wikipedia.org/wiki/Wikipedia:StubGoogle ScholarGoogle Scholar
  15. http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/AssessmentGoogle ScholarGoogle Scholar
  16. http://en.wikipedia.org/wiki/Wikipedia:WIAFAGoogle ScholarGoogle Scholar
  17. Kanefsky, B., Barlow, N.G., & Gulick, V.C. Can Distributed Volunteers Accomplish Massive Data Analysis Tasks? Lunar and Planetary Science (2001).Google ScholarGoogle Scholar
  18. Katz, R., & Tushman, M. Communication patterns, project performance, and task characteristics: An empirical evaluation and integration in an R&D setting. Organizational Behavior and Human Decision Processes, 23, (1979) 139--162.Google ScholarGoogle ScholarCross RefCross Ref
  19. Kittur, A., Suh, B., Chi, E., & Pendleton, B. A. He says, she says: Conflict and coordination in Wikipedia. CHI 2007, ACM Press (2007), 453--462. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kittur, A., Chi, E., & Suh, B. Crowdsourcing user studies with Mechanical Turk. CHI 2008, ACM Press (2008). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kittur, A., Chi, E., Pendleton, B. A., Suh, B., & Mytkowicz, T. Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. Alt.CHI, 2007, San Jose, CA (2007).Google ScholarGoogle Scholar
  22. Laughlin, P. R., VanderStoep, S. W., & Hollingshead, A. B. Collective versus individual induction: Recognition of truth, rejection of error, and collective information processing. Journal of Personality and Social Psychology, 61, 1 (1991), 50--67.Google ScholarGoogle ScholarCross RefCross Ref
  23. Leavitt, H. Some effects of certain communication patterns on group performance. Journal of Abnormal and Social Psychology, 46 (1951), 38--50.Google ScholarGoogle ScholarCross RefCross Ref
  24. Malone, T. W. Modeling coordination in organizations and markets. Management Science, 33, 10 (1987), 1317--1332. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. March, J. G., & Simon, H. A. Organizations. New York: Wiley (1958).Google ScholarGoogle Scholar
  26. Mockus, A., Fielding, R. T., & Herbsleb, J. D. Two case studies of open source software development: Apache and mozilla. ACM Transactions on Software Engineering and Methodology, 11, 3 (2002), 309--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Priedhorsky, R., Chen, J., Lam, S., Panciera, K., Terveen, L., & Riedl, J. Creating, destroying, and restoring value in Wikipedia. Proc GROUP 2007, ACM Press (2007), 259--268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rouse, W.B., Cannon-Bowers, J. A., & Salas, E. The role of mental models in team performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics, 22 (1992) 1296--1308.Google ScholarGoogle ScholarCross RefCross Ref
  29. Shepperd, J. A. Productivity loss in performance groups: A motivation analysis. Psychological Bulletin, 113, 1 (1993), 67--81.Google ScholarGoogle ScholarCross RefCross Ref
  30. Steiner, I. D. Group process and productivity. New York: Academic Press (1972).Google ScholarGoogle Scholar
  31. Stvilia, B., Twidale, M. B., Smith, L. C., & Gasser, L. Assessing information quality of a community-based encyclopedia. In Proc. ICIQ (2005), 442--454.Google ScholarGoogle Scholar
  32. Surowiecki, J. The wisdom of crowds. New York: Doubleday (2005). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Thompson, J. Organizations in action. New York: McGraw-Hill (1967).Google ScholarGoogle Scholar
  34. Tushman, M. Work characteristics and subunit communication structure: A contingency analysis. Administrative Science Quarterly, 24, (1979), 82--98.Google ScholarGoogle ScholarCross RefCross Ref
  35. Van de Ven, A. H., Delbecq, A. L., & Koenig, R., Jr. Determinants of coordination modes within organizations. American Sociological Review, 41 (1967), 322--338.Google ScholarGoogle Scholar
  36. Viégas, F. B., Wattenberg M., & Dave, K. Studying cooperation and conflict between authors with history flow visualizations. CHI 2004, ACM Press (2004), 575--582. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Viégas, F. B., Wattenberg, M., Kriss, J., & van Ham, F. Talk before you type: Coordination in Wikipedia. HICSS (2007).Google ScholarGoogle Scholar
  38. Viégas, F. B., Wattenberg, M., & McKeon, M. M. The hidden order of Wikipedia. In HCII (2007).Google ScholarGoogle Scholar
  39. Wilkinson, D. M., & Huberman, B. A. Assessing the value of cooperation in Wikipedia. First Monday, 12, 4 (2007).Google ScholarGoogle ScholarCross RefCross Ref
  40. Wittenbaum, G. W., Vaughan, S. I., & Stasser, G. Coordination in task-performing groups. In Theory and research on small groups, Plenum Press, NY (1998).Google ScholarGoogle Scholar

Index Terms

  1. Harnessing the wisdom of crowds in wikipedia: quality through coordination

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Conferences
          CSCW '08: Proceedings of the 2008 ACM conference on Computer supported cooperative work
          November 2008
          752 pages
          ISBN:9781605580074
          DOI:10.1145/1460563

          Copyright © 2008 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 8 November 2008

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          Overall Acceptance Rate2,235of8,521submissions,26%

          Upcoming Conference

          CSCW '24

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader