Methods
Card Sorting
Facts:
Types: Closed card sort, Open card sort, Reverse card sort
Lifecycle stage: Requirements, Design
See also: Affinity diagramming, Information architecture
Contributors: Bill Killam, Alice Preston, Shannon McHarg, Chauncey Wilson. Based on Jacek Wachowicz contributions in COST Action 294: MAUSE.
Version: 6/2009
The card sorting method is used to generate information about the associations and grouping of specific data items. Participants in a card sort are asked to organize individual, unsorted items into groups and may, depending on the technique, also provide labels for these groups. In a user-centered design process, it is commonly used when developing a site architecture but has also been applied to developing workflows, menus, toolbars, and other elements of system design.
- Card sorting may be conducted as a low tech method using index cards or post-it notes, or may be automated using one of several software packages
- Card sorting may be conducted as a series of individual exercises, as a concurrent activity of a small group, or as a hybrid approach where individual activity is followed by group discussion of individual differences
- Card sorting is usually conducted as a specific activity in the early design phase of a project for defining an architecture, but can similarly be used during a product evaluation to determine if usability issues are due to problems with grouping or group labels
Sorting and grouping have long been studied within psychology and the research dates back at least to the 1950s. Numerous, non-peer reviewed descriptions, case studies, and blogs have been written in the last several years on the technique and its use in the user-centered design process, but only a few peer reviewed articles on the technique have been published and little is known of its validity or reliability as a means of directly producing a useful and usable architecture. Instead, card sorts are generally used to provide insight that is used by a practitioner to generate an architecture.
- Basic Description
- How To
- Special Considerations
Benefits, Advantages and Disadvantages
Advantages
- Simple – Card sorts are easy for the organizer and the participants.
- Cheap – Typically the cost is a stack of index cards, sticky notes, a pen or printing labels, and some time.
- Quick to execute – it is possible to perform several sorts in a short period of time, which provides significant amount of data.
- Established – The technique has been used for over 10 years, by many designers.
- Involves users – Because the information structure suggested by a card sort is based on real user input
- Provides a good foundation for the structure of a site or product.
Disadvantages
- Does not consider users’ tasks – Card sorting is an inherently content-centric technique. If used without considering users’ tasks, it may lead to an information structure that is not usable when users are attempting real tasks.
- Results may vary –The card sort may provide fairly consistent results between participants, or may vary widely.
- Analysis can be time consuming – The sorting is quick, but the analysis of the data can be difficult and time consuming, particularly if there is little consistency between participants.
- May capture “surface” characteristics only – Participants may not consider what the content is about or how they would use it to complete a task and may just sort it by surface characteristics
Appropriate Uses
Card sorting can be used to:
- Identify themes or patterns from qualitative data
- Develop the information and navigational architecture for a Web site or application
- Design or redesign a site or application
- Organize icons, images, menu items, and other objects into related groups
- Determine how a specific individual classifies items from a particular domain
- Examine how different groups (users versus developers, for example) view the same subject matter
- Rank or rate items on specific dimensions.
Read More About It
Bousfield, W.A. & Barclay, W.D. (1950). The Relationship Between Order and Frequency of Occurrence of Restricted Associative Responses. Journal of Experimental Psychology.
Boy, G.A. (1997). The group elicitation method for participatory design and usability testing. interactions 4(2), 27-33.
Chaparro, B. S., Hinkle, V. D., & Riley, S, K. (2008, November).The Usability of Computerized Card Sorting: A Comparison of Three Applications by Researchers and End Users.
This study reports on the usability of three commercially available electronic card sort applications (CardZort, WebSort, and OpenSort) by researchers and by end users.
Dong, J., Martin, S., & Waldo, P. (2001). A user input and analysis tool for information architecture. In Proceedings of ACM CHI 2001 Conference on Human Factors in Computing Systems, v.2 p.23-24
Gaffney, G. (2000). What is Card Sorting?
IAwiki page on card sorting: Card Sorting.
Lamantia, J. Analyzing Card Sort Results with a Spreadsheet Template. Boxes and Arrows.
Kaufman, J. (2006). Card sorting: An inexpensive and practical usability technique. Intercom, 17-19, Nov-2006.
Maurer, D. & Warfel. T. (2008). Card sorting: a definitive guide. Boxes and Arrows, Apr-2008.
Muller, M. J. (1991). PICTIVE—an exploration in participatory design. In S. P. Robertson, G. M. Olson, and J. S. Olson, (Eds), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching Through Technology.(pp. 225-231). CHI '91. ACM, New York, NY.
Muller, M.J. (2001). Layered Participatory Analysis: New Developments in the CARD Technique. Proceedings of ACM Conference on Human Factors in Computing Systems (pp. 90- 97) Seattle, Washington, USA. ACM.
Nielsen, J. (2004). Card Sorting - How Many Users to Test?
Paul, C.L. (2008). A modified Delphi Approach to a New Card Sorting Methodology. Journal of Usability Studies. 4(1), 7-30, Nov-2008.
Syntagm Ltd. Design for Usability - Card Sorting.
Tudor, L.G., Muller, M.J., Dayton, T., & Root, R.W. (1993). A participatory design technique for high-level task analysis, critique, and redesign: The CARD method. In the proceedings of Human Factors and Ergonomics Society Annual Meetings (pp. 295-299), Human Factors and Ergonomics Society.
Tullis, T. S. (2003). Using Card-sorting Techniques to Organize Your Intranet. Intranet Journal of Strategy and Management. Mar-2003.
Tullis, T. & Wood, L., (2004), How many users are enough for a card-sorting study? In Proceedings of the UPA 2004 Conference, Minneapolis, MN, USA. Jun-2004.
Usability.gov. (ND). Perform Card Sorting.
UsabilityNet.org. (ND). Card Sorting.
Weller, S.C. & Romney, A.K. (1988). Systematic data collection. Thousand Oaks, CA: Sage.
Wood, J. & Wood, L. (2008). Card Sorting: Current Practices and Beyond. Journal of Usability Studies. 4(1), 1-6, Nov-2008.
Originators/Popularizers
The card sorting method originates from psychological research on determining human competences with abstract reasoning. In the early 90’s it was adopted for designing user interfaces as a quick and easy to perform method of prototyping. The first well-documented adaptation of this technique to exploring web-content structure was done by Jakob Nielsen during designing Sun Web in 1994.
History The Card sorting technique originates in cognitive psychology techniques. One of most significant of the methods is known as the Wisconsin Card Sorting Task (WCST), which was introduced in 1946. The WCST is quick and easy to administer, requiring no specialized equipment but the deck of cards itself and a book to compare scores to. It's used to determine competence with abstract reasoning, and ability to change problem solving strategies when needed.
Authoritative References
Coxon, A. P. M. (1999). Sorting data: Collection and analysis. Thousand Oaks, CA: Sage. Chapters 1 and 2, 1-28.
Rugg, G. and McGeorge, P. (1997). The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Systems, 14(2):80 -- 93.
Published Studies
Nakhimovsky, Y., Schusteritsch, R., & Rodden, K. 2006. Scaling the card sort method to over 500 items: restructuring the Google AdWords help center. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). CHI '06. ACM, New York, NY, 183-188.
Upchurch, L., Rugg, G., & Kitchenham, B. (2001). Using Card Sorts to Elicit Web Page Quality Attributes. IEEE Software, 84-89.
Weller, S. C. (1998). Structured interviewing and questionnaire construction. In H. Russell Bernard (Ed.) Handbook of methods in cultural anthropology. Lanham, MD: Altamira Press.
