In Chapter_6 we described how to create a custom reputation model by identifying the objects in you application, selecting appropriate inputs, and developing the processes you'll need to generate your reputations. But your work doesn't end there. Far from it. Now you have decisions to make about how to use the reputations that your system is tabulating.

In this chapter and the next, we'll discuss the many options for using reputation to improve the user experience of your site, enrich content quality, and provide incentives for your users to become better, more active participants. In this chapter specifically, we'll discuss options for whether to display reputation, whom to display it to, how to display it, and help you decided which display forms are right for your application.

For each of reputation you are creating to display or use, you should ask each of these questions before proceeding:

1. Who will be able to see the reputation?
   • Is it personal-hidden from other users but visible to the reputation holder?
   • Is it public-displayed to friends or strangers, or visible to search engines?
   • Or is it corporate- limited to internal use - for improving the site or discreetly recognizing outliers in ways that may not be visible to the community?
2. How will the reputation be used to modify your site's output?
   • Will you use the reputation to filter the lowest- or highest-quality items in a set?
   • Will you use the reputation to sort or rank items?
   • And/or will this score be used to make other decisions about how the site flows or your business operates?
3. Is this reputation for a content item or a person? Each requires a fundamentally different approach.

Though you may choose multiple answers from the list above for each reputation, try to keep it simple at first-don't try to do too much with a single reputation. Confounding the purposes of a reputation-by, for example, surfacing participation points in a public karma score-can encourage undesirable user behavior and may even backfire by discouraging participation. Read Chapter_7 and Chapter_8 completely for a solid understanding of the issues related to overloading a single reputation.

So far, the reputation you're calculating is little more than a cold numerical score rolled up from the aggregate actions of people interacting with your site. You've carefully determined the scope of the reputation, chosen the inputs that contribute to it, and thought at length about the effect that you want the reputation to generate in the community.

Now you must decide whether it makes sense to display the reputation on your site at all and, if so, to whom. How you display reputation information-how much and how prominently-will influence the actions that users take on your site, their trust in your site and one another, and their long-term satisfaction with your community.

Compelling reasons exist to keep reputations hidden from users. In fact, in some circumstances, you may want to obscure the fact that you're tracking them at all. It may sound rather Machiavellian, but the truth of the matter is this: a community under public scrutiny behaves differently (and, in many ways, less honestly) than one in blissful ignorance.

Several trade offs are involved. Displaying reputations takes up significant page real estate, requires user interface design and testing, and can compete with your content for the user's attention and understanding. Quickly, show Digg.com (Figure_7-1 ) to 10 of your friends and ask them, What kind of site is this? News? Entertainment? Community? Odds are good that at least a few of them will answer: “This appears to be some sort of contest.”

The impression that Digg makes is not a bad thing. It just demonstrates that Digg made a conscious decision to display content reputation prominently-in fact, the display of reputation is the central interaction mechanism on the site. It's practically impossible to interact with Digg, or get any use out of it, without some understanding of how community voting affects the selection and display of popular items on the site. (Digg is perhaps the most well-known example of a site that employs the vote-to-promote pattern. See Chapter_6 .)

Juxtapose Digg's approach with that of Flickr. The popular photo-sharing and discovery service also makes use of reputation to surface quality content, but it does not display explicit reputations: rather, it prominently displays items that achieve a certain reputation and that can be browsed (daily, weekly, or monthly) in the “Explore” gallery (at http://www.flickr.com/exploreFigure_7-2 . The result is a very consistent and impressive display of high-quality photos with very little indication of how those photos are selected.

Flickr's interestingness algorithm determines which photos make it into the “Explore” gallery and which don't. The same algorithm lets users sort their own photos by interestingness.

Digg and Flickr represent two very different approaches to reputation display, but the results are very much the same. Theoretically, you can always glance at the front page of Digg, or Flickr's “Explore” gallery, to see where the good stuff is-what are people watching, commenting on, or interacting with the most on the site.

How do you decide whether or not to display reputations on your site? And how prominently? Generally, follow the rule of least disclosure: do not display a reputation that doesn't add specific value to the objects being evaluated.

Likewise, don't bother asking users for reputation input (see Chapter_6 ) that you'll never use: you'll confuse users and encourage undesired patterns of “invented significance,” including abuse.

Are you tracking a reputation primarily to keep users informed about how well they or their creations are performing in the community? Consider displaying that reputation only to its owner, as a personal communication between site and user.

Personal reputations are used extensively for applications like social bookmarking, lists of favorites, training recommendation systems, sorting and filtering news feeds, providing content quality and feedback, fine-grained experience point tracking, and other performance metrics. Most of the same user interface patterns used for displaying public reputation apply to personal ones too, but take care to ensure that each user knows when her reputations will and will not be displayed to others.

Google Analytics (Figure_7-3 ) is an example of rich personal reputation information. It provides detailed information about the performance of your web site, across a known range of score types, and it is only available to you, the site owner (or others you grant access to). While that information is invaluable to you in gauging the response of a community (in this case, the entire Web) to your content, exposing it to everyone would offer very little practical benefit. In fact, it would be a horrible idea.

Some reputation display patterns provide both a personal and a public representation of a reputation. In the named-levels display pattern Chap_7-Display_Named_Levels the personal representation of the reputation score often is numeric, representing the exact score, whereas the public representation obscures exactly where in the level the target's score actually is. Online games usually report only the level to other users and the exact experience points to the player.

When the whole community would benefit from knowing the reputations of either people or content, consider displaying public reputations. Public reputations may be displayed to everyone in the community or only to users who are members of a group, are connected through a social network, or have achieved status as senior, trusted members of the community by surpassing some reputation threshold.

When is it a good idea to display public reputations? Remember our original definition: reputation is information used to make a value judgment about a person or an object in a given context for a specific time. Consider the following questions:

• What decisions am I asking users to make on my site?
• Compare items' quality against each other
• Determine someone's credibility or trustworthiness
• Decide whether something's worth reading or not

• Am I asking users to make time-sensitive decisions, or decisions in which additional, well-placed information would save them heartache?
• Can I present the reputation in a way that is fair and comprehensible, and that doesn't overwhelm the presentation of the content?

Public reputations are used for hundreds of purposes on the Web: to compare items in a list on the basis of community member feedback, to evaluate particular targets for online transaction trustworthiness, to filter and display top-rated message board posts, to rank the best local Indonesian restaurants, to show today's gallery of the most interesting photos, to display leaderboards of the top-scoring reputation targets, and much more.

Over time, public reputations can evolve to represent your community's understanding of its own zeitgeist. And there's the rub-depending on how you use public reputation, you can alienate users who aren't part of the in crowd. Yelp is all about public ratings and reviews of local restaurants, but it isn't used extensively by users over 50. Most of the reviews are written by twentysomethings [most “Yelpers” are between the ages of 26 and 35] who seem to be mostly interested in a restaurant's potential as a dating hangout.

Almost every web site with a large volume of user-generated content is using hidden reputation scores internally-as a means of tracking exactly who is saying what about a content item or another user.

• When users click the “Spam” button in a webmail application, they contribute to a database of IP addresses for abusive mail servers.
• Web crawlers constantly scan the Web to examine what sites link to what other sites and to calculate a hidden score such as Google's PageRank.
• Yahoo! Answers tracks corporate reputation for users who are particularly good at identifying bad content and gives them more power to hide bad content quickly.

And internally used reputation scores need not always be acted on immediately by scripts or bots-they can also be a very helpful tool for human decision making. Community managers often use corporate reputation reports on the most active, connected, and highest-quality user contributions and creators. They might use the information to generate high-quality best-of galleries to promote a site, or they might invite top contributors to participate in early testing of new designs, products, or features. Finally, user actions often are aggregated into reputations for behavioral targeting of advertising, customer care planning and budgeting, product feature needs assessment, and even legal compliance.

After deciding which reputation scores to display to whom, you'll need to decide how to use the scores to change the way your application works. It's easy to think that all you need to do is display a few stars here or a few points there-but if you stopped there, you wouldn't capture the most possible value from reputation.

To use reputation without displaying it, focus on how to identify the outlying reputable entities (users and content) to improve the quantity and quality of interaction on your site. When you're selecting patterns, review the goals you set for your system (see Chapter_5 ). If you're primarily concerned about identifying abusive behavior, focus on filtering and decisions. If you're going to display a lot of public reputation over many entities, focus on ranking and sorting to help users explore your content.

We'll cover patterns for making use of the reputation of entities in Chapter_8 .

At its simplest, filtering consists of sorting by one or more reputation dimensions and looking only at the first or last entries in the list to identify the highest and lowest scoring entities for potential further, even automatic, action. In reality, many reputations used for filtering are often made of more numerous and complex inputs than reputations built for public display in rankings or sorted lists.

Consider Flickr's interestingness filter reputation: it is corporate (used internally and not displayed to any user); it is complex (made up of many inputs: views, favorites, comments, and more); and it is used to automatically and continuously generate a public gallery. But the score is never displayed to users; you cannot query a photo to get its interestingness score. Perhaps the easiest way to think about a filter reputation is that, if it is not ever displayed to users, users don't have to understand what it's made up of. If users can see a reputation indicator, they'll want to know what it means and how it's calculated.

By far the most common displays of reputation are in the form of explicit lists of reputable entities, such as the restaurants in the local neighborhood with the highest average overall rating, or the list of players with the highest Elo ranks for Chess, or even which keyword search marketing terms are generating the most clicks per dollar spent.

Typically, the reputation score is used alone or in conjunction with other metadata filters, such as geographic location, to make it easy for users to sort between multiple entities at a glance. For example, to list top rated hotels in a five mile radius of a zip-code, one would combine the distance and reputation into a rank-score before displaying the list.

The primary purpose of allowing such sorting is to allow users to select an item to examine in more detail. Note that the reputation score need not be displayed to allow sorting or ranking entities. For example, to avoid encouraging abuse, public search engines typically hide search ranking scores.

The specific reputation usage patterns related to ranking and sorting are quality-sort search results, leaderboards, related items, recommendations, search relevance (such as Google's PageRank), corporate community health metrics, and advertising performance metrics.

This entire class of use patterns often is overlooked because it typically happens behind the scenes, out of sight of users. Though you may not be aware of it, more hidden decisions are made on the basis of reputation than are actually reflected directly to users either with filtering or ranking.

Billions of email messages are processed daily across the world. ISPs secretly track the IP addresses of the senders; they use this reputation to decide whether the item should be dropped, put in a bulk folder, or sent on to another content-based reputation check before being delivered to your inbox. This is only one example of many patterns used by Web 2.0 site operators around the world to manage user-generated content without exposing the scores or the methods for their calculations. When used for abuse mitigation, the value of the reputation score can be directly correlated with cost savings from increased efficiency in customer care and community management, as well as in hardware and other operational costs. Each year, the IP reputation system for Yahoo! Mail saves tens of millions of dollars in real costs for servers, storage, and overhead.

When a reputation score is complex, such as karma (see next section), it may be suitable for public display as a standalone score so that others can make specific, context-sensitive decisions. EBay's feedback and other reputation scores are a good example of a publicly shared karma. Since the transactions for items are often one of a kind, content filtering and ranking don't provide enough information for anyone to make a decision about whether to trust the seller or buyer.

Of course, some reputation is nonnumeric and can't be ranked at all-for example, comments, reviews, video responses, and personal metadata associated with source users who evaluate your entities. These forms of input must be displayed so that users can interpret the input directly. For instance, a 20-year-old single woman in Los Angeles who is looking for a new sweater might want to discount the ratings given by a 50-year-old married man living in Alaska. Nonnumeric reputation often provides just enough additional context for people to make more informed judgments about entities.

Here are the specific reputation usage patterns related to decisions: critical threshold, automatic rejection, flag for moderation, flag for promotion, reviews and comments

Reputable entity refers to everything in a database, including users and content items, with one or more reputations attached to it. All kinds of reputation score types and all kinds of display and use patterns might seem equally valid for content reputation and karma, but usually they're not. To highlight the differences between content reputation and karma, we've categorized them by the ways in which they're typically calculated: simple and complex reputation.

• Simple Reputation
  • Simple reputation is any reputation score that is generated directly by user evaluation of a reputable entity and that is subject to an elementary aggregation calculation, such as simple average. For example, simple reputation is used on most ratings-and-reviews sites. Simple reputation is direct and easy to understand.

• Complex Reputation
  • Complex reputation is a score aggregated from multiple evaluations, including evaluations of different but related targets, calculated with an opaque method. email IP spammer, Google PageRank, and eBay feedback reputations are examples of complex reputation. It's an indirect evaluation, and users may not understand how it was calculated even if the score is displayed.

Content reputation scores may be simple or complex. The simpler the score is-that is, the more it directly reflects the opinions or values of users-the more ways you can consider using and presenting it. You can use them for filters, sorting, ranking, and in many kinds of corporate and personalization applications. On most sites, content reputation does the heavy lifting of helping you to find the best and worst items for appropriate attention.

Content reputation is about things-typically inanimate objects without emotions or the ability to directly respond in any way to its reputation.

But karma represents the reputation of users, and users are people-they are alive, they have feelings, and they are the engine that powers your site. Karma is significantly more personal and therefore sensitive and meaningful. If a manufacturer gets a single bad product review on a web site, it probably won't even notice. But if a user gets a bad rating from a friend-or feels slighted or alienated by the way your karma system works-she might abandon an identity that has become valuable to your business. Worse yet, she might abandon your site altogether and take her content with her. (Worst of all, she might take others with her.)

Take extreme care in creating a karma system. User reputation on the web has undergone many experiments, and the primary lesson from that research is that karma should be a complex reputation and it should be displayed rarely.

Be careful with Karma-sometimes making things as simple and explicit as possible is the wrong choice for reputation:

• Rating a user directly should be avoided. Typical implementations only require a user to click once to rate another user and are therefore prone to abuse. When direct evaluation karma models are combined with the common practice of streamlining user registration processes (on many sites opening a new account is an easier operation than changing the password on an existing account), they get out of hand quickly. See the example of Orkut in Chap_7-Display_Numbered_Levels .
• Asking people to evaluate others directly is socially awkward. Don't put users in the position of lying about their friends.
• Using multiple inputs presents a broader picture of the target user's value.
• Economics research into “revealed preference,” or what people actually do, as opposed to what they say, indicates that actions provide a more accurate picture of value than elicited ratings.

Karma calculations may be opaque because the score is valuable as status, has revenue potential, and/or unlocks privileged application features.

There are several important things to consider when displaying karma to the public:

• Publicly displayed karma should be rare because, as with content reputation, users are easily confused by the display of many reputations on the same page or within the same context.
• Publicly displayed karma should be rare because it can create the wrong incentives for your community. Avoid sorting users by karma. See Chap_7-Leaderboards_Considered_Harmful .
• If you do display it publicly, make karma visually distinct from any nearby content reputation. Yahoo!'s EU message board displays the karma of a post's author as a colored medallion, with the message rated with stars. But consider this: Slashdot's message board doesn't display the karma of post authors to anyone. Even the display of a user's own karma is vague: “positive,” “good,” or “excellent.” After originally displaying karma publicly as a number, over time Slashdot has shifted to an increasingly opaque display of karma.
• Public displayed karma should be rare because it isn't expected. When Yahoo! Shopping added Top Reviewer karma to encourage review creation, they displayed a Top Reviewer badge with each review and rushed it out for the Christmas 2006 season. After the New Year had passed, user testing revealed that most users didn't even notice the badges. When they did notice them, many thought they meant either that the item was top rated or that the user was a paid shill for the product manufacturer or Yahoo!.

Though karma should be complex, it should still be limited to as narrow a context as possible. Don't mix shopping review karma with chess rank. It may sound silly now, but you'd be surprised how many people think they can make a business out of creating an Internet-wide trustworthiness karma.

Yahoo! holds reputation for karma scores to a higher standard than reputation for content. Be very careful in applying terminology and labels to people, for several reasons:

• Avoid labels that might appear as attacks. They set a hostile tone that will be amplified in users' responses. This caution applies both to overly positive labels (such as “hotshot” or “top” designations) or negative ones (such as “newbie” or “rookie” ).
• Avoid labels that introduce legal risks. What if a site labeled members of a health forum “experts,” and these “experts” then gave out bad advice?

These are rules of thumb that may not necessarily apply to a given context. In role-playing games, for example, publicly shared simple karma is displayed in terms of experience levels, which are inherently competitive.

Reputation data can be displayed in numerous formats. By now, you've actually already done much of the work of selecting appropriate formats for your reputation data, so we'll simply describe pros and cons of a handful of them-the formats in most common use on the Web.

The formats you select will depend heavily on the types of inputs that you selected in Chapter_6 . If, for instance, you've opted to let users make explicit judgments about a content item with 5-star ratings, it's probably appropriate to display those ratings back to the community in a similar format.

However, that consistency in display doesn't work when the reputation you want to display is an aggregation or transformation of scores derived from very different input methods. For instance, Yahoo! Movies provides a critic's score as a letter grade compiled from scores from many professional critics, each of whom uses a different scale (some use 4- or 5-star ratings, some thumb votes, and still others use customized iconic scores). Such scores are all transformed into normalized scores, which can then be displayed in any form.

Here are the four primary data classes for reputation claims:

• Normalized Score
  • Most composite reputations are represented as decimal numbers from 0.0 to 1.0, with all inputs converted, or normalized, to this range. See Chapter_6 for more on the specific normalization functions. Displaying a reputation in the various forms presented in the remainder of this chapter is also known as denormalization: the process of converting reputation data into a presentable format.

• Summary Count, Raw Score, and Other Transitional Values
  • Sometimes a reputation must hold other numeric values to better represent the meaning of the normalized score when it is displayed. For example, in a simple-mean reputation, the summary count of the inputs that contribute to the reputation are also tracked, allowing a display patterns that can override or modify the score. For example, a pattern could require a minimum number of inputs (see Chap_3-Low_Liquidity_Effects .)

In cases where information may be lost during the normalization process, the original input value, or raw score, should also be stored. Finally, other related or transitional values may also be available for display, depending on the reputation statement type. For example, the simple average claim type keeps the rolling sum of the previous ratings along with a counter as transitional values in order to rapidly recompute the average when new ratings arrives.

• Freeform Content
  • Freeform inputs provided by users may be constrained along certain dimensions, such as format or length, but they are otherwise completely up to the users' discretion. Some examples of this class of data are user comments and video responses. Notice that items like the title of a product review (if the review writer is given the option to provide one) is also a freeform element: it gives review writers an opportunity to provide an opinion about a target. Content tags are also a type of freeform content element.

Freeform content is a notable class of data because, while deriving computable values from them is harder, users themselves derive a lot of qualitative benefit from it.

• Metadata
  • Sometimes, machine-understood information about an object can yield insight into its overall quality or standing within a community. For comparative purposes, for example, you might want to know which of two different videos was available first on your site. Examples of metadata relevant to reputation include the following:
  • Time stamp
  • Geographical coordinates
  • Format information, such as the length of audio, video, or other media files
  • The number of links to an item or the number of times the item itself has been embedded in another site

A normalized score ranges from 0.0 to 1.0 and represents a reputation that can be compared to other reputations no matter what forms were used for input. When displaying normalized scores to users, convert them to percentages (multiply by 100.0), the numeric form most widely understood around the world. From here on, we assume this transformation when we discuss display of a percentage or normalized score to users.

The percentage may be displayed as a whole number or with fixed decimal places, depending on the statistical significance of your reputation and on user interface and layout considerations. Remember to include the percent symbol (%) to avoid confusion with the display of either points or numbered levels.

Things to consider before displaying percentages

• Use this format when the normalized reputation score is reasonably precise and accurate. For example, if hundreds or thousands of votes have been cast in an election, displaying the exact average percentage of affirmative and negative votes is easier to understand than just the total of votes cast for and against.
• Be careful how you display percentages if the input claim type isn't suitable for normalized output of the aggregated results. For example, consider the displaying the results of a series of thumb votes; though you can display the thumb graphic that got the majority of votes, you'll probably still want to display either the raw votes for each or the percentages of the total up votes and down votes.

Figure_7-4 displays content reputation as the percentage of thumbs-up ratings given on Yahoo! Television for a television episode. Notice that the simple average calculation requires that the total number of votes be included in the display to allow users to evaluate the reliability of the score.

• Consider that a graphical sliding scale or thermometer view will makes the reputation easier to understand at a glance. If necessary, also display the numeric value alongside the graphic.

Figure_7-5 shows a number of Okefarflung's karma scores as percentage bars, each representing his reputation with various political factions on Worlds of Warcraft. Printed over each bar is one of the current named levels (see Chap_7-Display_Named_Levels ) that his current reputation falls in.

Points are a specific example of an accumulator reputation display pattern: The score simply increases or decreases in value over time, either monotonically (one at a time) or by arbitrary amounts. Accumulator values are almost always displayed as digits, usually alongside a units designation, for example 10,000XP or Posts: 1,429. The aggregation of the vote-to-promote input pattern is an accumulator.

If an accumulator has a maximum value that is understood by the reputation system, an alternative is to display it using any of the display patterns for normalized scores, such as percentages and levels.

Using Points and Accumulators

• Display counts of actions collected from many users, such as voting and favorites.

Figure_7-6 shows an entry from Digg.com, which displays two different accumulators: The number of Diggs and Comments. Note the Share and Bury buttons. Though these effect the chance that an entity is displayed on the home page, the counts for these actions are not displayed to the users.

• Publicly display points when you wish to encourage users to take actions that increase or decrease the value for an entity.

Figure_7-7 shows a typical participation-points enabled website, in this case Yahoo! Answers. Points are granted for a very wide range of activities including loggin in, creating content, and evaluating other's contributions. Note that this mini-profile also displays a numbered level (see Chap_7-Display_Numbered_Levels ) to simplify comparison between users. The number of points accumulated in such systems can get pretty large.

• Alternatively, consider keeping a point value of personal and presenting any public display as either a numbered level or a named level.

One very useful strategy for reputation display is to use statistical evidence: simply include as many of the inputs in a content item's reputation as possible, without attempting to aggregate them in visible scores. Statistical evidence lets users zero in on the aspects of a content item that they consider the most telling. The evidence might consist of a series of simple accumulator scores:

• Number of views
• Number of links
• Number of comments
• Number of times marked as a favorite or voted on

Using Statistical Evidence

• Use this display format when a variety of data points would provide a well-rounded view of an entity's worth or performance

Figure_7-8 shows YouTube.com's many different statistics associated with each video, each subject to different subjective interpretation. For example, the number of times a video is Favorited can be compared to the total number of Views to determine relative popularity.

• Use statistical evidence in displays of counts of actions collected from many users, such as voting and favorites.

Yahoo! Answers provides a categorical breakdown of statistics by contributor, as shown in Figure_7-9 . This allows readers to notice if the user is an answer-person (as shown here) or a question-person or something else.

• Optionally, you might extend statistical evidence to include even more information about how a particular score was derived.

Figure_7-10 shows how Yahoo! Answers displays not only how many people have “starred” a question (that is, found it interesting); it also shows exactly who starred it. However, displaying that information can have negative consequences: among other things, it may create an expectation of social reciprocity (for example, your friends might become upset if you opted not to endorse their contributions).

Levels are reputation display patterns that remove insignificant precision from the score. Each level is a bucket holding all the scores in a range. Levels allow you to round off the results and simplify the display. Notice that the range of scores in each level need not be evenly distributed, as long as the users understand the relative difficulty of reaching each level.

Common display patterns for levels include numbered levels and named levels.

Using Levels

• Use levels when the reputation is an average and inputs are limited to a small, fixed set, such as 5 stars.
• Levels are helpful when the reputation is an average and may be calculated from a very small number of inputs. Levels will hide irrelevant precision.
• Most applications use level when reputation accumulates at a nonlinear rate. For example, in many role-playing games, each experience level requires twice as many experience points as the previous level.
• Use levels if some features of your application are unlocked depending on the reputation score; users will want to know that they've achieved the required threshold.
• Be careful using levels when the input was gathered using a different scale. If the user clicks a thumb icon, displaying the resulting score as 5 stars will be confusing.
• Be careful when listing entities by level not to surface relative position within a level. Doing so can encourage undesired competition for specific page positions. Sort by the lower precision level value, not the high precision normalized value.

Numbered levels are the most basic form of level display. This display pattern consists of a simple numeric value or a list of repeated icons representing the level that the reputation score falls into. Usually levels are 0 or 1 to n, though arbitrary ranges are possible as long as they make sense to users. The score may be an integer or a rounded fraction, such as 3½ stars. If the representation is unfamiliar to users, consider adding an element to the interface to explain the score and how it was calculated. Such an element is mandatory for reputations with nonlinear advancement rates.

Using Numbered Levels

• Assign numbered level if the reputation will be displayed in a rank-ordered sort a list of entities.

Figure_7-11 shows a typical Stars-and-Bars display pattern for ratings and reviews. Stars and Bars are numbered levels, which happen to be displayed as graphics. In this example, each has a numbered level of 0 to 5. Though each review's ratings are useful when displayed alongside the entity, the average of the overall score is used to rank-order results on search results pages.

• It is typical to use numbered levels to display aggregate reputation if the inputs were also numbered levels. Did you input stars? Output stars.

Figure_7-12 is the Karma ratings from Orkut.com. The Fans indicator is an accumulator (see Chap_7-Points_and_Accumulators ), and the Trusty, Cool, and Sexy ratings are numeric levels. The user's simply click on the smiling faces, ice cubes, and hearts next to their friends profiles to influence their scores. Many sites don't allow direct karma ratings such as these with good reason (see Chap_7-Displaying_Karma .)

• If you need to display more than 10 levels, use numbered levels. Consider using numbered levels instead of named levels if you display more than five levels.

Figure_7-13 displays two forms, out of many, of numbered levels for the game World of Warcraft. The user controls a character who's name is shown in the Members column, the first numbered level is labeled “Level” and ranges from 1 to 80, representing the amount of time and skill the user has dedicated to this character. The Guild Rank is a reverse-rank numbered level that represents the status of the user in the guild-this score is assigned by the guild master, who has the lowest guild rank.

In a named levels display pattern, a short, readable string of characters is substituted for a level number.

The name adds semantic meaning to each level so that users can more easily recognize the entity's reputation when the reputation is displayed separately. Is the user a “silver contributor” or is the beef prime, choice, select, or standard?

Using Named Levels

• Named levels are useful when the number of labels is five or less, so that each level can have a name that accurately expresses its meaning.

Table_7-1 and Figure_7-14 show the meat grading levels use by the United States Department of Agriculture.The labels are descriptive, representing existing industry terms, and several are shared across different animal species-providing consumers a consistent standard for comparison.

• Named levels are particularly useful numeric levels are too impersonal or encourage undesired competition.
• If you're considering using numeric levels, but find that the top and bottom levels should feel closer together than the numeric distance between them would otherwise indicate-consider using named levels instead. This is especially useful with karma scores so that new participants don't get stuck with a demeaning level indicator, like “Level 1 of 10” .

Figure_7-15 displays the current named levels used by WikiAnswers.com for user contributions. The original three categories were Bronze, Silver, and Gold-named after competitive medals. They are granted when non-linearly increasing thresholds are met. Over time, the system has been expanded on three separate occasions to reward the nearly compulsive contributions of a handful of users.

Any list based on highest or lowest reputation scores. Ranking systems are by their very nature comparative, and-human nature being what it is-are likely to be perceived by a community as a design choice by the product team to encourage of competition between users.

A leaderboard is a rank-ordered listing of reputable entities within your community or content pool. Leaderboards may be displayed in a grid, with rows representing the entities and columns describing those entities across one or more characteristics (name, number of views, and so on). Leaderboards provide an easy and approachable way to display the best performers in your community.

Using Ranked Lists

• Use leaderboards for content liberally. Provide filtered views of the boards to slice and dice by time (“Popular Today/This Week/All Time” ) or by reputation type (“Most Viewed/Top Rated” ).

Figure_7-16 shows YouTube's leaderboard ranking for most-viewed videos as a grid. With numbers this high, it's hard for potential reputation abusers to push inappropriate content onto the first page. Note that there are several leaderboards, one each for Today, This Week, This Month, and All Time.

• Use leaderboards for people sparingly, and only in contexts that are competitive by nature. Consider giving people leaderboards narrow scope (for example, only ranking me against my friends, to keep the comparisons fun and the stakes low).

Figure_7-17 displays Yahoo! Answer's leaderboard. The original version of this page was based solely on the number of points accumulated by participation, users quickly figured out which actions produced the most points for the least effort. When the user's best-answer percentage was eventually added to the profile display, it was discovered that the top-ranked users all had quality scores of less than 10%!

A specialized type of leaderboard where top-ranking entities are grouped into numerical categories of performance. Achieving top-10 status (or even top 100) should be a rare and celebrated feat.

Using Top-X Ranking

• Use top-X leaderboards for content to highlight only the best of the best contributions in your community.

Figure_7-18 shows a Top-X display for content: the BillBoard's Hot 100's list of top recordings. The artists themselves have very little, if any, direct influence over their song's rank on this list.

• Use top-X designations for people sparingly, and only in contexts that are competitive by nature. Because available categories in a top-X system are bounded, they will have greater perceived value in the community.

Figure_7-19 displays the new index of Top-X karma for Amazon.com review writers. The very high number of reviews written by each of these leaders creates value both for Amazon and the reviewers themselves. Authors and publishers seek them out to review/endorse their book-sometimes for a nominal fee. The original version of this reputation system, now known as “Classic Reviewer Rank” , suffered deeply from first-mover effects (see Chap_3-First_Mover_Effects ) and other problems detailed in this book. This eventually lead to the creation of the new model, as pictured.

It's still too early to speak in absolutes about the design of social-media sites, but one fact is becoming abundantly clear: ranking the members of your community-and pitting them one-against-the-other in a competitive fashion-is typically a bad idea. Like the fabled djinni of yore, leaderboards on your site promise riches (comparisons! incentives! user engagement!!) but often lead to undesired consequences.

The thought process involved in creating leaderboards typically goes something like this: there's an activity on your site that you'd like to promote; a number of people are engaged in that activity who should be recognized; and a whole bunch of other people won't jump in without a kick in the pants. Leaderboards seem like the perfect solution. Active contributors will get their recognition: placement at the top of the ranks. The also-rans will find incentive: to emulate leaders and climb the boards.

And that activity you're trying to promote? Site usage should swell with all those earnest, motivated users plugging away, right? It's the classic win-win-win scenario. In practice, employing this pattern has rarely been this straightforward. Here are just a few reasons why leaderboards are hard to get right.

Many leaderboards make the mistake of basing standings only on what is easy to measure. Unfortunately, what's easy to measure often tells you nothing at all about what is good. Leaderboards tend to fare well in very competitive contexts, because there's a convenient correlation between measurability and quality. (It's called “performance” -number of wins versus losses within overall attempts.)

But how do you measure quality in a user-generated video community? Or a site for ratings and reviews? It should have very little to do with the quantities of simple activity that a person generates (the number of times an action is repeated, a comment given or a review posted). But such measurements-discrete, countable, and objective-are exactly what leaderboards excel at.

Even if you succeed in leavening your leaderboard with metrics for quality (perhaps you weigh community votes or count send-to-a-friend actions), be aware that-because a leaderboard singles out these factors for praise and reward-your community will hold them in high esteem too. Leaderboards have a kind of “Code of Hammurabi” effect on community values: what's written becomes the law of the land. You'll likely notice that effect in the activities that people will-and won't-engage in on your site. So tread carefully-are you really that much smarter than your community, that you alone should dictate its character?

Even sites that don't display overt leaderboards may veer too closely into the realm of comparative statistics. Consider Twitter and its prominent display of community members' stats.

The problem may not lie with the existence of the stats but in the prominence of their display. (Figure_7-20 ) They give Twitter the appearance of a community that values popularity and the sheer size of a participant's social network. Is it any wonder, then, that a whole host of community-created leaderboards have sprung up to automate just such comparisons? Twitterholic, Twitterank, Favrd, and a whole host of others are the natural extension of this value-by-numbers approach.

In the earliest days of Orkut (Google's also-ran entry in social networking), the product managers put a fun little widget at the top of the site: a country counter, showing where members were from. Cute and harmless, right? Google had no way of knowing, however, that seemingly the entire population of Brazil would make it a point of national pride to push their country to the top of that list. Brazilian blogger Naitze Teng wrote: “Communities dedicated to raising the number of Brazilians on Orkut were following the numbers closely, planning gatherings and flash mobs to coincide with the inevitable. When it was reported that Brazilians had outnumbered Americans registered on Orkut, parties… were thrown in celebration.”

Brazil has maintained its number one position on Orkut (as of this writing, 51% of Orkut users are Brazilian; the United States and India are tied for a distant second with 17% apiece). Orkut today is basically a Brazilian social network. That's not a bad “problem” for Google to have, but it's probably not an outcome that they would have expected from such a simple, small and insignificant thing as a leaderboard widget.

The most insidious artifact of a leaderboard community may be that the very presence of a leaderboard changes the community dynamic and calls into question the motivations for every action that users take. If that sounds a bit extreme, consider Twitter: friend counts and followers have become the coins of that realm. When you get a notification of a new follower, aren't you just a little more likely to believe that it's just someone fishing around for a reciprocal follow? Sad, but true. And this despite the fact that Twitter itself never has officially featured a leaderboard-it merely made the statistics known and provided an API to get at them. In doing so, it may have let the genie out of the bottle.

This entire chapter has focused on the explicit display of reputation, usually directly to users. Though important, this isn't usually the most valuable use for this information. Chapter_8 describes using reputation to modify the utility of an application-to separate the best entities from the pack, and to help identify and destroy the most harmful ones.