Ratings Panel and Working Group

Panel chair: Dale Kunkel, Ph.D., University of Arizona

Panel presenters in order of presentation were:
Joanne Cantor, Ph.D, University of Wisconsin
Douglas Gentile, Ph.D., Iowa State University, National Institute on Media and the Family
Patricia Vance, President, Entertainment Software Rating Board
Kimberly M. Thompson, ScD, Kids Risk Project, Harvard School of Public Health

The Ratings Panel and Working Group presented and discussed perspectives of a select group of academic researchers, who have published scientific studies involving media ratings and video games, and perspectives from the video game industry, represented by the ESRB president.

Presentations of researcher perspectives
The academic researchers noted the video game rating system was an improvement over other media rating systems and complimented the ESRB for providing content descriptors and for being willing to listen to the concerns of parents and researchers.

However, the researchers described studies demonstrating that the video game ratings still have weaknesses that need to be addressed. They used research to propose changes to provide consumers with more objective, reliable, valid and detailed ratings information.

Among the key research cited and discussed by the academic researchers:

Desired features of video game ratings
Joanne Cantor, Ph.D., University of Wisconsin

View presentation

Parents prefer descriptors of media content, over age recommendations

  • A national survey of parents found that, by a wide margin, parents favor ratings based on content information rather than generalized ratings that suggest appropriate ages for users. (Cantor, Stutman & Duran, 1996; Bushman & Cantor, 2003) A meta-analysis confirmed these findings.
Ratings sometimes attract viewers to controversial content
  • Restrictive ratings sometimes backfire, making entertainment media, including video games, movies and TV programming, more enticing to youth. However, the effect is small to moderate and varies, depending on the gender and age of the audience. The effect is moderate (r = .25) for males overall and nonexistent for females. It is stronger for males aged 18-22 (r = .31) than for adolescent males (r = .21). Both age and content ratings can make programs more attractive. (Bushman & Cantor, 2003)
Based on the research findings, Cantor concluded that because ratings can have effects that are both positive (informing parents) and negative (attracting young viewers), ratings should provide enough concrete content information to outweigh the negative effects.

Descriptors should be much more explicit
  • Video game descriptors provided by web sites such as familymediaguide.com provide many more explicit details than the ESRB video game ratings.
Cantor proposed that video game product labels include the type of information that would be more likely to affect a parents' decision. Rather than the ESRB descriptors of "blood and gore" and "intense violence" (for Grand Theft Auto: Liberty City Stories, for example), parents might appreciate knowing that the player can "punch and kick people to death," see "blood spurting from victims' bodies," "sever their heads with swords," "shoot a person's head off with a sniper rifle," and "shoot and kill police officers" (which they learn from the familymediaguide.com descriptors on the same game.)

Research Citations
Bushman & Cantor, 2003
Cantor, Stutman & Duran, 1996

A Model for Reliable & Valid Media Ratings
Douglas Gentile, Ph.D., Iowa State University, National Institute on Media and the Family

View presentation (Coming Soon)

Achieving Scientific Reliability & Validity
Research, especially involving TV and movies, has demonstrated that ratings can be scientifically reliable and valid, and that one system can be applied to multiple media:
  • As one example, consistent standards for a reliable and valid universal media rating system have been demonstrated through KidScore, a rating system established by the National Institute on Media and the Family in 1996, and run until about 2002, after which point it became a consumer-based rating system.
  • The KidScore ratings were validated nationally and applied by trained parent raters to 253 TV programs, 276 movies or videos, and 166 computer or video games that were popular or marketed to children. When the KidScore ratings were compared to industry ratings, it was clear that, in particular, TV and movie ratings were not in agreement with parent ratings, with the exception of the most restrictive ratings (TV-MA, R movies). Parents had highest disagreement with respect to video game ratings in the Teen category.
Gentile noted that the experience of KidScore demonstrates that a universal media rating system, capable of producing reliable and valid ratings for all media types, is possible and practical and would provide parents a more consistent source of information on all types of media consumed by their children.

Considering a universal media rating system
Among the considerations for a universal media rating system are:
  • There is research addressing what makes a good rating system, what parents want in a rating system, and what would be most effective.
  • Although several studies demonstrate that parents would prefer a universal system, barriers to the creation of such a system remain, such as resistance by the Motion Picture Association of America to improve the movie rating system.
  • Although parental concerns about each media form differ, in general, parents want to know about the content of each, such as amount and depiction of violence, sexual content, etc.
  • The video game industry has been the most responsive industry to adjust its rating system to parent and research concerns and to serve consumers better. The industry could continue to be leaders by working to create a new 2.0 standard, universal media rating system that incorporates scientific research and meets parent information needs.
  • Other possible options for the creation of a universal rating system include an independent ratings oversight group, which could either assign ratings or could train existing industry raters to use a single system reliably.
Research Citations
Walsh & Gentile, 2001
KidScore Ratings

Research comparing actual content observed to information provided by ESRB
Kimberly M. Thompson, ScD, Kids Risk Project, Harvard School of Public Health

Usefulness of game ratings based on evidence

View presentation
  • ESRB ratings provide important information and parents should use the ratings. But the ratings do not convey complete information about game content for all games. For example, they often fail to mention depictions of substances.
  • Confusion exists about whether the content descriptors are intended to alert parents to all of the potential types of content that may be of potential concern to them or to provide a reason for the age-based rating. Appeals to context-based ratings as opposed to more objective criteria represents an important shift that consumers need to understand.
  • Much greater clarity and consistency is needed to ensure that consumers know what the rating information means and what to expect.
Strengths & limitations of current game ratings:
  • Strengths include: Good visibility (although content descriptors are only on the back of the package), information available at the point of purchase; online information includes age-based ratings and content descriptors
  • Limitations include: No standards for content descriptors; inconsistencies observed in content analyses, non-transparent process and no accountability, no content descriptors for "positive" content, may be desirable to have one more age category (T-15)
Research regarding accuracy of video game ratings:
  • Independent, peer-reviewed assessments of the accuracy of the content descriptors performed by the Kids Risk Project on video games rated "E" "T" and "M" found inconsistencies. See Thompson & Haninger, 2001; Haninger and Thompson, 2004; Haninger et al., 2004; Thompson et al, 2006.
  • Inconsistencies may result from assignment of ratings prior to final product; the rater is asked to assign ratings prior to being able to play the games in the version sold to consumers
  • The industry currently provides mixed message about whether it seeks to provide "accurate" and "objective" rating information to parents
Addressing limitations to improve video game ratings:
  • Clear, publicly-available standards for assigning age-based ratings and content descriptors, including Web posting of the submission package questions online so parents know what information is collected as part of the process
  • Transparency in the process and a real board of independent individuals that will improve accountability
  • Changing the rating process to include experience of game play of each game submitted by the rating board before assigning the age-based rating and content descriptors for that game
Research Citation Presentation by video game industry ratings board president
Patricia Vance, president, Entertainment Software Rating Board

View presentation

The Entertainment Software Rating Board perspective is that the video game ratings system is effective, parents have high awareness of and are using the ratings, and the ESRB promotes use of ratings and other resources on media.

Parents confirm effectiveness of ESRB rating system
  • Kaiser Family Foundation study in Fall 2004 reported more than half of 1,000 parents surveyed found ESRB ratings to be "very useful," more than any other media ratings.
  • ESRB-commissioned surveys confirm parental usefulness of ESRB ratings: 83% of parents with kids who play video games are aware of ESRB ratings and three in four use ratings regularly in making decisions to buy or rent video games.
  • Parents agreed with ESRB ratings 82% of the time and found the ratings "too strict" another 5% of the time. A comparison of ESRB and National Institute on Media and the Family ratings showed that virtually all age ratings assigned were either identical, or varied by only 1-2 years in either direction.
Rating assignments inherently subjective
  • Rating assignments are inherently the result of human beings making a subjective judgment about what they review and experience when reviewing it. The act of assigning a rating is not a science, and its effectiveness cannot be evaluated in a laboratory. (Note - ESRB remarks)
  • Video game ratings are based on social/cultural norms, to reflect the values of parents making the video game purchases.
  • Ratings are inherently limited to concise labels on game packaging
  • Ratings must be assigned prior to the product's release
  • Ratings are intended to provide consumers with basic information at the point of purchase.
The ESRB encourages consumers to use a wide range of resources in aggressively promoting use of its ratings and to be informed and involved about the media their children consume. The ESRB aggressively promotes use of its ratings through a multi-channel marketing program.

Improvement efforts: focus on rating awareness and uses
  • The ESRB has received minimal complaints from consumers
  • Parents have been relatively silent when it comes to requesting that the ESRB do things differently
  • Video game research should focus on how and why parents choose video games.
  • Parents as a specific group or organization were not participants at the Summit.
  • Parents need to take responsibility for their children's media consumption.
  • The ESRB currently is running PSA ads in leading consumer magazines, encouraging parents to step up and take responsibility for the computer and video games that their children consume.
  • PSAs and other materials have been developed specifically to target minority populations.
Discussion
Following the panel presentations, the panelists and Summit participants discussed the extent to which a different ratings process might reduce subjectivity, whether and how ratings could be consistent when based on subjective social norms, and what other information could be communicated through ratings.

The selection of video game raters was discussed regarding the raters' backgrounds, experience and any training they may receive. In particular, the academic researchers suggested the raters should have some educational background or training in child development in order to understand what appropriate age ranges would be for video games.

The discussion progressed to the Ratings Working Group, which proposed ideas and made recommendations regarding the video game rating process.

Ratings Research & Policy Working Group Recommendations
The working group was chaired by:
Dale Kunkel, Ph.D., University of Arizona

Members of the Ratings Research & Policy Working Group included:
Joanne Cantor, Ph.D, University of Wisconsin
Jeanne Funk, Ph.D. University of Toledo
Douglas Gentile, Ph.D. Iowa State University, National Institute on Media and the Family
Bradley S. Greenberg, Ph.D, Michigan State University
Jay Senter, Common Sense Media
Patricia Vance, President, Entertainment Software Rating Board

The discussions of the working group were divided into four categories: rating research, rating process, rating structure and rating oversight.

The working group was consistent in agreeing on several recommendations, especially regarding the need for research.

Note:
  • * Recommendations that did not have unanimous consensus are noted by the asterisk. Points of disagreement were often between individuals who were proposing incremental changes to the current system and individuals with major revisions and long-term goals of creating a new rating system. In forwarding these recommendations to the full Summit, the Ratings Working Group noted those recommendations that merited consideration but were not unanimously agreed upon.
Ratings research
The working group agreed that research on ratings should involve parental input and research, addressing these points:
  • How to make video game content descriptors more effective and useful
  • Measure the level of parents' knowledge and understanding of the ratings (not simply awareness)
  • Identify the most effective means to inform and motivate parents to use video game ratings
Several additional areas for academic scholars to research or advance proposals were identified, as follows:
  • Apply behavioral usage research to study how ratings are actually used in practice
  • Identify reasons why parents disregard the M rating (Mature, for age 17 and older)
  • Study how to build ratings awareness and use among communities with high-risk youth
Rating Process
  • Apply media effects research as an important, but not sole, factor in the rating process
  • Identify the characteristics a video game rater should have:
    • * Consider optimal rater characteristics and employ those types of raters
  • * Increase transparency of ratings: determine what/ whether parents want additional information on video content, and if so, how and where it should be provided
Ratings Structure
Recommendations for the rating structure are subdivided into two categories: fine tuning the rating structure and the consideration of "positive" ratings:

Fine tuning
  • Explore how best to respond to content added by video game users (YouTube effect)
  • * Change the M rating from "17 and older" to "18 and older"
  • * Include more descriptor information of gender/ethnic issues
  • * Allow for non-standardized descriptors and supplementary information
  • * Have special consideration in ratings when video games contain victimization
Consideration of "positive" ratings
  • * Rate video games for pro-social or educational information in descriptors
    Dissenting concerns raised on this recommendation were that a positive descriptor could be used to undercut negative descriptors, such as violent or sexually explicit game content.
Other
  • Hold a conference of experts across multiple media to determine if a universal rating system is needed and what it might look like, or how best to amend current rating systems; if a need is indicated, develop a universal rating system
Ratings oversight
  • * Establish an independent ratings board
  • * Establish an expert advisory board on the rating process.
  • * Establish industry ratings agencies as "accountable" to Congress