skip to main content

Critical thinking tools – the CRAP test Critical thinking tools – the CRAP test

Long reads
Authors: Pru Mitchell
Critical thinking tools – the CRAP test

For two months now fake news has been the news. It has taken a particular level of blatancy to get the topic of critical thinking and information literacy into mainstream media.

This culminated in the Oxford Dictionaries nominating post-truth as its 2016 word of the year. They describe this as the ‘circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.

This view of the world is at odds with the philosophy underlying education. Curriculum statements place ethical understanding and critical thinking at the core of intellectual activity. Critical thinking ‘involves students learning to recognise or develop an argument, use evidence in support of that argument, draw reasoned conclusions, and use information to solve problems,’ (ACARA, 2016). The ability to ‘critically analyse information and ideas from a range of sources’ appears across subject areas.

Wikipedia as a trusted source of facts

Knowing which sources, news and information we can trust is a fundamental element of information literacy.

One source that has been somewhat contentious in education is Wikipedia. While it has been popular to direct students not to use it, Wikipedia is recognised globally as the world’s fifth most popular website (Alexa, 2017). Many teachers and students rely on Wikipedia as an accessible source that provides an overview of a topic and an initial list of references. As we re-evaluate traditional sources in response to the reality of fake news, filter bubbles, bias, propaganda, spam and deliberately misleading information, it is worth revisiting Wikipedia’s position in this landscape.

On 15 January each year the Wikipedia community celebrates the fact that, since it started in 2001, this online encyclopedia has remained ad-free, not-for-profit and written entirely by volunteers. Over 100 000 people passionate about facts contributed to the English language Wikipedia last month (Wikipedians, 2017). A fundraising letter from co-founder Jimmy Wales late last year made it clear that facts matter:

"You may have heard people say that in today's online media environment, facts have become subjective. At Wikipedia, we reject that idea … at the end of the day, people share a common thirst for high-quality, neutral information."

Wikipedians use two well-established academic conventions to verify facts: citation, and peer review. Wikipedia relies on references to quality published information, and works closely with educators and librarians to improve its references. The Wikipedia Library has a #I1ib1ref campaign underway in early 2017 to encourage librarians to add references in the places where the [citation needed] flag appears in Wikipedia. It provides tutorials on how to add or edit references.

Building critical thinking

Just as we cannot rely solely on internet filters to protect students, providing lists of trusted news and information sources does not enable students to develop the skills they need to evaluate information themselves.

A recent Stanford study by Wineburg & McGrew (2016) tested the civic online reasoning of school and university students and found that young people lacked the ability to judge the credibility of information. In their activities, 80 per cent of students did not correctly identify sponsored content on a website as an advertisement, and many absorbed social media news without considering the source or even following a given link.

While teachers and librarians already address critical thinking (or digital literacy, media literacy, information literacy) across year levels and in many subject areas, the Stanford study shows the importance of reviewing both the sources being addressed and the approach being taken.

CRAP testing

One critical thinking tool popular in schools is the memorable mnemonic CRAP (Orenic, 2008), which teaches students to consider four key elements when evaluating information.

  • Currency
  • Reliability
  • Authority
  • Purpose/Point of View

As an example of how to use the CRAP test, here are some questions we use to guide students in evaluating a Wikipedia article:

  • Currency – Use the [View history] tab on a Wikipedia article to check when it was created, and whether it has been kept up-to-date. Review the publication dates on the references at the end of the article;
  • Reliability – While vandalism and spam can be an issue for Wikipedia, every edit is recorded, and all versions can be easily restored. There are automated filters plus human patrollers who monitor recent edits and pages of interest to them. Wikipedia has a checklist of reliability criteria that considers accuracy of information, appropriateness of images, style and focus of articles, exclusion and removal of false information, comprehensiveness, scope and coverage, susceptibility to editorial and systemic bias, and quality of writing;
  • Authority – All statements in Wikipedia should be facts backed up by references to reputable third-party sources. A quality article should have no [citation needed] flags, and a quality reference list. Readers can check who has made edits on an article in the [View history] tab by clicking on the editor’s username and a link to the editor’s other Wikipedia contributions;
  • Purpose/Point of view – Wikipedia is an encyclopedia, written from a neutral point of view. Its content should include facts, not opinions. To check for debates and contentious issues related to topics, click on the [Talk page] of an article and read the discussion.

Wikipedia is just one source educators can use to help students build their referencing and critical thinking skills (as opposed to telling them to disregard it as a source).

Here’s a video guide on how to evaluate whether a source is scholarly using the CRAP test, posted by Charles Sturt University’s Library Division.

There is no simple solution to the significant cultural and political breakdown that allows fake news and false information to take hold in the community.

While none of us is immune from bias and filter bubbles, education must be part of the solution. Schools, especially those with information specialists on staff, are in a position to build the critical thinking and CRAP detection skills of their students, staff and communities.

Pru Mitchell is a volunteer with Wikimedia Australia, a chapter of the not-for-profit Wikimedia Foundation that operates Wikipedia and its sister projects.

Stay tuned: In tomorrow’s episode of our monthly podcast series The Research Files, Teacher chats to Sarah McGrew, one of the co-authors of the Stanford University study mentioned in this article, about why young people experience difficulties when it comes to evaluating information they find online.

References

ACARA. (2016). Critical and creative thinking. Australian Curriculum, v.8.3. Retrieved from http://www.australiancurriculum.edu.au/generalcapabilities/critical-and-creative-thinking

Alexa. (2017). wikipedia.org traffic statistics. Retrieved from http://www.alexa.com/siteinfo/wikipedia.org

Orenic, K. (2008). The CRAP test. http://loex2008collaborate.pbworks.com/w/page/18686701/The%20CRAP%20Test

Wikipedians. (2017). Retrieved from https://en.wikipedia.org/wiki/Wikipedia:Wikipedians

Wineburg, S. & McGrew, S. (2016). Evaluating information: The cornerstone of civic online reasoning. Stanford History Education Group. Retrieved from http://sheg.stanford.edu/upload/V3LessonPlans/Executive%20Summary%2011.21.16.pdf

For two months now fake news has been the news. It has taken a particular level of blatancy to get the topic of critical thinking and information literacy into mainstream media.

This culminated in the Oxford Dictionaries nominating post-truth as its 2016 word of the year. They describe this as the ‘circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.

This view of the world is at odds with the philosophy underlying education. Curriculum statements place ethical understanding and critical thinking at the core of intellectual activity. Critical thinking ‘involves students learning to recognise or develop an argument, use evidence in support of that argument, draw reasoned conclusions, and use information to solve problems,’ (ACARA, 2016). The ability to ‘critically analyse information and ideas from a range of sources’ appears across subject areas.

Wikipedia as a trusted source of facts

Knowing which sources, news and information we can trust is a fundamental element of information literacy.

One source that has been somewhat contentious in education is Wikipedia. While it has been popular to direct students not to use it, Wikipedia is recognised globally as the world’s fifth most popular website (Alexa, 2017). Many teachers and students rely on Wikipedia as an accessible source that provides an overview of a topic and an initial list of references. As we re-evaluate traditional sources in response to the reality of fake news, filter bubbles, bias, propaganda, spam and deliberately misleading information, it is worth revisiting Wikipedia’s position in this landscape.

On 15 January each year the Wikipedia community celebrates the fact that, since it started in 2001, this online encyclopedia has remained ad-free, not-for-profit and written entirely by volunteers. Over 100 000 people passionate about facts contributed to the English language Wikipedia last month (Wikipedians, 2017). A fundraising letter from co-founder Jimmy Wales late last year made it clear that facts matter:

"You may have heard people say that in today's online media environment, facts have become subjective. At Wikipedia, we reject that idea … at the end of the day, people share a common thirst for high-quality, neutral information."

Wikipedians use two well-established academic conventions to verify facts: citation, and peer review. Wikipedia relies on references to quality published information, and works closely with educators and librarians to improve its references. The Wikipedia Library has a #I1ib1ref campaign underway in early 2017 to encourage librarians to add references in the places where the [citation needed] flag appears in Wikipedia. It provides tutorials on how to add or edit references.

Building critical thinking

Just as we cannot rely solely on internet filters to protect students, providing lists of trusted news and information sources does not enable students to develop the skills they need to evaluate information themselves.

A recent Stanford study by Wineburg & McGrew (2016) tested the civic online reasoning of school and university students and found that young people lacked the ability to judge the credibility of information. In their activities, 80 per cent of students did not correctly identify sponsored content on a website as an advertisement, and many absorbed social media news without considering the source or even following a given link.

While teachers and librarians already address critical thinking (or digital literacy, media literacy, information literacy) across year levels and in many subject areas, the Stanford study shows the importance of reviewing both the sources being addressed and the approach being taken.

CRAP testing

One critical thinking tool popular in schools is the memorable mnemonic CRAP (Orenic, 2008), which teaches students to consider four key elements when evaluating information.

  • Currency
  • Reliability
  • Authority
  • Purpose/Point of View

As an example of how to use the CRAP test, here are some questions we use to guide students in evaluating a Wikipedia article:

  • Currency – Use the [View history] tab on a Wikipedia article to check when it was created, and whether it has been kept up-to-date. Review the publication dates on the references at the end of the article;
  • Reliability – While vandalism and spam can be an issue for Wikipedia, every edit is recorded, and all versions can be easily restored. There are automated filters plus human patrollers who monitor recent edits and pages of interest to them. Wikipedia has a checklist of reliability criteria that considers accuracy of information, appropriateness of images, style and focus of articles, exclusion and removal of false information, comprehensiveness, scope and coverage, susceptibility to editorial and systemic bias, and quality of writing;
  • Authority – All statements in Wikipedia should be facts backed up by references to reputable third-party sources. A quality article should have no [citation needed] flags, and a quality reference list. Readers can check who has made edits on an article in the [View history] tab by clicking on the editor’s username and a link to the editor’s other Wikipedia contributions;
  • Purpose/Point of view – Wikipedia is an encyclopedia, written from a neutral point of view. Its content should include facts, not opinions. To check for debates and contentious issues related to topics, click on the [Talk page] of an article and read the discussion.

Wikipedia is just one source educators can use to help students build their referencing and critical thinking skills (as opposed to telling them to disregard it as a source).

Here’s a video guide on how to evaluate whether a source is scholarly using the CRAP test, posted by Charles Sturt University’s Library Division.

There is no simple solution to the significant cultural and political breakdown that allows fake news and false information to take hold in the community.

While none of us is immune from bias and filter bubbles, education must be part of the solution. Schools, especially those with information specialists on staff, are in a position to build the critical thinking and CRAP detection skills of their students, staff and communities.

Pru Mitchell is a volunteer with Wikimedia Australia, a chapter of the not-for-profit Wikimedia Foundation that operates Wikipedia and its sister projects.

Stay tuned: In tomorrow’s episode of our monthly podcast series The Research Files, Teacher chats to Sarah McGrew, one of the co-authors of the Stanford University study mentioned in this article, about why young people experience difficulties when it comes to evaluating information they find online.

References

ACARA. (2016). Critical and creative thinking. Australian Curriculum, v.8.3. Retrieved from http://www.australiancurriculum.edu.au/generalcapabilities/critical-and-creative-thinking

Alexa. (2017). wikipedia.org traffic statistics. Retrieved from http://www.alexa.com/siteinfo/wikipedia.org

Orenic, K. (2008). The CRAP test. http://loex2008collaborate.pbworks.com/w/page/18686701/The%20CRAP%20Test

Wikipedians. (2017). Retrieved from https://en.wikipedia.org/wiki/Wikipedia:Wikipedians

Wineburg, S. & McGrew, S. (2016). Evaluating information: The cornerstone of civic online reasoning. Stanford History Education Group. Retrieved from http://sheg.stanford.edu/upload/V3LessonPlans/Executive%20Summary%2011.21.16.pdf

Where does responsibility for information literacy and critical thinking sit in your school?

How are you keeping up-to-date with the explosion of new information sources and media?

What model/s do you use across the school to build student skills in critically analysing information across all forms of media?

Where does responsibility for information literacy and critical thinking sit in your school?

How are you keeping up-to-date with the explosion of new information sources and media?

What model/s do you use across the school to build student skills in critically analysing information across all forms of media?


Skip to the top of the content.