skip to main content

Critical thinking tools – the CRAP test

Long reads
Critical thinking tools – the CRAP test

For two months now fake news has been the news. It has taken a particular level of blatancy to get the topic of critical thinking and information literacy into mainstream media.

This culminated in the Oxford Dictionaries nominating post-truth as its 2016 word of the year. They describe this as the ‘circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.

This view of the world is at odds with the philosophy underlying education. Curriculum statements place ethical understanding and critical thinking at the core of intellectual activity. Critical thinking ‘involves students learning to recognise or develop an argument, use evidence in support of that argument, draw reasoned conclusions, and use information to solve problems,’ (ACARA, 2016). The ability to ‘critically analyse information and ideas from a range of sources’ appears across subject areas.

Wikipedia as a trusted source of facts

Knowing which sources, news and information we can trust is a fundamental element of information literacy.

One source that has been somewhat contentious in education is Wikipedia. While it has been popular to direct students not to use it, Wikipedia is recognised globally as the world’s fifth most popular website (Alexa, 2017). Many teachers and students rely on Wikipedia as an accessible source that provides an overview of a topic and an initial list of references. As we re-evaluate traditional sources in response to the reality of fake news, filter bubbles, bias, propaganda, spam and deliberately misleading information, it is worth revisiting Wikipedia’s position in this landscape.

On 15 January each year the Wikipedia community celebrates the fact that, since it started in 2001, this online encyclopedia has remained ad-free, not-for-profit and written entirely by volunteers. Over 100 000 people passionate about facts contributed to the English language Wikipedia last month (Wikipedians, 2017). A fundraising letter from co-founder Jimmy Wales late last year made it clear that facts matter:

"You may have heard people say that in today's online media environment, facts have become subjective. At Wikipedia, we reject that idea … at the end of the day, people share a common thirst for high-quality, neutral information."

Wikipedians use two well-established academic conventions to verify facts: citation, and peer review. Wikipedia relies on references to quality published information, and works closely with educators and librarians to improve its references. The Wikipedia Library has a #I1ib1ref campaign underway in early 2017 to encourage librarians to add references in the places where the [citation needed] flag appears in Wikipedia. It provides tutorials on how to add or edit references.

Building critical thinking

Just as we cannot rely solely on internet filters to protect students, providing lists of trusted news and information sources does not enable students to develop the skills they need to evaluate information themselves.

A recent Stanford study by Wineburg & McGrew (2016) tested the civic online reasoning of school and university students and found that young people lacked the ability to judge the credibility of information. In their activities, 80 per cent of students did not correctly identify sponsored content on a website as an advertisement, and many absorbed social media news without considering the source or even following a given link.

While teachers and librarians already address critical thinking (or digital literacy, media literacy, information literacy) across year levels and in many subject areas, the Stanford study shows the importance of reviewing both the sources being addressed and the approach being taken.

CRAP testing

One critical thinking tool popular in schools is the memorable mnemonic CRAP (Orenic, 2008), which teaches students to consider four key elements when evaluating information.

  • Currency
  • Reliability
  • Authority
  • Purpose/Point of View

As an example of how to use the CRAP test, here are some questions we use to guide students in evaluating a Wikipedia article:

  • Currency – Use the [View history] tab on a Wikipedia article to check when it was created, and whether it has been kept up-to-date. Review the publication dates on the references at the end of the article;
  • Reliability – While vandalism and spam can be an issue for Wikipedia, every edit is recorded, and all versions can be easily restored. There are automated filters plus human patrollers who monitor recent edits and pages of interest to them. Wikipedia has a checklist of reliability criteria that considers accuracy of information, appropriateness of images, style and focus of articles, exclusion and removal of false information, comprehensiveness, scope and coverage, susceptibility to editorial and systemic bias, and quality of writing;
  • Authority – All statements in Wikipedia should be facts backed up by references to reputable third-party sources. A quality article should have no [citation needed] flags, and a quality reference list. Readers can check who has made edits on an article in the [View history] tab by clicking on the editor’s username and a link to the editor’s other Wikipedia contributions;
  • Purpose/Point of view – Wikipedia is an encyclopedia, written from a neutral point of view. Its content should include facts, not opinions. To check for debates and contentious issues related to topics, click on the [Talk page] of an article and read the discussion.

Wikipedia is just one source educators can use to help students build their referencing and critical thinking skills (as opposed to telling them to disregard it as a source).

Here’s a video guide on how to evaluate whether a source is scholarly using the CRAP test, posted by Charles Sturt University’s Library Division.

There is no simple solution to the significant cultural and political breakdown that allows fake news and false information to take hold in the community.

While none of us is immune from bias and filter bubbles, education must be part of the solution. Schools, especially those with information specialists on staff, are in a position to build the critical thinking and CRAP detection skills of their students, staff and communities.

Pru Mitchell is a volunteer with Wikimedia Australia, a chapter of the not-for-profit Wikimedia Foundation that operates Wikipedia and its sister projects.

Stay tuned: In tomorrow’s episode of our monthly podcast series The Research Files, Teacher chats to Sarah McGrew, one of the co-authors of the Stanford University study mentioned in this article, about why young people experience difficulties when it comes to evaluating information they find online.

References

ACARA. (2016). Critical and creative thinking. Australian Curriculum, v.8.3. Retrieved from http://www.australiancurriculum.edu.au/generalcapabilities/critical-and-creative-thinking

Alexa. (2017). wikipedia.org traffic statistics. Retrieved from http://www.alexa.com/siteinfo/wikipedia.org

Orenic, K. (2008). The CRAP test. http://loex2008collaborate.pbworks.com/w/page/18686701/The%20CRAP%20Test

Wikipedians. (2017). Retrieved from https://en.wikipedia.org/wiki/Wikipedia:Wikipedians

Wineburg, S. & McGrew, S. (2016). Evaluating information: The cornerstone of civic online reasoning. Stanford History Education Group. Retrieved from http://sheg.stanford.edu/upload/V3LessonPlans/Executive%20Summary%2011.21.16.pdf

Where does responsibility for information literacy and critical thinking sit in your school?

How are you keeping up-to-date with the explosion of new information sources and media?

What model/s do you use across the school to build student skills in critically analysing information across all forms of media?

Shiv Satchit 16 January 2017

During the last three decades academic researchers in tandem with voluntary organisations deserve much credit for framing media literacy debates against the perspective of possible negatives of the mass media. But with the advent of digitalisation the torrent of media texts (text, images, audio, video and graphics) crisscrossing our universe has become the new impetus for media literacy. 

Digital convergence also has wide implications for our digital economy which is increasingly based on digital knowledge where a policy-driven multi-literacy pedagogy is long overdue. However, consciousness for a new thinking in education is evident in different countries besides where educators have begun to take on board what academics and advocacy groups have highlighted for years.

In the UK the government ambivalence towards media literacy is disintegrating but in a somewhat confused manner. Our socio-cultural and economic society is impacted by multi-literacies. Any attempt to introduce high order literacy in all levels of curriculum is contingent up digital literacy, information literacy and computer literacy.

Given that the digitally wirelessly and wired networked technology, it is unrealistic to regulate the unprecedented level of user-generated-content (UGC) through the social media. Hence, a protectionist approach is a plausible strategy in guarding ourselves against the negative externality effects of the media.

Technology is not enough and education needs to accept the new paradigm, evidently insinuating the development of higher order thinking skills.(HOTS)‘ ‘critical thinking to be purposeful, self-regulatory judgement which results in interpretation, analysis,evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgement is based. CT is essential as a tool of inquiry. (The Delphi Report’, Facione, 1990)

The Hewlett Foundation, the Partnership for 21st Century Learning,(P21) and others have also called for the inclusion of four Cs:  (a) critical thinking, (2) communication, (3) collaboration, and (4) creativity skills while strengthening personal attributes like persistence and leadership for success in college and the workforce.  So the heat is on, let’s us wait and see educators see some sense in enpowering students and adults through formal education buttressed by digital pedagogy - e-learning.

divp 24 January 2017

Thanks for sharing brief info

Leave a comment




Skip to the top of the content.