IO1-D. Education-related layers & harm potential of content types
We have earlier introduced the major disinformation categories (IO1-B. Major disinformation categories and content types) and the 11 types of disinformation (IO1-A. Eleven types of disinformation).
Now we introduce what we consider to the education-related layers of concern to create a meaningful matrix which highlights the disinformation content types which are more likely to inflict harm, and to outline the process through which harm is done.
We have also provided examples that come predominantly from education and training contexts. Our harm potential spider web is a quick way to remind learners and teachers where danger lies.
Education-related layers of concern - working definitions and assumptions, examples
When approaching the task of defining the education-related areas of concern with regard to mis- and disinformation, our objective was to provide a clear and comprehensive overview of the different modalities of impact. And to arrive at a point where we will know more about which types of information have the higher impact potential, and precisely what the nature of that harmful impact is.
It is important to note that an example may fall within more than one of these aspects!
Item and Description
Examples
Infiltration into teaching/learning content
This layer of concern evaluates to what extent it is likely that a particular content type can infiltrate into the sources and materials which educators use to prepare for their teaching, and learners use in their cognitive process to prepare for assignments.
On the learners side, such materials they use may be generated by their teacher, by other educators, or have an origin completely detached from education and training contexts (e.g. information generated by businesses, organisations and institutions, or by content-generating individuals). We believe that the infiltration potential may be present regardless of what was the initial motivation for creating the information, i.e. false-context vs. manipulated vs fabricated. It is the relevance and relatedness of the information to the teaching/learning topic which triggers the search for, and access of, information.
〉A national statistical office of a country/region (official government data) issues an annual statistical report with GDP data. A teacher incorporates this data in their lecture/text materials and forgets about it. An year from now, the students find the now old data in a sentence which reads "the last year's GDP was 3.2%" without references to the year or the source. The student accepts this at face value.
〉A student includes in their assignment paper information from product advertisement brochure stating that product X is superior to all others, failing to verify the information and confirm/reject the conclusion of such a comparison.
〉A teacher suddenly decides to tell an exciting story to illustrate a concept, but does not remember well some of the facts. The students are impressed by teh story and take a careful not of all details, including the misrepresented facts. They start using the now changed and flawed story, and do it with confidence.
Likely to stick around for a long time: zombie potential
The zombie layer refers to the assessment whether an information item has a short or a long life.
Whereas a piece of advertisement is usually in circulation for weeks or months, some extremely creative ads may go viral, gain popularity and have information materials written about them, or even start a new life as part of a meme. Conspiracy theories and hoaxes typically exhibit a high zombie potential, so do pseudo science and AI-generated fabricated content.
〉Preparing an assignment on the history of human space exploration, a student finds a number of internet resources with high level of elaboration and detail, a lot of supporters, even a lively communty of like-minded people.
They read about the claim that people have never been to the Moon, and that the video footage from the 1969 Apollo 11 landing on the Moon was fabricated in a film studio instead. There seems to be a large amount of material supporting that view over several decades.
It is possible that a student uses this information in their assignment, and because the conspiracy theory is "settled" (survived many decades), such information enjoys long zombie life. This happens without the need for the information to be actively shared or be part of an active propaganda campaign - it is in a state of latency, and so is its harm potential.
Recursive impact: swing mode potential
The swing-mode potential is about information items which, once created, are recurring in time.
This is usually related to an issue that is recurring itself, and to which the information item is related in some way. Election cycles are such recurring events: they lead to already existing information about public figures or political parties resurface, to that information being put to use in the current cycle, and then to retire until brough up again in the next cycle.
〉Most media outlets publish short articles/videos related to repetitve events, such as:
- switching to daylight savings time and back
- events having happened "on this day in history"
- end of year retrospects
- new year committements
- spring diets, summer diets, autumn diets, winter diets
- holiday shopping tips
- top 5 places to go, top 10 things to do when (...)
- once in a year holidays and their origin
- traditional, culture-specific or religion-specific holidays
- customs and habits (of a nation, of millenials, of generation Y...)
Such materials are not considered to be of the same importance as the mainstream publications of media outlets, and they are therefore often put together by junior staff or interns. In fact, even experienced journalists may fall short of the professional standards, and pressed for time, they also opt for the quick fix - take everything that has been published on the same topic in the last 5 years, change the date, edit (or not) a line or two, perhaps replace the feature image, and re-publish, as new, and in this way reviving disinformation or misinformation, should the original information contain such.
Intangible impact: opinion change potential
Opinion change may be triggered and facilitated by a wide array of motivations and information. Masterful manipulation can result in opinion change, but this can also happen if the moment is right and the individual is open to, or actively exploring, a related issue.
This is one of the assessment layers to which we attach a paramount importance. We attempted to attribute to each type of information the probability of changing the opinion of the learner who is exposed to it. This is particularly hard to assess, because it is a function of, among others, the motivation to create the information in the first place, the motivation to distribute/share it, and the skillfulness of the author and/or distributor. For example, if there is an explicit motivation to deceive or manipulate, i.e. we are talking about manipulated or fabricated content, then there is an obvious objective to either change opinions or reinforce particular opinions. And when the author is a skillful manipulator of words, or the distribution is planned and coordinated for maximum impact, then the opinion change potential is very high. We refer to this as an intangible impact because it works directly at cognitive level and is not a conscious process for the receiving side (learner).
〉A student is asked to prepare, as part of a learnign assignment, an overview/presentation of the torism industry in a particular geographical area. They conduct a research on the available tourist accommodations and facilities, and they find customer reviews and ratings for the 2 major hotels in that area. This scenario has 2 endings:
Scenario A. One of the hotels has produced fake reviews on its website which are extremely positive, without even a hint that something can be less than perfect about the hotel and its services. The student, and the audience for which the rpesentation is made, now have a very positive opinion on the hotel, and will likely choose it for their next booking, rather than the other hotel. Here the manupulation attepmt is made by the hotel, and the student extends this manipulation to other people, unwillingly.
Scenario B. In a public review/rating system, a client who was very, very dissatisfied by the service received in one of the hotels writes a series of very negative reviews with an anonymous client profile. This changes the overall rating of the hotel from 8.7 to 6.4. As a result, people who see this information will be less likely to book with the hotel. The manipulation attempt by enitrely fabricated content, is made by the hotel's client, and the student extends this manipulation to other people, unwillingly.
〉A young family is reminded at the doctor's office that it is time for a vaccine for their infant. They decide to seek information about the vaccination online. They are not familiar with the concepts of fact-checking and source-checking, and they have no previous knowledge on the topic. They easily find a very large number of manipulative websites which claim that vaccinations are dangerious and parents should refuse their children having the shot. The parents are uncertain and scared by the gloomy and horrendous claims. They fail to conduct proper search and get information from realiable sources backed with scientific data, and they fall prey to the manipulation. They now have a strong - and misinformed - opinion against vaccines and are likely to share it with others. Students preparing a learning assignment on the vaccination topic are just as likely to suffer from the manipulation.
Tangible impact: direct use of content in learning
Direct use of content in new learning/training materials implies that original content has not been modified or altered. Original content has tangible presence in the new material.
With this assessment layer we identify whether a content is likely to find its way to training/learning content in its original (or only slightly changed - e.g. abridged) form, visual style or wording. Assuming that several information types can have such a tangible impact (because we are able to detect the original), this can range from fully-identical use (e.g. text or data citation) to a very washed-down use (e.g. short and non-intrusive/non-emphasised mention). In most cases, the driver for one use or the other is the relevance and relatedness of the original information type to the learning material in which it is included or integrated.
We are assessing the potential for direct use only - not re-phrasings, build-upons, or abstractions based on the original content. (which we consider earlier in the table, under infiltration potential and opinion change potential).
〉A student writes a research-type assignment on the Holocaust. They find the website of a Holocaust-denial organisation. Not applying critical thinking or questioning the scientific evidence, they simply copy a large portion of text from the website which contains false and manipulative claims. They do so because the material which they found seems to take care of the assignment (relevance to the topic) and the teacher has not given specific instructions how sources and facts must be checked, so this is the easiest thing to do. The assignment is done, and in very little time (because the source text was directly taken, without any reflection or attempt to corroborate).
Elaboration and complexity: level of detail
This item is about the level of elaboration and detail of the information item. It is a slightly different aspect from the rest of the column. We believe this is an aspect which is both meaningful and potentially relevant to the development of detection algorithms. It refers to information item properties such as the content complexity/elaboration, whether it is a single, isolated statement, fact, number, or whether there are many additional layers created to support the dis-informative claim. The rationale is that the more elaborate a piece of disinformation is, the more likely there is (1) a large number of originating agents (organised/coordinated) and/or (2) there are bigger vested interests in the disinformation effort.
However, this item differs slightly in nature from the rest of the list. It might be equally -- if not more -- practical to consider it in the light of the Eggshell model, and its suggestive list of questions related to the critical evaluation of a given information item (IO1-F. Eggshell model: establishing information credibility), and the related fact-checking and source-checking.
* Click the image to enlarge!
〉There is a comprehensive study by Matthew Hindman of the George Washington University released in October 2018 which exemplifies how complex manipulation and disinformation operations are carried on Twitter. (EN)
The study is focused on the US 2016 elections campaign and post-election period. It is a long read, but we highly recommend it!
Harmful impact on education: danger zone
The danger zone is, for the moment, a result from application of an arbitrary threshold. Its purpose is to help focus the attention on the information types which are deemed to have the greatest impact on education and training. In this way a critical mind can turn on an alert mode when it meets such information types, and address the issue by triggering a set of actions, notably source-checking and fact-checking. It can also lead to assessing the situation by applying the 4 key cognitive science pillars which underpin our model: systemic causality, motivated cognition, emphasis and equivalency frames, and frames/framing.
We have also suggested a graphical representation of this aggregate indicator:
