IO1-F. Eggshell model: establishing information credibility

In our search of, and focus on, pragmatism, the following model for analysing and estimating the credibility of a piece of information was put together. We view it as a precursor to adequate content-checking and source-checking. We designed it to a balanced critical approach to information which also takes into consideration the time and effort required for full and thorough checks. The elements of the model are constructed in a pragmatic manner to allow a comprehensive and reliable, yet fast, checking of key elements of the information body and carrier.

Eggshell model: a structural approach to information

L. LOAD
Header, Recency, Content, Medium

S. STRUCTURE
Author, Text, Metrics, Source

L1. LOAD-HEADER
Sensation or fear, uncertainty, deceit

The header is often the part of the information which attracts the immediate attention of the eye. A critical look at the header from a content perspective would include consideration of whether it has been worded to create fear, uncertainty or deceit (sometimes referred to as FUD). Another look could be a judgement on whether the header is seeking to provoke immediate action, is sensational in nature, conveys impartial information or demonstrates clear bias, plays an emotional appeal.

* Issues regarding the structure of the heading or use of all caps are dealt with in S2. Structure-Text.

S1. STRUCTURE-AUTHOR
Name, handle, avatar

The section of the piece of information, whatever the carrier, which displays the author name, social media handle, profile picture/avatar, user status, etc., is an important element to screen when attempting to validate that piece of information. Strange or impersonating/deceitful spelling of the author-related strings are an early warning for disinformation. Profile images/avatars which do not represent the author, impersonate others, use official logos, names, etc. without being related to them, or which are in some other way not consistent with the context of the medium/carrier, are another flashing-red light.

L2. LOAD-RECENCY
Date, zombie potential, swing-mode potential

The recency of the information can be quite revealing: is the date indicated at all, is it indeed current/recent, is the date corresponding to the novelty assumed by the title, is there a clear connection between the date and the facts/events which are the subject of the information. Information which tends to stick around, or resurfaces every now and then, while pretending to be news, must be weighed in carefully. There is a significant chance that such information may be misleading, not last because it is apparently detached from the current event which has triggered its reappearance.

S2. STRUCTURE-TEXT
Caps, spelling

Both research and evidence suggest that people react differently to different language structures that are otherwise logically equivalent. Articles and social media message formats often exploit this, resulting in heavy use of caps and emphasised formatting. Another key element to watch is whether spelling and other language mistakes point to possible dissonances with other elements (author, content, medium).

L3. LOAD-CONTENT
Consistency, fact or opinion, parody, joke

A critical look at the content, or body part, is important in informing our judgment. We should be asking questions such as: is what I read (see, hear) consistent throughout the text (video), is it consistent with the author’s previous pattern of behaviour/thinking/writing, is it consistent and coordinated with facts and data that is available?

It is important to be able to tell whether the content is about facts or opinions, and if that is made clear by the author. Is the author of the information an authority on the subject? Does the author have a history of biased opinions?

A somewhat more difficult assessment would be whether the content is a serious one or a parody/joke. It is not rare for people to misread a parody and spread it by raising it to a factual status. A good question on the overall content would be: who has an interest in this being published, and does that interest precludes of excludes manipulative motives.

S3. STRUCTURE-METRICS
Likes, shares, comments, reach

False and deceitful information is often presented in information types and structures which are popular or familiar to the user, e.g. tweet, FB/instagram post. Bogus information and misleading advertisement commonly display large numbers of likes/shares/comments/followers, and even produce false comments which do not behave in the usual way (e.g. FB-like looking comments thread not clickable/not linked to FB profiles, clear domination of positive review with untraceable link to the claimants. High values of the metrics are not an indicator of increased likelihood of the information not being true, however, a lot of misleading fabricated content put the focus on high numbers, as people are more likely to accept something as true when is looks as if it has already been accepted, supported or shared by many other people.

L4. LOAD-MEDIUM
Scam/bogus medium, relevance of images

There are many examples of scam/websites or individual pieces of information (posts, emails, articles, comments, product reviews). Many of them are easily recognisable by the lacking functionality in places where you would expect to find it: unclickable links, non-functional menus, pictures of text used instead of text, mimicking and misleading social media profiles, comments, threads, posts. Email scams where you are informed of having won a great prize or received an inheritance are just on of the many examples. Same goes for communications presumably sent by banks, tax authorities, major courier service providers etc. attempting to lure the user into a specific action.

Images (still and video) are among the key targets of any mis- and disinformation detection methods, and rightly so. Images are powerful and have strong and lasting impact on people. The relevance (incl. recency) of images to the claim could range from full (image depicting an act) to circumstantial (potentially proving the claim, but not directly), to illustrative (neutral with regard to supporting the claim, may depict a person, a building, a product), to fully-detached (not relevant to the specific claim at all, but used to leave a strong emotional imprint or trigger an action - e.g. a bloody knife picture pointed to a child in an article about a missing child). But relevance is not the only indicator. Images can be real, without any intervention, could be changed (e.g. to illustrate a parody), or be completely doctored (manipulated in a way to mislead). Rapidly-emerging and developing technolgies now allow computers to generate images from text, that are entirely fabricated, though may satisfy the relevance criterion.

S4. STRUCTURE-SOURCE
Transparency, URL

Source is as important as facts when it comes to checking a piece of information. Is the source clearly mentioned? Is it traceable to the origin (is there a reference to the original source, or a link to the original material)? False information is usually structured in a way which implies that someone said/did something, but offer no proof that can be checked. When available URLs are an important indicator in the overall assessment: does it belong to an official organisation/institution, does it mimic closely a name of a person, company or media, is it working, is it pointing to specific proof/source, or is it just a generic link to a home page? Does the URL with proof/source belong to the person or entity who are supposed to be the source of the information (e.g. a national/public statistical office) or is it a third-party URL? As a rule of thumb, most trustworthy content links to the primary origin, less reliable content links to intermediaries/unnamed reports, unreliable content links to no proof or evidence.

Eggshell model for analysing information items to establish credibility

The Eggshell model is currently our draft model proposal for analytical approach to information items with a purpose of establishing information credibility. The model embodies our current understanding of their structure (S) and meaning load (L) elements.

Eggshell model: practical questions inventory

The questions which can help establish the credibility of an information item are divided in the two main categories of the model, and further broken down by element. We consider this list a starting point and the approval of the overall approach by the De Facto partners will trigger further work on it.

L. LOAD
Load aspects are focused on the content, its semantics, style and purported meaning/feeling conveyed, appropriateness and relevance of the used medium/carrier, internal message coherence.

S. STRUCTURE
Structural aspects relate to technical parameters and are not directly related to the information or meaning which the carrier conveys.

L1. LOAD-HEADER
  1. Is the title/header of the message sensational?
  2. Does the title/header provoke fear or uncertainty?
  3. Does the title/header urge/prompt readers’ action?
S1. STRUCTURE-AUTHOR
  1. Who is the author of the original post/information?
  2. Who is posting/re-posting/sharing?
  3. One author/thread or multiple authors/threads?
  4. Does the post fit the usual content profile of the author? Is it consistent with other author’s posts?
  5. Is the author clearly marked?
  6. Is the username/handle/nickname verified?
  7. Is the username/handle/nickname close to, but not exactly the same as the one held by the original person/organisation it pretends to represent?
L2. LOAD-RECENCY
  1. Is the publication dated and is the claim or statement referring to an event with a specified date?
  2. If the information is about data, is the time period known/mentioned for which the data is presumably valid?
S2. STRUCTURE-TEXT
  1. Are there many misspellings or awkward grammar, text structure and text layout?
  2. Does the text look like it has been produced by an automated tool, rather than a person?
  3. Is it possible that the text has been passed through a machine translation software?
L3. LOAD-CONTENT
  1. Is the post about ideology?
  2. Is the post about a highly divisive topic (religion, politics, race, ethnicity)?
  3. Does the post provoke fear, uncertainty or doubt?
  4. Does the post contain an image for which the text makes claims or suggests a link to a claim?
  5. Is the post about facts or opinions?
  6. Is the post about results from polling and surveys, i.e. numeric results which are not precise by definition?
  7. Is it possible that the post/information is a joke?
  8. Does the publication contain a valid URL link which confirms the contents?
  9. Do you find confirmation of the content in other media?
  10. If the post relates to a purported statement by a person, does the suggested statement correspond to the usual style and thinking/speaking/writing of that person?
  11. If the information is about data, is the source of the data known and mentioned?
S3. STRUCTURE-METRICS
  1. What is the number of shares/likes/comments?
  2. What is the depth of the thread (e.g. Twitter)?
  3. Do you find it likely that the author/sharer has checked the facts/alleged facts before passing the information on to you/other people?
  4. If the post shares content located in a separate message/carrier, does the shared post include a comment or position statement from the sharer, or is it blank sharing?
L4. LOAD-MEDIUM
  1. Have you seen the picture before in a different context? In what context does the content image appear in the search-engine results? Does the context relate to/match the claim?
  2. Are there photos or videos which do not directly refer to the claims and statements in the text? Are they likely to be taken out of context?
  3. If you are reading/evaluating information on a website, is the website fully-functional? Are there a lot of inactive/unclickable menu items or large sections?
S4. STRUCTURE-SOURCE
  1. Is the URL close to the authentic but not exactly the same (i.e. is the URL mimicking a real one)?
  2. If you are reading/evaluating information on a website, is the website fully-functional? Are there a lot of inactive/unclickable menu items or large sections?
  3. Does the publication mention the source of the information?
  4. Does the publication contain a valid link to a source?

Print Friendly, PDF & Email