Emotion May Predict Susceptibility to Fake News but Emotion Regulation Does Not Seem to Help.

Reference

Bago, Bence, Leah R. Rosenzweig, Adam J. Berinsky, and David G. Rand. 2022. “Emotion May Predict Susceptibility to Fake News but Emotion Regulation Does Not Seem to Help.” Cognition and Emotion, June, 1–15. https://doi.org/10.1080/02699931.2022.2090318.

Intervention

Code

intervention_info <- tibble(
    intervention_description = 'Only studies 2, 3 and 4 tested an intervention. The intervention aimed at emotion regulation. In study 2, participants were asked to apply either emotional suppression (`intervention_label` = `emotion_suppression`) or emotion reappraisal techniques (`intervention_label` = `emotion_reappraisal`) when considering the veracity of several headlines. In the emotion reappraisal condition, participants read: "As you view and read the headlines, please try to adopt a detached and unemotional attitude. Please try to think about what you are reading objectively. Read all of the headlines carefully, but please try to think about what you are seeing in such a way that you feel less emotion." In the emotion suppression condition, they read: "As you view and read the headlines, if you have any feelings, please try your best not to let those feelings show. Read all of the headlines carefully, but try to behave so that someone watching you would not know that you are feeling anything at all." In studies 3 and 4 of the paper, the authors only tested the emotion suppression treatment.',
    intervention_selection = "suppression",
    intervention_selection_description = 'Since we can only include one treatment per control condition in the meta-analysis, we remove participants in the emotion reappraisal condition in study 2. This allows us to compare across their studies.
',
    control_format = "picture, lede, source",
    originally_identified_treatment_effect = FALSE)

# display
show_conditions(intervention_info)

intervention_description	intervention_selection_description
Only studies 2, 3 and 4 tested an intervention. The intervention aimed at emotion regulation. In study 2, participants were asked to apply either emotional suppression (`intervention_label` = `emotion_suppression`) or emotion reappraisal techniques (`intervention_label` = `emotion_reappraisal`) when considering the veracity of several headlines. In the emotion reappraisal condition, participants read: "As you view and read the headlines, please try to adopt a detached and unemotional attitude. Please try to think about what you are reading objectively. Read all of the headlines carefully, but please try to think about what you are seeing in such a way that you feel less emotion." In the emotion suppression condition, they read: "As you view and read the headlines, if you have any feelings, please try your best not to let those feelings show. Read all of the headlines carefully, but try to behave so that someone watching you would not know that you are feeling anything at all." In studies 3 and 4 of the paper, the authors only tested the emotion suppression treatment.	Since we can only include one treatment per control condition in the meta-analysis, we remove participants in the emotion reappraisal condition in study 2. This allows us to compare across their studies.

Notes

The authors did not find evidence that emotion regulation helped people distinguish false from true news headlines, in any of the studies.

The authors sampled from a pool of 24 news items–a single participant only rated 16 items from this pool

“We used a pool of 24 items, taken from Pennycook and Rand (2019), half of which were real (true) and the other fake (false). Moreover, half of the items were Republican-consistent and the other half were Democrat-consistent items, based on a pre-test. Participants were presented with 16 randomly selected headlines altogether; 4 from each category (i.e. Republican-consistent fake, Republican-consistent real, Democrat-consistent fake, Democrat-consistent real).”

Reference:

Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50. https://doi.org/10.1016/j.cognition.2018.06.011

For study 4, they used different headlines, but again a pool of 24 with a random sample of 16 per participant.

“We used newer headlines taken from Pennycook et al., (2021a).”

Reference:

Pennycook, G., Binnendyk, J., Newton, C., & Rand, D. (2021a). A practical guide to doing behavioural research on fake news and misinformation Collabra: Psychology, 7(1), 25293. https://doi.org/10.1525/collabra.25293

Data Cleaning

Study 2

Read data.

Code

d2 <- read_csv("bago_2022-study_2.csv")

Rows: 16112 Columns: 15
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (3): reality, condition, concordancy
dbl (11): ID, accuracy, share, click, seen, Gender, libcons, DemRep_C, age, ...
lgl  (1): Gender_TEXT

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Code

head(d2)

# A tibble: 6 × 15
     ID reality accuracy share click  seen Gender Gender_TEXT libcons DemRep_C
  <dbl> <chr>      <dbl> <dbl> <dbl> <dbl>  <dbl> <lgl>         <dbl>    <dbl>
1     1 real           0     0     0     0      1 NA                1        3
2     1 fake           1     0     0     0      1 NA                1        3
3     1 real           1     0     0     0      1 NA                1        3
4     1 fake           1     0     0     0      1 NA                1        3
5     1 real           1     0     0     0      1 NA                1        3
6     1 real           1     0     0     1      1 NA                1        3
# ℹ 5 more variables: condition <chr>, age <dbl>, concordancy <chr>,
#   perceived_accu <dbl>, item <dbl>

`veracity`

Code

# check levels of accuracy 
table(d2$reality, useNA = "always")


fake real <NA> 
8056 8056    0

Code

d2 <- d2 |> 
  mutate(veracity = ifelse(reality == "fake", "false", "true"))

`accuracy_raw` and `scale`

Code

# check levels of accuracy 
table(d2$perceived_accu, useNA = "always")


   0    1 <NA> 
8374 7738    0

Code

d2 <- d2 |> 
  mutate(accuracy_raw = perceived_accu, 
         scale = "binary")

`news_id`, `recycled_news`, `recycled_news_reference`

Code

d2 |> 
  group_by(item) |> 
  count()

# A tibble: 24 × 2
# Groups:   item [24]
    item     n
   <dbl> <int>
 1     1   674
 2     2   673
 3     3   671
 4     4   671
 5     5   668
 6     6   671
 7     7   666
 8     8   673
 9     9   670
10    10   673
# ℹ 14 more rows

We rename the variable

Code

d2 <- d2 |> 
  mutate(news_id = item, 
         recycled_news = TRUE, 
         recycled_news_reference = "Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50. https://doi.org/10.1016/j.cognition.2018.06.011")

Concordance (`concordance`, `partisan_identity`, `news_slant`)

A concordance variable is already present.

Code

# check levels of concordance 
table(d2$concordancy, useNA = "always")


Concordant Discordant       <NA> 
      8008       8008         96

From the legend that the authors provide, we know that DemRep_C describes the partisan identity (“Which of the following best describes your political preference? 1: Strong Democrat 2: Democrat 3: Lean Democrat 4: Lean Republican 5: Republican 6: Strong Republican”). We will collapse this variable into a binary one.

We alse add a news slant variable.

Code

d2 <- d2 |> 
  mutate(concordance = tolower(concordancy),
         partisan_identity = ifelse(DemRep_C <= 3, "democrat", "republican"),
         news_slant = case_when(partisan_identity == "democrat" & 
                                  concordance == "concordant" ~ "democrat", 
                                partisan_identity == "republican" & 
                                  concordance == "concordant" ~ "republican", 
                                partisan_identity == "democrat" & 
                                  concordance == "discordant" ~ "republican", 
                                partisan_identity == "republican" & 
                                  concordance == "discordant" ~ "democrat", 
                                TRUE ~ NA_character_
                                )
         )

Conditions (`intervention_label`, `condition`)

Code

# check levels of condition 
levels(as.factor(d2$condition))

[1] "Control"     "Reappraisal" "Suppression"

Code

d2 <- d2 |> 
  # only assign a label to interventions conditions, not control conditions
  mutate(intervention_label = ifelse(condition == "Control", NA, tolower(condition)),
         condition = ifelse(condition == "Control", "control", "treatment")
         )

`age`

Code

# check levels of age
table(d2$age, useNA = "always") # 98 is likely code for NA


  19   20   22   23   24   25   26   27   28   29   30   31   32   33   34   35 
  80   80  128  304  240  544  416  480  544  640  704  784 1040  832  816  608 
  36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51 
 816  640  528  592  608  368  368  208  240  192  304  160  240  192  144  160 
  52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67 
 176  160  128  176  128   80  144  112  144  128  144   80   64   96  112   80 
  68   69   70   71   74   98 <NA> 
  16   32   32   48   16   16    0

Code

d2 <- d2 |> 
  mutate(age = ifelse(age == 98, NA, age))

`year`

There is no date variable, and in the paper, data collection date is not recorded for Study 2. However, it says that Study 3 has been conducted in September 2020. We therefor use 2020

Code

d2 <- d2 |> 
  mutate(year = 2020)

Identifiers (`subject_id`, `experiment_id`)

Code

d2 <- d2 |> 
  mutate(subject_id = ID, 
         experiment_id = 2)

Study 3

We proceed as for the previous study.

Code

d3 <- read_csv("bago_2022-study_3.csv")

Rows: 48016 Columns: 12
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (5): reality, Gender_TEXT, condition, consistent, concordancy
dbl (7): ID, Gender, libcons, DemRep_C, Age, perceived_accu, item

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Code

names(d3)

 [1] "ID"             "reality"        "Gender"         "Gender_TEXT"   
 [5] "libcons"        "DemRep_C"       "condition"      "Age"           
 [9] "consistent"     "concordancy"    "perceived_accu" "item"

Code

# check levels of condition variable
levels(as.factor(d3$condition))

[1] "Control"     "Suppression"

Code

# check levels of age
table(d3$Age, useNA = "always") # 90 seems old, but NA's are coded explictly as such


  19   20   22   23   24   25   26   27   28   29   30   31   32   33   34   35 
 368  336  464  672  624  608 1456  992 1184 1584 1744 2576 1872 2160 1632 1456 
  36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51 
1936 1536 1248 1488 1424 1456  944 1056 1040  976  832  800  736  704  880  784 
  52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67 
 624  640  704  560  768  560  496  528  480  448  640  336  480  384  304  496 
  68   69   70   71   72   73   74   75   76   77   78   79   80   82   90 <NA> 
 352  368  336  160  176   80  112   96   64   16   96   48   48   16   16   16

Code

d3 <- d3 |> 
  mutate(subject_id = ID, 
         news_id = item, 
         recycled_news = TRUE, 
         recycled_news_reference = "Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50. https://doi.org/10.1016/j.cognition.2018.06.011",
         veracity = ifelse(reality == "fake", "false", "true"),
         intervention_label = ifelse(condition == "Control", NA, tolower(condition)),
         condition = ifelse(condition == "Control", "control", "treatment"), 
         accuracy_raw = perceived_accu,
         scale = "binary",
         concordance = tolower(concordancy),
         partisan_identity = ifelse(DemRep_C <= 3, "democrat", "republican"),
         news_slant = case_when(partisan_identity == "democrat" & 
                                  concordance == "concordant" ~ "democrat", 
                                partisan_identity == "republican" & 
                                  concordance == "concordant" ~ "republican", 
                                partisan_identity == "democrat" & 
                                  concordance == "discordant" ~ "republican", 
                                partisan_identity == "republican" & 
                                  concordance == "discordant" ~ "democrat", 
                                TRUE ~ NA_character_
                                ),
         experiment_id = 3,
         age = Age, 
         year = 2020)

Study 4

We proceed as for the previous studies. Note, however, that we add different news source

Code

d4 <- read_csv("bago_2022-study_4.csv")

Rows: 48336 Columns: 14
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (6): reality, screener1, Gender_TEXT, condition, consistent, concordancy
dbl (8): ID, sceener2, Gender, libcons, DemRep_C, Age, perceived_accu, item

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Code

head(d4)

# A tibble: 6 × 14
     ID reality screener1 sceener2 Gender Gender_TEXT libcons DemRep_C condition
  <dbl> <chr>   <chr>        <dbl>  <dbl> <chr>         <dbl>    <dbl> <chr>    
1     1 fake    cat              1      1 <NA>              7        4 Suppress…
2     1 real    cat              1      1 <NA>              7        4 Suppress…
3     1 fake    cat              1      1 <NA>              7        4 Suppress…
4     1 real    cat              1      1 <NA>              7        4 Suppress…
5     1 real    cat              1      1 <NA>              7        4 Suppress…
6     1 fake    cat              1      1 <NA>              7        4 Suppress…
# ℹ 5 more variables: Age <dbl>, consistent <chr>, concordancy <chr>,
#   perceived_accu <dbl>, item <dbl>

Code

# check levels of condition variable
levels(as.factor(d4$condition))

[1] "Control"     "Suppression"

Code

# check levels of age
table(d4$Age, useNA = "always") # again, 90 and 98 seem old, but NA's are coded explictly as such


  19   20   22   23   24   25   26   27   28   29   30   31   32   33   34   35 
 288  416  336  528  640  768 1136 1232 1152 1440 1584 2336 1984 1872 1920 1760 
  36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51 
1712 1712 1520 1744 1280 1472  848  960  944  960  880  960  720  928  832  704 
  52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67 
 656  720  432  624  720  496  512  560  656  384  560  480  432  464  400  304 
  68   69   70   71   72   73   74   75   76   77   78   79   80   81   84   85 
 352  368  320  304  160  144  128   96   48   48   64   64   16   32   16   16 
  90   98 <NA> 
  16   16  160

Code

d4 <- d4 |> 
  mutate(subject_id = ID, 
          news_id = item, 
         recycled_news = TRUE, 
         recycled_news_reference = "Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50. https://doi.org/10.1016/j.cognition.2018.06.011",
         veracity = ifelse(reality == "fake", "false", "true"),
         intervention_label = ifelse(condition == "Control", NA, tolower(condition)),
         condition = ifelse(condition == "Control", "control", "treatment"), 
         accuracy_raw = perceived_accu,
         concordance = tolower(concordancy),
         partisan_identity = ifelse(DemRep_C <= 3, "democrat", "republican"),
         news_slant = case_when(partisan_identity == "democrat" & 
                                  concordance == "concordant" ~ "democrat", 
                                partisan_identity == "republican" & 
                                  concordance == "concordant" ~ "republican", 
                                partisan_identity == "democrat" & 
                                  concordance == "discordant" ~ "republican", 
                                partisan_identity == "republican" & 
                                  concordance == "discordant" ~ "democrat", 
                                TRUE ~ NA_character_
                                ),
         experiment_id = 4,
         age = Age, 
         # again, no direct info on year, we impute year of data collection reported for Study 3
         year = 2020)

Combine and add identifiers (`country`, `paper_id`) and `news_selection`

In study 2 and 3, the same news pool has been used. In study 4, a different pool has been used. We take this into account when merging the different studies.

Code

## Combine + add remaining variables
bago_2022 <- bind_rows(d2, d3, d4) |> 
  mutate(scale = "binary", 
         country = "United States",
         paper_id = "bago_2022", 
         # news id 
         news_id = ifelse(experiment_id %in% c(2,3), news_id, 
                          paste0(experiment_id, "_", news_id)
         ), 
         news_selection = "researchers", 
         ) |> 
# add_intervention_info 
  bind_cols(intervention_info) |> 
  select(any_of(target_variables))

# check
# bago_2022 |>
#   group_by(news_id) |>
#   summarize(n_observations = n()) |> 
#   arrange(as.numeric(news_id))

bago_2022 |> 
  group_by(experiment_id, condition, intervention_label) |> 
  count()

# A tibble: 7 × 4
# Groups:   experiment_id, condition, intervention_label [7]
  experiment_id condition intervention_label     n
          <dbl> <chr>     <chr>              <int>
1             2 control   <NA>                5408
2             2 treatment reappraisal         5408
3             2 treatment suppression         5296
4             3 control   <NA>               23984
5             3 treatment suppression        24032
6             4 control   <NA>               24103
7             4 treatment suppression        24233

Write out data

Code

save_data(bago_2022)

Reference

Intervention

Notes

Data Cleaning

Study 2

veracity

accuracy_raw and scale

news_id, recycled_news, recycled_news_reference

Concordance (concordance, partisan_identity, news_slant)

Conditions (intervention_label, condition)

age

year

Identifiers (subject_id, experiment_id)

Study 3

Study 4

Combine and add identifiers (country, paper_id) and news_selection

Write out data

`veracity`

`accuracy_raw` and `scale`

`news_id`, `recycled_news`, `recycled_news_reference`

Concordance (`concordance`, `partisan_identity`, `news_slant`)

Conditions (`intervention_label`, `condition`)

`age`

`year`

Identifiers (`subject_id`, `experiment_id`)

Combine and add identifiers (`country`, `paper_id`) and `news_selection`