Automated fact-checking under human surveillance: AI implementation in the case of Newtral

Authors

DOI:

https://doi.org/10.24137/raeic.11.e.3

Keywords:

artificial intelligence, journalism, Newtral, fact-checking, case study

Abstract

Artificial Intelligence (AI), which has been progressively introduced into newsrooms since the early years of the 2010s, can serve as a potent tool to combat disinformation in the current scenario of a crisis of trust in traditional journalism and the rampant spread of hoaxes. This research paper delves into the AI tools that have been implemented in Newtral, a leading verification media outlet in Spain, with the aim to streamline the workflow in the newsroom and expedite the process of publishing disinformation. Through semi-structured interviews conducted with four professionals (comprising three from the technology team and one editor), it is discovered that AI projects significantly facilitate the work of journalists and foster positive discussions and synergies between engineers and information professionals. AI is employed to detect relevant topics and verifiable facts, transcribe audio content to text, and monitor social media posts. To circumvent algorithmic errors, human-in-the-loop techniques are employed to ensure that no content that has been manipulated by AI is published without undergoing human review.

Downloads

Download data is not yet available.

Author Biographies

  • Borja Quiles Morán, Universidad Miguel Hernández (UMH)

    Borja Quiles Morán is a predoctoral researcher in journalism at the University Miguel Hernández. He has a degree in Journalism from the same university (2022) and a Master in New Journalism, Political Communication and Knowledge Society from the University of Valencia (2023). He has participated in the research projects “JoIn Demos”, an international comparative study on innovation in journalism in five European countries (Spain, Germany, Austria, United Kingdom and Switzerland), and in the project “IBERIFIER”, an observatory of digital media in Spain and Portugal. In the academic year 2021-2022 he received a collaboration grant from the Ministry of Education to carry out research tasks on the use of Artificial Intelligence. He has received the Extraordinary Undergraduate and Master's Degree Award and an Academic Excellence Student Award (academic year 2018-2019). His research focuses are computational journalism and AI, disinformation and fact-checking.

  • Félix Arias Robles, Universidad Miguel Hernández (UMH)

    Félix Arias Robles is a Professor, Vice Dean of the Degree in Journalism, and Deputy Director of the Master's in Innovation in Journalism at Miguel Hernández University (UMH), where he coordinates courses in Technology, New Narratives and Data Journalism. His current research focuses on the intersection of artificial intelligence, data journalism, and local information. He has published in journals such as Journalism Studies, Journalism Practice, Communication & Society, El Profesional de la Información, and The Journal of Media Innovations. He has been a visiting researcher at JournalismAI, a think tank at the London School of Economics (LSE), the School of Journalism at the University of Sheffield, and the University of Leeds (United Kingdom). He has also trained journalists in newsrooms such as La Voz de Galicia, Vocento, and Onda Regional. Additionally, he coordinates the projects periodistasdedatos.com and localdatalab.umh.es.

References

Aguado-Guadalupe, G., & Bernaola-Serrano, I. (2020). Verificación en la infodemia de la Covid-19. El caso Newtral. Revista Latina de Comunicación Social, 78, 289-308. https://doi.org/10.4185/RLCS-2020-1478

Almansa-Martínez, A., Fernández-Torres, M. J., & Rodríguez-Fernández, L. (2022). Desinformación en España un año después de la COVID-19. Análisis de las verificaciones de Newtral y Maldita. Revista Latina de Comunicación Social, 80, 183-200. https://doi.org/10.4185/RLCS-2022-1538

Ananny, M. (2016). Toward an Ethics of Algorithms: Convening, Observation, Probability, and Timeliness. Science Technology and Human Values, 41(1), 93-117. https://doi.org/10.1177/0162243915606523

Beckett, C., & Yaseen, M. (2023). Generating Change. A global survey of what news organisations are doing with AI. https://cutt.ly/ow0ZgJ4I

Beltrán, J., Míguez, R., & Larraz, I. (2021). ClaimHunter: An unattended tool for automated claim detection on Twitter. KnOD’21 Workshop. https://cutt.ly/6eskkZjQ

Bennett, W. L., & Livingston, S. (2018). The disinformation order: Disruptive communication and the decline of democratic institutions. European Journal of Communication, 33(2), 122-139. https://doi.org/10.1177/0267323118760317

Boira, P., Pita, C., & Larraz, I. (2022, enero 25). Las calles de ellas. Newtral. https://cutt.ly/Jw0Zg5mX

Broussard, M., Diakopoulos, N., Guzman, A. L., Abebe, R., Dupagne, M., & Chuan, C. H. (2019). Artificial Intelligence and Journalism. Journalism and Mass Communication Quarterly, 96(3), 673-695. https://doi.org/10.1177/1077699019859901

Calvo-Rubio, L. M., & Ufarte-Ruiz, M. J. (2021). Artificial intelligence and journalism: Systematic review of scientific production in web of science and scopus (2008-2019). Communication and Society, 34(2), 159-176. https://doi.org/10.15581/003.34.2.159-176

Cameron, K. S., & Quinn, R. E. (2011). Diagnosing and Changing Organizational Culture: Based on the Competing Values Framework (3rd ed.). Wiley.

Canavilhas, J. (2022). Artificial intelligence in journalism: Automatic translation and recommendation system in the project “A European Perspective” (EBU). Revista Latina de Comunicacion Social, 80, 1-13. https://doi.org/10.4185/RLCS-2022-1534

Carvajal, M., Mondéjar, D., Valero-Pastor, J. M., De-Lara, A., García-Avilés, J. A., & Arias-Robles, F. (2022). The most prominent journalistic innovations in Spain (2010-2020): organizational, industrial and social characteristics and impact. Profesional de la Informacion, 31(3). https://doi.org/10.3145/epi.2022.may.04

Cazalens, S., Leblay, J., Lamarre, P., Manolescu, I., & Tannier, X. (2018). Computational fact checking. Proceedings of the VLDB Endowment, 11(12), 2110-2113. https://doi.org/10.14778/3229863.3229880

Christin, A. (2017). Algorithms in practice: Comparing web journalism and criminal justice. Big Data and Society, 4(2). https://doi.org/10.1177/2053951717718855

Cobisa, I. (2022). Inteligencia Artificial como palanca de generación de valor. Madurez del mercado en España. https://cutt.ly/Ow0Zhlpn

de Oliveira, N. R., Pisa, P. S., Lopez, M. A., de Medeiros, D. S. V., & Mattos, D. M. F. (2021). Identifying Fake News on Social Networks Based on Natural Language Processing: Trends and Challenges. Information, 12(1), 38. https://doi.org/10.3390/info12010038

Demartini, G., Mizzaro, S., & Spina, D. (2020). Human-in-the-loop Artificial Intelligence for Fighting Online Misinformation: Challenges and Opportunities. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 43(3), 65-74. https://cutt.ly/Besej4MD

Diakopoulos, N. (2015). Algorithmic Accountability: Journalistic investigation of computational power structures. Digital Journalism, 3(3), 398-415. https://doi.org/10.1080/21670811.2014.976411

Dörr, K. N., & Hollnbuchner, K. (2017). Ethical Challenges of Algorithmic Journalism. Digital Journalism, 5(4), 404-419. https://doi.org/10.1080/21670811.2016.1167612

Fernandes, E., Moro, S., & Cortez, P. (2023). Data Science, Machine learning and big data in Digital Journalism: A survey of state-of-the-art, challenges and opportunities. Expert Systems with Applications, 221, 119795. https://doi.org/10.1016/j.eswa.2023.119795

García-Vivero, G., & López, X. (2021). La verificación de datos en Europa. Análisis de 5 iniciativas europeas: Maldita.es, Newtral, Pagella Politica, Les Décodeurs y BBC Reality Check. adComunica, 21, 235-264. https://doi.org/10.6035/2174-0992.2021.21.12

García-Avilés, J. A., Arias-Robles, F., de Lara-González, A., Carvajal, M., Valero-Pastor, J. M., & Mondéjar, D. (2022). How COVID-19 is Revamping Journalism: Newsroom Practices and Innovations in a Crisis Context. Journalism Practice. https://doi.org/10.1080/17512786.2022.2139744

García-Avilés, J. A., Carvajal-Prieto, M., Arias, F., & De Lara-González, A. (2018). How journalists innovate in the newsroom. Proposing a model of the diffusion of innovations in media outlets. The Journal of Media Innovations, 5(1), 1-16. https://doi.org/10.5617/jomi.v5i1.3968

García-Marín, D. (2022). Modelos algorítmicos y fact-checking automatizado. Revisión sistemática de la literatura. Documentación de las Ciencias de la Información, 45(1), 7-16. https://doi.org/10.5209/dcin.77472

García-Orosa, B., Canavilhas, J., & Vázquez-Herrero, J. (2023). Algorithms and communication: A systematized literature review. Comunicar, 30(74), 9-21. https://doi.org/10.3916/C74-2023-01

Goodman, L. A. (1961). Snowball Sampling. The Annals of Mathematical Statistics, 32(1), 148-170. https://doi.org/10.1214/aoms/1177705148

Graves, L. (2018). Understanding the Promise and Limits of Automated Fact-Checking. https://cutt.ly/Hw0ZlQRV

Haim, M., & Graefe, A. (2017). Automated News. Digital Journalism, 5(8), 1044-1059. https://doi.org/10.1080/21670811.2017.1345643

Hassan, N., Adair, B., Hamilton, J. T., Li, C., Tremayne, M., Yang, J., & Yu, C. (2015). The Quest to Automate Fact-Checking. https://cutt.ly/3w0ZlJVj

Johnson, P. R. (2023). A Case of Claims and Facts: Automated Fact-Checking the Future of Journalism’s Authority. Digital Journalism. https://doi.org/10.1080/21670811.2023.2174564

Kim, D., & Kim, S. (2017). Newspaper companies’ determinants in adopting robot journalism. Technological Forecasting and Social Change, 117, 184-195. https://doi.org/10.1016/j.techfore.2016.12.002

Larraz, I., Míguez, R., & Sallicati, F. (2023). Semantic similarity models for automated fact-checking: ClaimCheck as a claim matching tool. Profesional de la información, 32(3), e320321. https://doi.org/10.3145/epi.2023.may.21

Lewis, S. C., Sanders, A. K., & Carmody, C. (2019). Libel by Algorithm? Automated Journalism and the Threat of Legal Liability. Journalism & Mass Communication Quarterly, 96(1), 60-81. https://doi.org/10.1177/1077699018755983

Lewis, S. C., & Zamith, R. (2017). On the Worlds of Journalism. En Remaking the News (pp. 111-128). The MIT Press. https://doi.org/10.7551/mitpress/10648.003.0012

López-García, X., & Vizoso, Á. (2021). High-tech journalism: A sign of the digital era of the third millennium. Profesional de la Informacion, 30(3), e300301. https://doi.org/10.3145/epi.2021.may.01

López-Marcos, C., & Vicente-Fernández, P. (2021). Fact Checkers Facing Fake News and Disinformation in the Digital Age: A Comparative Analysis between Spain and United Kingdom. Publications, 9(3), 36. https://doi.org/10.3390/publications9030036

Magallón-Rosa, R., Fernández-Castrillo, C., & Garriga, M. (2023). Fact-checking in war: Types of hoaxes and trends from a year of disinformation in the Russo-Ukrainian war. El Profesional de la información, 32(5), e320520. https://doi.org/10.3145/epi.2023.sep.20

Maldita.es lanza el primer chatbot de Whatsapp en español para verificar bulos automáticamente. (2020, junio 17). Maldita.es. https://cutt.ly/gw0Zl9fU

Maslej, N., Fattorini, L., Kiesow Cortez, E., Ngo, H., & Rahman, R. (2023). Artificial Intelligence Index Report 2023. https://cutt.ly/hw0ZzaMN

Meléndez-Malavé, N., & Pérez-Quintana, P. (2023). La verificación de bulos relacionados con el feminismo: Un análisis de los desmentidos sobre género publicados en Maldita.es y Newtral. Signo y Pensamiento, 42. https://doi.org/10.11144/Javeriana.syp42.vbrf

Mondría-Terol, T. (2023). Innovación MedIÁtica: aplicaciones de la inteligencia artificial en el periodismo en España. Textual & Visual Media, 17(1), 41-60. https://doi.org/10.56418/txt.17.1.2023.3

Oremus, W. (2014, marzo 17). The First News Report on the L.A. Earthquake Was Written by a Robot. Slate. https://cutt.ly/dw0Zzn7i

Parratt-Fernández, S., Mayoral-Sánchez, J., & Mera-Fernández, M. (2021). The application of artificial intelligence to journalism: An analysis of academic production. Profesional de la Informacion, 30(3). https://doi.org/10.3145/epi.2021.may.17

Pérez-Escoda, A., Pedrero-Esteban, L. M., Rubio-Romero, J., & Jiménez-Narros, C. (2021). Fake news reaching young people on social networks: Distrust challenging media literacy. Publications, 9(2). https://doi.org/10.3390/publications9020024

Porcu, O., Hermans, L., & Broersma, M. (2020). Unlocking the Newsroom: Measuring Journalists’ Perceptions of Innovative Learning Culture. Journalism Studies, 21(10), 1420-1438. https://doi.org/10.1080/1461670X.2020.1758956

Rao, A., & Verweij, G. (2022). Sizing the prize. PwC’s Global Artificial Intelligence Study: Exploiting the AI Revolution. https://cutt.ly/dw0ZzIRn

Rojas-Caja, F. (2020). El fact checking. Las agencias de verificación de noticias en España. https://cutt.ly/0w0ZP7JS

Rojas-Torrijos, J. L., & Toural-Bran, C. (2019). Automated sports journalism. The AnaFut case study, the bot developed by El Confidencial for writing football match reports. Doxa Comunicacion, 2019(29), 235-254. https://doi.org/10.31921/doxacom.n29a12

Sánchez-González, M., Sánchez-Gonzales, H. M., & Martínez-Gonzalo, S. (2022). Inteligencia artificial en verificadores hispanos de la red IFCN: proyectos innovadores y percepción de expertos y profesionales. Estudios sobre el Mensaje Periodístico, 28(4), 867-879. https://doi.org/10.5209/esmp.82735

Simon, F. M. (2022). Uneasy Bedfellows: AI in the News, Platform Companies and the Issue of Journalistic Autonomy. Digital Journalism, 10(10), 1832-1854. https://doi.org/10.1080/21670811.2022.2063150

Stray, J. (2019). Making Artificial Intelligence Work for Investigative Journalism. Digital Journalism, 7(8), 1076-1097. https://doi.org/10.1080/21670811.2019.1630289

Thurman, N., Dörr, K., & Kunert, J. (2017). When Reporters Get Hands-on with Robo-Writing: Professionals consider automated journalism’s capabilities and consequences. Digital Journalism, 5(10), 1240-1259. https://doi.org/10.1080/21670811.2017.1289819

Thurman, N., Lewis, S. C., & Kunert, J. (2019). Algorithms, Automation, and News. Digital Journalism, 7(8), 980-992. https://doi.org/10.1080/21670811.2019.1685395

Túñez-López, J. M., & Tejedor-Calvo, S. (2019). Artificial intelligence (AI) and journalism [monograph presentation]. Doxa Comunicacion, 2019(29), 163-168. https://doi.org/10.31921/doxacom.n29a8

Túñez-López, J.-M., Fieiras-Ceide, C., & Vaz-Álvarez, M. (2021). Impact of Artificial Intelligence on Journalism: transformations in the company, products, contents and professional profile. Communication & Society, 34(1), 177-193. https://doi.org/10.15581/003.34.1.177-193

Ufarte-Ruiz, M. J., & Manfredi-Sánchez, J. L. (2019). Algorithms and bots applied to journalism. The case of Narrativa Inteligencia Artificial: structure, production and informative quality. Doxa Comunicación. Revista interdisciplinar de estudios de comunicación y ciencias sociales, 29, 213-233. https://doi.org/10.31921/doxacom.n29a11

Valles, M. S. (2002). Entrevistas cualitativas. Centro de Investigaciones Sociológicas.

van Dalen, A. (2012). The algorithms behind the headlines: How machine-written news redefines the core skills of human journalists. Journalism Practice, 6(5-6), 648-658. https://doi.org/10.1080/17512786.2012.667268

Vázquez-Herrero, J., Vizoso, Á., & López-García, X. (2019). Innovación tecnológica y comunicativa para combatir la desinformación: 135 experiencias para un cambio de rumbo. El Profesional de la Información, 28(3). https://doi.org/10.3145/epi.2019.may.01

Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151. https://doi.org/10.1126/science.aap9559

Waisbord, S. (2018). Truth is What Happens to News: On journalism, fake news, and post-truth. Journalism Studies, 19(13), 1866-1878. https://doi.org/10.1080/1461670X.2018.1492881

Wu, S., Tandoc, E. C., & Salmon, C. T. (2019). Journalism Reconfigured. Journalism Studies, 20(10), 1440-1457. https://doi.org/10.1080/1461670X.2018.1521299

Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2021). A Survey of Human-in-the-loop for Machine Learning. https://doi.org/10.1016/j.future.2022.05.014

Zarocostas, J. (2020). How to fight an infodemic. The Lancet, 395(10225), 676. https://doi.org/10.1016/S0140-6736(20)30461-X

Zheng, Y., Zhong, B., & Yang, F. (2018). When algorithms meet journalism: The user perception to automated news in a cross-cultural context. Computers in Human Behavior, 86, 266-275. https://doi.org/10.1016/j.chb.2018.04.046

Downloads

Published

2024-07-31

Issue

Vol. 11 No. Especial (2024): Artificial intelligence and the democratisation of audiovisual creation

Section

Thematic issue

How to Cite

Automated fact-checking under human surveillance: AI implementation in the case of Newtral. (2024). Revista De La Asociación Española De Investigación De La Comunicación, 11(Especial), raeic11e03. https://doi.org/10.24137/raeic.11.e.3