Doing Open Science in a Developing Country – An Interview with Dr. Chuan-Peng Hu

Photo by courtesy of Dr. Hu

Open Science practices are becoming increasingly common and we at the Journal of European Psychology Students, are committed to Open Science practices and to promote researchers engaging in them.

Today, we have the privilege of interviewing one of these researchers. Dr. Chuan-Peng Hu is a postdoctoral researcher at the German Resilience Center (Deutsches Resilienz Zentrum, DRZ) in Mainz and an Assistant Director at the Psychological Science Accelerator (PSA). After studying Law and Psychology at University, he completed a Master’s programme in Social Psychology in Wuhan, China. In 2007, he completed his PhD in Beijing before moving to Germany. His research investigates the consolidation of positive memories, which may play a role in the resilience to stress.

Max: Thank you for taking the time to talk to us. Let us jump right in and start with the first question: Why is Open Science important to you? How do you implement and promote Open Science practices in your own work and at your institution?

Chuan-Peng: I think Open Science is important because it is an umbrella of solutions for the replication crisis in Psychology. My first experience with the replication crisis was my attempt to replicate the high-profile weight-embodiment study. The replication attempt failed, so I tried to contact the authors, who did not reply. That is when I first felt that something was wrong. Soon after, Bem published his famous precognition study, the replication of the elderly priming study failed and the scandal surrounding Diederik Stapel unfolded. All these events and subsequent online discussions made me think that our research culture is deeply flawed. Simultaneously, the events marked the birth of Open Science. Many researchers began to organize the “Many Labs” replications and started to promote pre-registration. Personally, I was convinced that being transparent and open is the way to alleviate the problems in psychology and science more broadly.

Admittedly, appreciating the idea of Open Science was considerably easier than implementing Open Science practices into my own research. Firstly, being open takes time. I taught myself R and spend half a year re-analysing my previous experiments. I learned how to use Matlab instead of E-prime. Secondly, I began to take statistical power seriously. Many experiments I conducted during my PhD used the old-fashioned way to determine sample size: Following previous studies, with generally low sample sizes. To improve on that, I followed suggestions from the “life after p-hacking” paper and increased the sample size of my subsequent studies. Thirdly, I started to pre-register my experiments, but I have to say that I could have done my pre-registrations much better. I also participated in projects like the pre-registration challenge, even though I failed to publish it on time. Finally, I started to participate in large-scale collaborations, such as the Human Penguin Project , the Kama Muta Project, and the PSA. I am particularly proud that we shared all our materials (questionnaires translated into nine languages) and data of the Human Penguin Project and published the data descriptor in Scientific Data. Nowadays, all my projects are built on Open Science principles. For example, one of my on-going projects is a direct replication study.

Overall, implementing open science practice did slow me down, but I think doing science in the right way is more important than finishing fast. Though the experiments during my Ph.D. are still not published, many of them are open now, two of these experiments were preprinted, and many more were open as Open Notebook.

I have also been active promoting Open Science in my institutes. For example, when I was at Tsinghua University, Beijing, I have given a few small-scale talks to promote the idea of Open Science. And one professor started to pay attention to the problem and later included replication crisis in his methodological course for graduates. Here in Mainz, I introduced the flexibility in fMRI data analysis and its consequence in our SPM course, and I have initiated a small group discussion on p-hacking during the PhD/PostDoc retreat of our program. However, maybe because I am a bit socially awkward, I feel more comfortable to promote Open Science through the internet. For example, I wrote Chinese blogs and organized journal club outside my institute when I was in Beijing, and engaged in discussions on Twitter.

Max: You are very active in promoting Open Science in China. What is the state of Open Science in China, and how does its implementation differ between China and western countries like Germany? Did you encounter any challenges in China you did not encounter in Germany, or vice versa?

Chuan-Peng: I started to promote Open Science in China around 2015. At that time, very few people were talking about Open Science and reproducibility there. For example, at the major psychological conferences in China there was not a single symposium about Open Science or reproducibility. So, I decided to take the risk and speak up. I teamed up with a few colleagues in my institute and together we wrote a comprehensive Chinese introduction about replication crisis and related methodological reforms. During the publication process, reviewers told us our tone was too negative, but luckily, we got it published eventually. Later, some grads told me that they really found this paper helpful and took it to their journal club. I also co-founded the Chinese Open Science Network.

As compared to the psychology community outside China, the situation in the mainland changes very slowly. Most psychology graduates are still unfamiliar with concepts such as direct replication, preregistration, open materials or open data. Few people talk about statistical power openly, and there is little support for researchers trying to adopt new practices. Also, very few Chinese research teams participate in international large-scale collaborations (e.g., only around ten teams from mainland China are part of the PSA).

The positive side is that things are changing in China. There are some mid-career researchers who strongly support Open Science. For example, three researchers from the Institute of Psychology, Chinese Academy of Sciences, are engaged in efforts to promote Open Science practices: Dr. Xi-Nian Zuo did a lot of work on the reliability of fMRI and created an open database for assessing reliability; Dr. Chao-Gan Yan organized a large-scale collaboration study on the brain structure of depressed individuals, and Dr. Huajian Cai is leading a large-scale project surveying the socio-cultural traits of Chinese people, probably the largest social psychology study ever conducted in China. In Hong Kong, Gilad Feldman is organizing a massive replication project. In Taiwan, Sau-chin Chen is leading a project of the PSA.

Beyond that, young Chinese grads started to develop toolboxes to facilitate the use of R (e.g., the bruceR). A preprint service, Chinaxiv, is curating thousands of preprints from many different fields. Even more encouraging is that, in 2018, the best Chinese psychological science journal, Acta Psychologica Sinica, started to encourage pre-registration, and now requires sample size justification. Given its prestige in China, these changes may have a strong impact.

There are many challenges for Open Science in China. Many of these challenges are rooted in the history of Psychological Science or Science in China. I will name a few. First, the psychological science community in China is relatively small and segregated from other fields (neuroscience, sociology, etc.). This status has historical reasons: the development of modern psychological science in China started only about forty years ago, so psychology as a discipline is relatively young. A small community, combined with our hierarchical social norms, makes changing the professional culture particularly hard. Very few researchers openly criticize others. Another challenge stems from the lack of communication between Chinese researchers and their international peers. For example, inadequate training in the English language or the Great Firewall hinder adaptation of Open Science practices.

In comparison, I think Open Science in Germany is growing fast. There is a large group of early career researchers (e.g., Julia M. Rohrer, Felix D. Schönbrodt) and mid-career researchers (e.g., Simon Eickhoff) who are doing great work in sharing data, developing tools/methods, and doing meta-research. In the last few years, many Open Science initiatives were founded in Germany — it is very encouraging.

Max: You mentioned that you co-founded the Chinese Open Science Network. Tell me a bit about your initiative.

Chuan-Peng: The Chinese Open Science Network is a loosely organized grassroots network, which I initiated online. The main platforms of our network include a WeChat account (Chinese Facebook) and a public project on the Open Science Framework. The WeChat account is organized like a blog. Users can also subscribe to us and receive updates on our work. WeChat has a very large user base in China, so it is an important source of information. Our account currently has 6500 subscribers and we regularly organize online discussions or ask for volunteers for Open Science related activities (e.g., translating tutorials or blogs). The Open Science Framework (OSF) project is where we store all materials related to Open Science. It includes the PowerPoint presentations of my talks, an online journal club, and recordings of lectures. We even have podcasts on there.

Through the Chinese Open Science Network, I could find people who are interested in Open Science and invite them to promote Open Science with me. For example, I recruited a few collaborators and finished a Chinese tutorial on how to calculate the confidence intervals of effect sizes. Now, we are preparing a tutorial on how to better interpret non-significant results. Also, we started some meta-research on Chinese researchers and Chinese papers such as the missunderstanding of p values in psychology and other fields . Moreover, I recruited Chinese colleagues from the Chinese Open Science Network to translate and proofread the Seven Deadly Sins of Psychology by Chris Chambers into Chinese.

Max: What advice would you give scientists trying to start an Open Science Initiative in their country?

Chuan-Peng: My suggestion is to push the limits a bit to do what you can do in your own culture. Promoting Open Science seems to be unpopular because you are against the old ways of doing science, senior researchers may be hostile to you. When we wrote the introduction to the replication crisis and methodological reform in Chinese, some colleagues suggested that we stop, because it could be risky for our careers. We feared that it might offend some senior researchers, and even though it attracted criticism and negative comments, it eventually got published. The lesson I learned from that is that you never know how many people share your feelings and ideals, until you tried it. Finally, I suggest researchers to use the internet well and keep in touch with the global Open Science community. If few people at your institute are interested in Open Science and you do not connect with the larger community, you may feel isolated and lonely, which makes your efforts not sustainable.

Max: You are an Assistant Director with the PSA. Can you tell me something about your work there?

Chuan-Peng: Yes, I am an Assistant Director of  the Translation and Cultural Diversity Committee. In this committee, we are responsible for two things: coordinating the translation of materials of PSA projects and increasing the (cultural, geographical, SES, etc.) diversity of PSA. For example, in coordinating the translation of PSA 006, my role is to contact teams from different languages and make sure that they know our standard translation process. Then, I will track the progress of translation for each language. To improve the quality of the translations, we are planning to draft a guideline about best practices for the translation process.

 Max: What advice would you give to aspiring scientists trying to implement Open Science practices in their work?

Chuan-Peng: “The first principle is that you must not fool yourself – and you are the easiest person to fool.” is a quote by Richard Feynman which serves as both inspiration and warning to me.

Max: Any Final Remarks?

Chuan-Peng: Thanks for having me.

Max: Thank you very much for taking the time.

Maximilian Primbs

Maximilian is a Research Master´s student in Behavioural Science at Radboud University and a Research Assistant at the Behavioural Science Institute. He´s interested in prejudice, stereotypes, faces, and research methodology. In his free time, he enjoys contact sports and metal music.

More Posts

Follow Me:
Twitter

Facebooktwitterrss

Exploratory and Confirmatory Hypothesis Testing

Introduction

The replication crisis has spread all across the scientific community. In the field of psychology, scientists were not able to replicate more than half of previous findings (Open Science Collaboration, 2015). For a long time this problem went unnoticed, but a critical moment occurred when Daryl Bem published his now infamous paper on humans’ ability to quite literally predict the future (Bem, 2011). Many readers doubted his findings as there was no logical basis for the ability to predict the future and years later Daniel Engber summarized it nicely when he wrote:

(…) the paper posed a very difficult dilemma. It was both methodologically sound and logically insane. (…). If you bought into those results, you’d be admitting that much of what you understood about the universe was wrong. If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk.“ (Engber, 2017)

Continue reading

Patrick Smela

Patrick is currently finishing his Bachelor in Psychology at the University Vienna. Afterwards, he will to do his Masters in General Psychology and Methodology. He is passionate about research methods, especially in the field Human Computer Interaction. Besides work, he likes to travel, read, and does a lot of voluntary work in the psychological faculty in Vienna.

More Posts

Facebooktwitterrss

Editors’ Picks: Summer Reading List

Your semester has ended and you are already bored by how much time the holidays freed up?
Do you want to dive deeper into issues around psychological science, but did not know where to start?

For the next weeks, we are going to be sharing our JEPS editors’ recommendations for your summer readings & listenings on different psychological topics. These will include all sorts of media, from newspaper articles or podcasts to journal articles we thought you should definitely read.


Continue reading

Leonhard Volz

Leonhard Volz

Leonhard currently is in his bachelor's studies in psychology and in statistics at the University of Vienna and a student assistant at the Educational Psychology department. His main areas of interest are research methodology and knowledge transfer in interdisciplinary psychological research - under the banner of Open Science principles. His personal happy moments are when he finds the time to open up a novel again.

More Posts

Follow Me:
Twitter

Facebooktwitterrss

Open Science Bottom-Up: An Interview with OSIP (PsyFaKo’s Open Science Initiative)

For us Editors of JEPS, one of the most important topics in current psychological science and beyond are the issues of replicability and reproducibility [for an introduction, see Galetzka, 2019], as well as possible paths to solutions.

The keyword here is Open Science, an umbrella term for activities which strive to make science more transparent, openly accessible, and reproducible, in an effort to increase our confidence in the results we read in the body of scientific literature.

While many Open Science initiatives are led by more senior researchers, the movement is fundamentally driven by bottom-up initiatives of early-career researchers, but students as well.

For this interview, we sat down and got together with one of these student-led initiatives: The PsyFaKo’s Open Science Initiative [OSIP, Open Science Initiative der PsyFaKo e.V., webpage in German], a working group in the German Convention of Student Councils of Psychology. They made headlines in the landscape of German psychology last year when they released a position paper on the Replication Crisis and Open Science, which had a considerable impact at German universities. Continue reading

Leonhard Volz

Leonhard Volz

Leonhard currently is in his bachelor's studies in psychology and in statistics at the University of Vienna and a student assistant at the Educational Psychology department. His main areas of interest are research methodology and knowledge transfer in interdisciplinary psychological research - under the banner of Open Science principles. His personal happy moments are when he finds the time to open up a novel again.

More Posts

Follow Me:
Twitter

Facebooktwitterrss

7 Easy Steps to Building your Own Shiny App from Scratch

Shiny is a powerful tool in R for you to show off your work to the world, without explaining all the complicated code behind your analysis. Because of its free and open-source development and deployment structure, sharing your methods or work online was never easier. For example, in our recent publication in the Journal of European Psychology Students my colleagues and me used Shiny to implement a network method in which we used the concept of network centrality to determine the most relevant articles in a research field. Because I believe there are a lot of benefits in sharing one’s methods, my hopes are that this blog post has the possibility to also inspire you to share your own work through Shiny. I will walk you though developing your own R Shiny application from scratch, tailoring it to your design choices, and publishing it online, in 7 easy steps. Continue reading

Koen Derks

Koen Derks is a PhD student at Nyenrode Business University. Before he started researching Bayesian statistics in financial auditing, he studied psychological methods at the University of Amsterdam (UvA), where his interests for statistics and programming developed itself. Today, he is still collaborating with the UvA to develop software tools for students and practitioners to make data analysis fun and intuitive.

More Posts

Facebooktwitterrss

Publishing the results of coursework research: An interview with Julian Burger and Koen Derks

submit-you-mustBeing an undergrad is hard. With the days spent in lecture rooms and the nights devoted to catching up with essays and assignments, one wonders how is it even possible for undergrads to do any research – let alone publish it. While there is no expectation from undergrads to publish, a rough (and very anecdotal) approximation is that around 1 in 100 students publish during their undergraduate studies in either a peer-reviewed journal or other online outlets. (However, this highly depends on the field and publishing culture of the affiliated institution). There are also many benefits to publishing as undergrad; as illustrated by Griffith (2001), an early publication – regardless of the importance of the findings or prominence of the outlet – can increase student’s confidence and inspire a prolific academic career in the future. So how do these acclaimed one-in-a-hundred undergrads manage to publish amid challenges of the student life? Continue reading

Karla Matić

Karla Matić is a psychology graduate of University of Leuven with interests in cognitive neuroscience, large-scale neuroimaging methodology, and science policy. She is currently an intern in the European Research Council (ERC) in Brussels. If she didn't aspire for an academic career, she would be running a book-café on a small Croatian island.

More Posts

Follow Me:
Twitter

Facebooktwitterrss

Between science and policy: an interview with Dr Toby Wardman

Even though scientists are oftentimes lost in the ivory towers of their scientific work, academic research in any discipline – and especially psychology – is tightly connected to the society. It contributes to the improvement of the living conditions in the population. It supports the decision-making process of policy-makers with scientific evidence. And it is paid for by the tax-payers’ money. In an attempt to ensure that this natural relationship between science and society is always well-balanced, we make policies – governmental policies, international policies, institutional policies. The field at the interplay between science and policy-making – very intuitively coined ‘science policy’ – therefore concerns itself with topics such as the allocation of resources for scientific research, the careers of scientists, and the systems of efficient communication between scientists and policy-makers (Pielke, 2005). Continue reading

Karla Matić

Karla Matić is a psychology graduate of University of Leuven with interests in cognitive neuroscience, large-scale neuroimaging methodology, and science policy. She is currently an intern in the European Research Council (ERC) in Brussels. If she didn't aspire for an academic career, she would be running a book-café on a small Croatian island.

More Posts

Follow Me:
Twitter

Facebooktwitterrss

Publishing a Registered Report as an Undergraduate: An Interview with Tatiana Kvetnaya

In the past, we have talked a lot about Registered Reports and their potential to increase the rigor and reproducibility of psychological science (see here, here, and here). In a previous blog post, James Bartlett interviewed Dr. Hannah Hobson, who published a Registered Report as part of her PhD project.

In this blog post, we talk with Tatiana Kvetnaya who received her Bachelor degree from the University of Tübingen, and who is currently pursuing her graduate studies at the Goethe University Frankfurt. Excitingly, Tatiana recently published her bachelor thesis as a Registered Report with the Journal of European Psychology Students. Below, she recounts how she first came in contact with Registered Reports, her experience publishing one herself, and tips for students thinking about doing the same. Continue reading

Fabian Dablander

Fabian Dablander is doing a PhD at the Department of Psychological Methods at the University of Amsterdam. You can find more information at https://fdabl.github.io/.

More Posts - Website

Follow Me:
Twitter

Facebooktwitterrss

Technology-assisted Therapy: An Interview with “Aaron T. Beck” Professor Daniel David

The technological developments we see today set a whole new view of life as we know it. Starting with the Industrial Revolution, and getting to robot assisted mass production of goods, we get to use intelligent machines in order to make life easier and evolve as a species. And psychology is not an exception. Ever since ELIZA was developed to simulate a psychotherapist in the ‘60s (try it for yourself here) computers have been widely used within clinical psychology and psychotherapy. Today, we will be talking about the efforts of the Clinical Psychology and Psychotherapy School of “Babeș-Bolyai” University of Cluj-Napoca, Romania in pursuing Virtual Reality (VR) and Artificial Intelligence (AI) research and practice excellence.

Continue reading

Ioana Piscoi

Currently a first year Master Student of Clinical Psychology, Counselling and Psychotherapy at "Babes-Bolyai" University of Cluj-Napoca, Romania, Ioana is interested in developing her skills to become an ACT Therapist. She is particularly interested in the field of Personality Disorders and Techlonogy Assisted Psychotherapies, wishing to pursue a PhD in the future.

More Posts

Follow Me:
LinkedIn

Facebooktwitterrss

Writing a Systematic Literature Review

Investigating concepts associated with psychology requires an indefinite amount of reading. Hence, good literature reviews are an inevitably needed part of providing the modern scientists with a broad spectrum of knowledge. In order to help, this blog post will introduce you to the basics of literature reviews and explain a specific methodological approach towards writing one, known as the systematic literature review. Continue reading

Eva Štrukelj

Eva Štrukelj

Eva Štrukelj is currently studying Clinical and Health Psychology at the University of Algarve in Portugal. Her main areas of interest are social psychology and health psychology. Regarding research, she is particularly curious about stigma and with it related topics.

More Posts

Facebooktwitterrss