Photo by courtesy of Dr. Hu
Open Science practices are becoming increasingly common and we at the Journal of European Psychology Students, are committed to Open Science practices and to promote researchers engaging in them.
Today, we have the privilege of interviewing one of these researchers. Dr. Chuan-Peng Hu is a postdoctoral researcher at the German Resilience Center (Deutsches Resilienz Zentrum, DRZ) in Mainz and an Assistant Director at the Psychological Science Accelerator (PSA). After studying Law and Psychology at University, he completed a Master’s programme in Social Psychology in Wuhan, China. In 2007, he completed his PhD in Beijing before moving to Germany. His research investigates the consolidation of positive memories, which may play a role in the resilience to stress.
Max: Thank you for taking the time to talk to us. Let us jump right in and start with the first question: Why is Open Science important to you? How do you implement and promote Open Science practices in your own work and at your institution?
Chuan-Peng: I think Open Science is important because it is an umbrella of solutions for the replication crisis in Psychology. My first experience with the replication crisis was my attempt to replicate the high-profile weight-embodiment study. The replication attempt failed, so I tried to contact the authors, who did not reply. That is when I first felt that something was wrong. Soon after, Bem published his famous precognition study, the replication of the elderly priming study failed and the scandal surrounding Diederik Stapel unfolded. All these events and subsequent online discussions made me think that our research culture is deeply flawed. Simultaneously, the events marked the birth of Open Science. Many researchers began to organize the “Many Labs” replications and started to promote pre-registration. Personally, I was convinced that being transparent and open is the way to alleviate the problems in psychology and science more broadly.
Admittedly, appreciating the idea of Open Science was considerably easier than implementing Open Science practices into my own research. Firstly, being open takes time. I taught myself R and spend half a year re-analysing my previous experiments. I learned how to use Matlab instead of E-prime. Secondly, I began to take statistical power seriously. Many experiments I conducted during my PhD used the old-fashioned way to determine sample size: Following previous studies, with generally low sample sizes. To improve on that, I followed suggestions from the “life after p-hacking” paper and increased the sample size of my subsequent studies. Thirdly, I started to pre-register my experiments, but I have to say that I could have done my pre-registrations much better. I also participated in projects like the pre-registration challenge, even though I failed to publish it on time. Finally, I started to participate in large-scale collaborations, such as the Human Penguin Project , the Kama Muta Project, and the PSA. I am particularly proud that we shared all our materials (questionnaires translated into nine languages) and data of the Human Penguin Project and published the data descriptor in Scientific Data. Nowadays, all my projects are built on Open Science principles. For example, one of my on-going projects is a direct replication study.
Overall, implementing open science practice did slow me down, but I think doing science in the right way is more important than finishing fast. Though the experiments during my Ph.D. are still not published, many of them are open now, two of these experiments were preprinted, and many more were open as Open Notebook.
I have also been active promoting Open Science in my institutes. For example, when I was at Tsinghua University, Beijing, I have given a few small-scale talks to promote the idea of Open Science. And one professor started to pay attention to the problem and later included replication crisis in his methodological course for graduates. Here in Mainz, I introduced the flexibility in fMRI data analysis and its consequence in our SPM course, and I have initiated a small group discussion on p-hacking during the PhD/PostDoc retreat of our program. However, maybe because I am a bit socially awkward, I feel more comfortable to promote Open Science through the internet. For example, I wrote Chinese blogs and organized journal club outside my institute when I was in Beijing, and engaged in discussions on Twitter.
Max: You are very active in promoting Open Science in China. What is the state of Open Science in China, and how does its implementation differ between China and western countries like Germany? Did you encounter any challenges in China you did not encounter in Germany, or vice versa?
Chuan-Peng: I started to promote Open Science in China around 2015. At that time, very few people were talking about Open Science and reproducibility there. For example, at the major psychological conferences in China there was not a single symposium about Open Science or reproducibility. So, I decided to take the risk and speak up. I teamed up with a few colleagues in my institute and together we wrote a comprehensive Chinese introduction about replication crisis and related methodological reforms. During the publication process, reviewers told us our tone was too negative, but luckily, we got it published eventually. Later, some grads told me that they really found this paper helpful and took it to their journal club. I also co-founded the Chinese Open Science Network.
As compared to the psychology community outside China, the situation in the mainland changes very slowly. Most psychology graduates are still unfamiliar with concepts such as direct replication, preregistration, open materials or open data. Few people talk about statistical power openly, and there is little support for researchers trying to adopt new practices. Also, very few Chinese research teams participate in international large-scale collaborations (e.g., only around ten teams from mainland China are part of the PSA).
The positive side is that things are changing in China. There are some mid-career researchers who strongly support Open Science. For example, three researchers from the Institute of Psychology, Chinese Academy of Sciences, are engaged in efforts to promote Open Science practices: Dr. Xi-Nian Zuo did a lot of work on the reliability of fMRI and created an open database for assessing reliability; Dr. Chao-Gan Yan organized a large-scale collaboration study on the brain structure of depressed individuals, and Dr. Huajian Cai is leading a large-scale project surveying the socio-cultural traits of Chinese people, probably the largest social psychology study ever conducted in China. In Hong Kong, Gilad Feldman is organizing a massive replication project. In Taiwan, Sau-chin Chen is leading a project of the PSA.
Beyond that, young Chinese grads started to develop toolboxes to facilitate the use of R (e.g., the bruceR). A preprint service, Chinaxiv, is curating thousands of preprints from many different fields. Even more encouraging is that, in 2018, the best Chinese psychological science journal, Acta Psychologica Sinica, started to encourage pre-registration, and now requires sample size justification. Given its prestige in China, these changes may have a strong impact.
There are many challenges for Open Science in China. Many of these challenges are rooted in the history of Psychological Science or Science in China. I will name a few. First, the psychological science community in China is relatively small and segregated from other fields (neuroscience, sociology, etc.). This status has historical reasons: the development of modern psychological science in China started only about forty years ago, so psychology as a discipline is relatively young. A small community, combined with our hierarchical social norms, makes changing the professional culture particularly hard. Very few researchers openly criticize others. Another challenge stems from the lack of communication between Chinese researchers and their international peers. For example, inadequate training in the English language or the Great Firewall hinder adaptation of Open Science practices.
In comparison, I think Open Science in Germany is growing fast. There is a large group of early career researchers (e.g., Julia M. Rohrer, Felix D. Schönbrodt) and mid-career researchers (e.g., Simon Eickhoff) who are doing great work in sharing data, developing tools/methods, and doing meta-research. In the last few years, many Open Science initiatives were founded in Germany — it is very encouraging.
Max: You mentioned that you co-founded the Chinese Open Science Network. Tell me a bit about your initiative.
Chuan-Peng: The Chinese Open Science Network is a loosely organized grassroots network, which I initiated online. The main platforms of our network include a WeChat account (Chinese Facebook) and a public project on the Open Science Framework. The WeChat account is organized like a blog. Users can also subscribe to us and receive updates on our work. WeChat has a very large user base in China, so it is an important source of information. Our account currently has 6500 subscribers and we regularly organize online discussions or ask for volunteers for Open Science related activities (e.g., translating tutorials or blogs). The Open Science Framework (OSF) project is where we store all materials related to Open Science. It includes the PowerPoint presentations of my talks, an online journal club, and recordings of lectures. We even have podcasts on there.
Through the Chinese Open Science Network, I could find people who are interested in Open Science and invite them to promote Open Science with me. For example, I recruited a few collaborators and finished a Chinese tutorial on how to calculate the confidence intervals of effect sizes. Now, we are preparing a tutorial on how to better interpret non-significant results. Also, we started some meta-research on Chinese researchers and Chinese papers such as the missunderstanding of p values in psychology and other fields . Moreover, I recruited Chinese colleagues from the Chinese Open Science Network to translate and proofread the Seven Deadly Sins of Psychology by Chris Chambers into Chinese.
Max: What advice would you give scientists trying to start an Open Science Initiative in their country?
Chuan-Peng: My suggestion is to push the limits a bit to do what you can do in your own culture. Promoting Open Science seems to be unpopular because you are against the old ways of doing science, senior researchers may be hostile to you. When we wrote the introduction to the replication crisis and methodological reform in Chinese, some colleagues suggested that we stop, because it could be risky for our careers. We feared that it might offend some senior researchers, and even though it attracted criticism and negative comments, it eventually got published. The lesson I learned from that is that you never know how many people share your feelings and ideals, until you tried it. Finally, I suggest researchers to use the internet well and keep in touch with the global Open Science community. If few people at your institute are interested in Open Science and you do not connect with the larger community, you may feel isolated and lonely, which makes your efforts not sustainable.
Max: You are an Assistant Director with the PSA. Can you tell me something about your work there?
Chuan-Peng: Yes, I am an Assistant Director of the Translation and Cultural Diversity Committee. In this committee, we are responsible for two things: coordinating the translation of materials of PSA projects and increasing the (cultural, geographical, SES, etc.) diversity of PSA. For example, in coordinating the translation of PSA 006, my role is to contact teams from different languages and make sure that they know our standard translation process. Then, I will track the progress of translation for each language. To improve the quality of the translations, we are planning to draft a guideline about best practices for the translation process.
Max: What advice would you give to aspiring scientists trying to implement Open Science practices in their work?
Chuan-Peng: “The first principle is that you must not fool yourself – and you are the easiest person to fool.” is a quote by Richard Feynman which serves as both inspiration and warning to me.
Max: Any Final Remarks?
Chuan-Peng: Thanks for having me.
Max: Thank you very much for taking the time.
Maximilian Primbs
Maximilian is a Research Master´s student in Behavioural Science at Radboud University and a Research Assistant at the Behavioural Science Institute. He´s interested in prejudice, stereotypes, faces, and research methodology. In his free time, he enjoys contact sports and metal music.
Follow Me:
