Author Archives: Karla Matić

Karla Matić is a psychology graduate of University of Leuven with interests in cognitive neuroscience, large-scale neuroimaging methodology, and science policy. She is currently an intern in the European Research Council (ERC) in Brussels. If she didn't aspire for an academic career, she would be running a book-café on a small Croatian island.

Publishing the results of coursework research: An interview with Julian Burger and Koen Derks

submit-you-mustBeing an undergrad is hard. With the days spent in lecture rooms and the nights devoted to catching up with essays and assignments, one wonders how is it even possible for undergrads to do any research – let alone publish it. While there is no expectation from undergrads to publish, a rough (and very anecdotal) approximation is that around 1 in 100 students publish during their undergraduate studies in either a peer-reviewed journal or other online outlets. (However, this highly depends on the field and publishing culture of the affiliated institution). There are also many benefits to publishing as undergrad; as illustrated by Griffith (2001), an early publication – regardless of the importance of the findings or prominence of the outlet – can increase student’s confidence and inspire a prolific academic career in the future. So how do these acclaimed one-in-a-hundred undergrads manage to publish amid challenges of the student life?

One way is to publish the outputs of coursework assignments – be it an empirical study or a review article. This is precisely what Julian Burger and Koen Derks from the University of Amsterdam did with the group assignment from one Image result for r shiny logoof their Research Masters courses. Together with their classmates, they developed a method for ranking publications in a literature review, and wrote it all up in an article that was recently published in JEPS. In addition, they created an interactive tool in R Shiny – a package implemented in R that allows for effective and didactic illustration of one’s analysis tools and methods. We were curious about the process of publishing the results of coursework assignments, so we invited Julian and Koen to share some of their insights. We hope you enjoy the read – and hopefully get inspired to publish some of your own coursework research as well!

Could you tell us a little bit about the study you recently published in JEPS?

Our recently published study in JEPS has its origin in our 2016 Research Master course “Good Research Practices”, which elaborated on the methodological do’s and don’ts of scientific research. Our course coordinator at the time based the course around the topic of a prevalent misunderstanding in the use of ANCOVA, namely that when groups differ on a covariate, removing the variance associated with the covariate also removes the variance associated with the group (Miller & Chapman, 2001). As such, an ANCOVA with covariates that are too intimately related to group membership yields unreliable results. As preparation for group-projects, every student searched for 40 articles (20 before and 20 after Miller & Chapman) related to the issue under consideration. Out of a need for a concise literature overview, we started thinking about possible solutions to aggregate this literature. We were under the impression that there had to be a way to statistically find the top-something relevant articles that everybody could read to get a simple, but complete, overview of the topic. This is how the idea of a network model of our literature search was born. We came up with two distinct methods with which we created separate networks, one of their citation structure and one of their co-occurrence frequency. These networks, as they describe the relationships between articles, give relevant insight in the relative importance of the articles in the literature.

Creating a Shiny App sounds like an exciting way of presenting one’s research. Was it hard for you and your colleagues to build the app?

We have had some experience with programming in R during the Bachelor and Master programmes. However, we had never implemented a Shiny app to demonstrate our work publicly. Programming the network method itself was the most difficult part. The Shiny app is of course a nice way to promote the network method and let other people benefit from it, and since Shiny is well documented, it was a fairly straightforward task. Shiny has so many advantages for demonstrating your work and making it publicly available for others, it is amazing for these projects.

derks-app-viewA view inside the Shiny app for network visualization of literature search. The app accompanies the publication, and instructions for use can be found in Appendix C of online supplementary materials.

Publishing a paper with 52 co-authors sounds like a challenging collaborative endeavour. Could you recap the process for us – how did it all go, from the first idea to final manuscript edits?

Within one group of the course, we worked on a way to aggregate the literature collected by all students. In this group we applied the two different network models to the articles collected by all contributors. From our experience it is very important to assign a clear role distribution from an early time-point on and communicate the obligations that come with this well. Throughout the course, we presented the results of the analyses in class, collected feedback by the students and after the course wrote the article in a team of two. We appointed a main contributor who took care of the main writing and fine-tuning and a second contributor who wrote another part of the article. The course coordinator took a supervising role and was available for meetings and feedback on the writing.

And what were the most difficult aspects?

Coordinating the feedback and rewriting the article. If you work with multiple authors that takes a lot of time. Because some of our classmates were following other courses, it was not always easy to contact them for feedback in the later stages of the publishing process.

Do you have any tips for students who are thinking about publishing the results of their coursework research? Or maybe for lecturers who consider structuring their course assignments in a similar way?

From our experience in working on this project, we think these points might help in coordinating a project with a large group of students:

1. A clear role distribution. To prevent misunderstandings regarding responsibility, we advise to spend enough time on clarifying who is working on what.

2. Not too many contributors involved in the actual writing. From our experience, it worked well to have two people involved in the main writing. It is of course useful to collect feedback from the group, but to have a coherent story, not too many writers should be involved.

3. Have one main responsible contributor. It worked very well for us to appoint one main contributor, who took the main responsibility of incorporating and coordinating feedback.

4. Make use of feedback sessions. From our experience, the main benefit of working in a big group is that you can use a lot of input from different perspectives.

What are your future career plans?

Koen: This year I started a PhD focused on statistical auditing. My main topic is developing Bayesian alternatives to classical audit methods and implementing these methods in JASP for Audit (JfA), which is built to support auditors in their statistical journey. In the future, I plan on staying in academia and continue learning about Bayesian statistics.

Julian: I started my PhD this year on network models/dynamical systems in psychopathology. My plan is to make these models more accessible for clinical practice and at some point in the future to combine the research with working as a therapist myself.

Any final comments?

The Journal of European Psychology Students is a great outlet for publishing findings from such a student course project, so we think it is definitely worthwhile trying to write down your work and get to know the process of scientific practice and publishing!

If you’re inspired by Julian’s and Koen’s story and wish to put some of your own coursework research in writing, check out out submission guidelines at!

Karla Matić

Karla Matić is a psychology graduate of University of Leuven with interests in cognitive neuroscience, large-scale neuroimaging methodology, and science policy. She is currently an intern in the European Research Council (ERC) in Brussels. If she didn't aspire for an academic career, she would be running a book-café on a small Croatian island.

More Posts

Follow Me:


Between science and policy: an interview with Dr Toby Wardman

Even though scientists are oftentimes lost in the ivory towers of their scientific work, academic research in any discipline – and especially psychology – is tightly connected to the society. It contributes to the improvement of the living conditions in the population. It supports the decision-making process of policy-makers with scientific evidence. And it is paid for by the tax-payers’ money. In an attempt to ensure that this natural relationship between science and society is always well-balanced, we make policies – governmental policies, international policies, institutional policies. The field at the interplay between science and policy-making – very intuitively coined ‘science policy’ – therefore concerns itself with topics such as the allocation of resources for scientific research, the careers of scientists, and the systems of efficient communication between scientists and policy-makers (Pielke, 2005).

Precisely this area of science communication is the field of interest of Dr Toby Wardman. He works in SAPEA – Scientific Advice for Policy by European Academies. SAPEA is a part of European Commission’s strivings to efficiently communicate with academic researchers and inform the decisions on new policies with scientific evidence. Its role is to provide timely, independent and evidence-based scientific opinion on a diverse set of relevant topics to both the EU policy-makers and the wider public. By bringing together the knowledge and expertise of scientists from Academies and Learned Societies in over 40 countries across Europe, SAPEA plays a crucial role in bringing the scientific findings from the lab bench to the policy desk.

When it comes to big questions, such as genetic modification, or cybersecurity, the quality of the policy will necessarily depend on the quality of the scientific evidence. In an attempt to gather comprehensive evidence and policy advice in a robust and efficient system, European Union relies on an in-house advisory body called Scientific Advice Mechanism. This body is run by 7 prominent scientists, and through SAPEA (and several other mechanisms) it provides independent scientific advice directly to European Commissioners. By acquiring scientific evidence on relevant policy initiatives and debates from many hubs of expertise in different European countries, this mechanism optimises the objectivity of evidence-based policy recommendations.

Scientific Advice Mechanism (SAM) – the process of closing the gap between scientific evidence and policy in the European Commission.

The scientists’ role in this process is pivotal; after all, each researcher is the best connoisseur of their own research. Communicating research outside narrow academic circuits is also a highly valuable skill within the academic community – it improves academic prospects and brings new perspectives into researchers’ scientific work. Yet many believe that scientists still do not engage enough in public outreach, and call for more education on science communication (including strategies to counteract the post-truth culture that propagates misinformation). In an interview following our meeting at a workshop on science communication, I talked with Toby about science, policy, communication – and everything in-between. Enjoy the interview! (And want to know more? Check out the ‘Further readings’ below.)


You started off as an academic studying philosophy of science, but then took a turn towards science and policy communication. Tell us a bit about your background, where did you start from, and how did you end up where you are today?

Actually, I started work in communications straight after my Bachelors degree and have worked full-time ever since – I studied both my masters and my PhD part-time alongside working. My jobs have been a mix of science communication (which I do now) and political communications, mostly in the UK. I worked on the EU referendum on the Remain side, and when it all went wrong, my wife and I decided that enough was enough and we moved to Brussels. Now I work for SAPEA, part of the European Commission’s Scientific Advice Mechanism that provides science advice for EU policy-makers.

Many people these days argue that political decisions should be more informed by the results of scientific research. However, scientists and politicians tend to speak completely different languages – which poses a massive challenge to evidence-based policy. How does the EU bridge this gap between science the policy?

The EU actually has a pretty robust setup so that policy-makers can get advice from scientists. EU policies generally start life as drafts from the European Commission, so that’s where the Scientific Advice Mechanism comes in. Before drafting a new policy, Commissioners can ask us any question about the state of science in that area. In response, we give them two documents: a comprehensive and independent review of the evidence, which is done by SAPEA (‘Science Advice for Policy by European Academies’), consisting of experts from more than 100 European academies working together; and a Scientific Opinion which is drafted by the Group of Chief Scientific Advisors based on SAPEA’s evidence review. Those two documents are then used by the Commission to decide what policy action to take. Equally, the scientists themselves can offer advice to the Commission on topics which they think are important, rather than waiting to be asked the question.

And do you think that this mechanism works well? In other words, can we say that EU policies are evidence-based, or is there still space for improvement?

Yes, I think the EU sets a pretty good example in this field. The Commission has access to very high-quality science advice, and the impact of that advice can be clearly seen in the proposals they write, especially in technical fields, which are often very closely based on the science.

Of course, that’s not to say that every new EU law looks exactly how scientists might want it to look. Firstly, there is always room for improvement. And secondly, it’s important to remember that we don’t really want our public policy to be dictated exclusively by science. We live in a democracy, not a technocracy. The role of scientists is to provide evidence and advice, and the role of politicians is to weigh up that evidence and advice, along with many other considerations, when deciding what policy measures to adopt. Science is value-neutral: it can give us information about the way the world works and what effect certain actions might have, but it can’t tell us what is the right thing to do, or what the public supports.

Many of our readers are students and young researchers who might be thinking about continuing their careers outside of academia. What are the pros and cons of your current job, as compared to doing full-time academic work?

For me, science communication in general sits in the sweet spot between academia and the rest of the world. On the plus side, I get to work closely with Europe’s top researchers and dive into all kinds of interesting scientific areas, while also keeping a foot in the policy side of things and seeing our work have a real, concrete impact on legislation and people’s behaviour. The obvious downside, compared to academia, is the fact that you are always working on someone else’s research – never doing your own.

One of the projects you worked on included training early-stage social science researchers in communication. Why is it important for the early-career researchers to work on their communication skills? And do you have some good resources to recommend to our readers?

We need scientists who can communicate! And most importantly, we need good young communicators who are able to reach out authentically to people of their own generation. That means early career researchers. The training programmes I ran were only a few years ago, and when I recall the content, it already seems embarrassingly out-of-date.

As for resources, honestly the best advice I can give is to practise! Communication, outreach, dissemination – whatever you want to call it, it should be a key part of every academic’s skillset, not an optional extra. So seize every opportunity to write for, or speak to, a non-specialist audience. There are loads of outlets out there which are crying out for good quality content. And even academic journals increasingly have an ‘editorial’ or ‘magazine’ section, or an online companion magazine, which welcomes contributions. As a professional science communicator, I probably shouldn’t say this, but communicating isn’t rocket science. You can just jump in and do it, and learn on the job. That’s how I learned, anyway!

You had the pleasure – or maybe the displeasure – of working on the EU referendum campaign in the UK. How do you think Brexit will affect science and academic collaboration in Europe?

Ugh. It’s already damaging it. Even before the Brexit vote happened, there was clear evidence that UK researchers were starting to get overlooked when it came to putting together project collaborations and applying for funds. That’s really bad news for the UK, obviously, which has always done disproportionately well when it comes to winning EU research funding. But it’s also bad news for research across Europe, because Britain has traditionally had a lot to offer.

It’s depressing, because research collaboration is really one of the most obvious benefits of EU membership. And no matter what you think about issues of sovereignty or migration or whatever, you surely have to agree that combining our firepower – both in terms of collaborations and funding – is obviously a good idea when it comes to research. It simply makes no sense for 28 countries to spend money 28 times over.

Now that Brexit is happening, it’s not just loss of collaborations and loss of funding, but also the very real danger of “brain drain”. If you’re a talented researcher from anywhere in the world, and European universities are competing to attract you, why would you choose the UK? Of course, individual researchers will still be able to collaborate, but without the EU’s framework, it will just make everything much harder and therefore less likely to happen.

 Any last comments, thoughts, or recommendations for our readers?

Sometimes I think of science communication as a necessary and temporary evil: we only need science communicators because scientists haven’t learned to communicate, and if only the next generation of researchers would learn how to communicate themselves directly, then they wouldn’t need people like me. So if you have an interest in outreach as well as an interest in your particular research area, then great! Being able to sell what you do to a wider audience will always be a boost to your academic career, and it’s only going to get more important.

But then at other times I think that’s nonsense. Sure, some researchers are good at communicating, and that’s wonderful. But many others aren’t, because they don’t have the right skillset or the right interests – and why should they? If you’re good at doing something, you should be able to focus on it even if you’re not also good at talking about it. So I guess I’m saying: if you don’t have an interest in outreach, that’s ok too. It will keep people like me in a job!


Want to embrace Toby’s advice and try out your science communication skills by writing a blogpost for JEPS Bulletin? Write to us at!


Continue reading

Karla Matić

Karla Matić is a psychology graduate of University of Leuven with interests in cognitive neuroscience, large-scale neuroimaging methodology, and science policy. She is currently an intern in the European Research Council (ERC) in Brussels. If she didn't aspire for an academic career, she would be running a book-café on a small Croatian island.

More Posts

Follow Me: