Science is the collaborative attempt to understand ourselves and the world around us better by gathering and evaluating evidence. Ironically enough, we are pretty bad at evaluating evidence. Luckily, others rejoice in pointing out our flaws. It is this reciprocal corrective process which is at the core of science, and the reason why it works so well. Working collaboratively helps us catch and correct each other’s mistakes.
With the rise of social media, potential distractions have risen to unseen levels; they dominate our daily lives. Do you check Facebook, Twitter, Snapchat, Instagram, or Email on a constant basis? Do you have an embarrassing relationship with your alarm clock’s snooze button? Do you pass on social invites, telling other people that you are too busy? As a generation, we have lost the ability to focus sharply on the task at hand; instead, we work on a multitude of things simultaneously, lamenting that we do not achieve what we seek to achieve. Continue reading
There is no panacea for bad science, but if there were, it would certainly resemble Registered Reports. Registered Reports are a novel publishing format in which authors submit only the introduction, methods, and planned analyses without actually having collected the data. Thus, peer-review only focuses on the soundness of the research proposal and is not contingent on the “significance” of the results (Chambers, 2013). In one strike, this simple idea combats publication bias, researchers’ degrees of freedom, makes apparent the distinction between exploratory and confirmatory research, and calms the researcher’s mind. There are a number of journals offering Registered Reports, and this is arguable the most important step journals can take to push psychological science forward (see also King et al., 2016). For a detailed treatment of Registered Reports, see here, here, here, and Chambers (2015). Continue reading
Last summer saw the publication of the most important work in psychology in decades: the Reproducibility Project (Open Science Collaboration, 2015; see here and here for context). It stirred up the community, resulting in many constructive discussions but also in verbally violent disagreement. What unites all parties, however, is the call for more transparency and openness in research.
Eric-Jan “EJ” Wagenmakers has argued for pre-registration of research (Wagenmakers et al., 2012; see also here) and direct replications (e.g., Boekel et al., 2015; Wagenmakers et al., 2015), for a clearer demarcation of exploratory and confirmatory research (de Groot, 1954/2013), and for a change in the way we analyze our data (Wagenmakers et al., 2011; Wagenmakers et al., in press). Continue reading
Last summer saw the publication of a monumental piece of work: the reproducibility project (Open Science Collaboration, 2015). In a huge community effort, over 250 researchers directly replicated 100 experiments initially conducted in 2008. Only 39% of the replications were significant at the 5% level. Average effect size estimates were halved. The study design itself—conducting direct replications on a large scale—as well as its outcome are game-changing to the way we view our discipline, but students might wonder: what game were we playing before, and how did we get here? Continue reading
Bayesian statistics is what all the cool kids are talking about these days. Upon closer inspection, this does not come as a surprise. In contrast to classical statistics, Bayesian inference is principled, coherent, unbiased, and addresses an important question in science: in which of my hypothesis should I believe in, and how strongly, given the collected data? Continue reading
Would you like to collect data quick and efficiently? Would you like to have a sample that generalizes beyond western, educated, industrialized, rich and democratic participants? While you acknowledge social media as a powerful means to distribute your studies, you feel that there must be a “better way”? Then this practical introduction to crowdsourcing is exactly what you need. I will show you how to use Crowdflower, a crowdsourcing platform to attract participants from all over the world to take part in your experiments. However, before we get too excited, let’s quickly go through the relevant terminology. Continue reading