An interview
We recently talked to Felix Schönbrodt about his paper, "Testing Similarity Effects with Dyadic Response Surface Analysis", which will appear in an upcoming issue of EJP. Felix is Principal Investigator at the Ludwig-Maximilians-University, München.
Read more about the study below!
Q: Hello Felix! Can you tell us what this study is about?
Many theories in psychology entail congruence hypotheses or similarity hypotheses, for example theories of person-environment fit, self-enhancement, or self-knowledge. All of these theories predict that an outcome is optimal if there is a match between two variables, and that the outcome gets worse, the more discrepancy there is between both predictor variables.
Some of these theories are in the context of dyadic relationships (e.g., the relationship between two people), for example the research question, does similarity of personality predict relationship satisfaction of both partners? In this case you have (at least) one predictor for each member of the dyad, which can be more or less congruent, and an outcome variable for each member. This entails a dyadic interdependence structure, such that the data for one person cannot be considered to be independent from the data of their partner’s, because the outcomes (e.g., better relationship outcomes) of both partners most likely are correlated.
In our paper, we propose a "marriage" between a polynomial regression with response surface analysis (RSA; which is the appropriate model for similarity hypotheses) and the actor-partner-interdependence model (APIM), which is one of the standard models for dyadic data. Both parts of the marriage contribute a necessary feature for our research questions. First, APIMs can model dyadic interdependence in the data. Classical formulations of APIMs, however, only contained main effects or multiplicative interactions, which are not capable of testing congruence hypotheses. Second, polynomial regressions (i.e., regression that contain the predictor terms X², Y², and X*Y), and follow-up analyses on the response surface that these regressions create, provide the toolbox for an appropriate analysis of congruence hypotheses. RSA, however, has not yet been formulated for dyadic settings. By combining both approaches into one model, we can now appropriately test similarity/congruence hypotheses in dyadic settings.
In our case, we needed the dyadic RSA model for our own substantive research questions, for example the question whether the similarity in personality predicts better relationship outcomes in couples (e.g., Weidmann, Schönbrodt, Ledermann, & Grob, 2017). We gave a first hint of the dyadic RSA in a recent methodological EJP publication (Nestler, Grimm, & Schönbrodt, 2015), but we thought it was worth it to expand this to a full tutorial-style paper.
Q: What are some of the implications?
Statistical models should match as closely as possible to the verbal hypothesis. Most theories and models in psychology are vastly underdetermined, it’s more prose than exact descriptions and definitions. With such imprecise descriptions of latent constructs and assumed relationships (without specifying the exact functional form of the relationship, or even quantifying the strength of the relationship) these verbal models cannot make precise predictions; and a huge array of result configurations can be made to "fit" to the verbal model. Using RSA helps force you to think more precisely about your hypothesis, and enables you to precisely test them. A famous quote from George Box says: All models are wrong – but some models are useful. We think that the dyadic RSA is more useful and more valid for testing such hypotheses than previous approaches have been.
Q: What made you decide to study this topic?
It all started many years ago when I did a study about implicit-explicit motive congruence. The hypothesis in this line of research is that if your implicit motives do not match your explicit motives you will experience negative consequences, such as increased psychosomatic symptoms or reduced life satisfaction. Previous studies often tested this hypothesis with moderated regressions, which is an inappropriate statistical model – it simply does not match to the verbal hypothesis (although, at first glance, it can look like an appropriate model).
Then I read a lot of literature on response surface analyses and polynomial regression, which is in fact an appropriate model to test this hypothesis. This topic has been pioneered by Jeff Edwards in the Industrial/Organizational field, and he wrote some really excellent and accessible introductions and in-depth papers on this analytical method. I programmed an R package for running (non-dyadic) response surface analyses; I am happy that Sarah Humberg recently joined the development team for this package. We now extend the RSA approach in several ways – for example for multilevel models, together with Steffen Nestler.
Q: Where do you see yourself in the (near) future?
My work has three pillars: First, developing methods (which also includes attempts to explain complicated methods in an accessible way, and to evaluate methods) currently in the area of Bayes factors, meta-analysis, and machine learning with texts. Second, I do substantive research in the field of motivational psychology, mostly in close relationships. Currently our team does a lot of experience sampling and mobile sensing research, and we like to go more into within-person models, and how between-person traits relate to these within-person processes. Third, I continue to advocate for open science, mostly in our newly founded Ludwig Maximilians University (LMU) Open Science Center, which also includes meta-science research. I hope to maintain a good balance between these three areas.
Q: Do you have any tips or advice for young researchers?
I once saw a nice Tweet on survivorship bias, it goes like: "Asking a tenured academic for career advice is like asking a lottery winner for his numbers". So, everything I say should be taken with a grain of salt.
Be aware of the difficult career paths in academia (of course that partly depends on the country you are working in). The pyramid is extremely steep and there simply are not enough permanent positions at the top for all PhD graduates (even if you factor in that many leave to the industry). I was lucky to find a permanent position, but you cannot really expect that, even if you are dedicated, work hard, are talented, and are willing to sacrifice a lot in other areas of life. It's hardly controllable and depends on many lucky circumstances. But if you succeed, I think it is one of the best jobs in the world.
Be grounded in your values of good scientific practice. Incentive systems start to change, but there's still a lot of pressure to publish fast, publish sexy, and publish a lot. I sometimes imagine my future self in thirty years which looks back and asks: What publications (and other academic actions) are you proud of? Which made a real impact, a real difference? (That certainly only applies to a minority of my work). I think that this approach of slow but credible science makes more sense, is more responsible, and makes one more satisfied in the long run. But: it might be at odds with current incentives and might be hard(er) to follow if you have an untenured position.
Be open - in the sense of "open science", but also open to collaboration. Doing work together with fellow colleagues was the most gratifying way of doing research, and it makes so much more sense if you pool your resources and do something with impact. Openness might – at first – make you feel vulnerable, but I think it is absolutely worth the risk. In my entire academic life, I have never personally encountered scooping, fights for authorship positions, or the like (again, maybe I just had an abundance of luck). I think the benefits of cooperation and openness are so much worthier than the potential risks we might fear. If you are in a working environment that does not support you, or even bullies you into bad science (link), consider leaving and go to a better place.
Use social media (wisely). I recently deleted my Facebook account, but Twitter is the single most important place for me to get news about relevant research, or about the latest collaborative replication project. And it's always fun to meet a person in real life that you have "known" on Twitter for a long time.