An interview
We recently talked to Gabriel Olaru about his paper, "Ant Colony Optimization and Local Weighted Structural Equation Modeling. A tutorial on novel item and person sampling procedures for personality research", which will appear in an upcoming issue of EJP. Gabriel is a PhD student at the University of Kassel, Germany.
Read more about Gabriel and the study below!
Q: Hello Gabriel! Can you tell us a little about yourself?
I just finished my PhD thesis – I actually just turned it in – and I'm waiting for the reviews right now. I hope I can defend it next month. I started my PhD in Ulm for a year, then I moved to the United States to work there for a year and do my thesis on the side – which I found was very difficult if you work; I have a lot of respect for people who can do their PhD part-time while working full-time! And after that year, I came back to Germany, started in Bamberg and then finally came to Kassel (where I am currently) with my current supervisor. So, I have moved around a lot!
In my work, I focus on methods. I'm interested in new methods that can help us improve our findings so that they are more precise – especially if they are in R, which I really enjoy because it's openly available and very flexible. Right now, I'm mostly working on short scale construction or item selection, but not in the classical sense of just shortening your scale (e.g., creating a shorter scale which takes less time), but in the sense of improving your measurement, for instance by identifying the items that are most precise, reliable or comparable across age or cultures. This is also what we mention in the article in EJP, which we will discuss in a bit. In my PhD, I used these methods mostly to examine personality across age. For instance, an item such as “I like rollercoasters” is a great item to measure extraversion in young people, but it works terribly for old people. Such an item would probably be eliminated by the method, but also items with less obvious differences in how they function in young and older people. You may be able to identify some of these bad items but it gets tricky, especially when you have 240 items. So that is where the item selection methods come in handy.
I am now also applying these methods to personality scales that should be comparable across cultures, because the same issues may arise in that context. Okay, enough about academics!
I generally like to travel a lot (as you may also be able to tell from my time as a PhD), especially if it is to countries with very different cultures to mine. I just returned from Taiwan, which was really nice! It was beautiful, and the landscapes were varied. At home, I typically just play computer games and watch YouTube. I also like rock climbing, but honestly, I mostly just watch YouTube and play games!
Q: What was this study about?
The article is a tutorial on the item selection method that I just mentioned and also on a method for doing moderator analyses, which we used to look at whether age affects personality.
The item selection method is called "Ant Colony Optimization", because it's based on the behavior of ants with food. Ants will try to find the shortest road from the nest to the food source by leaving pheromone trails. Other ants will then follow the pheromone trails so that they find the shortest route. Ant colony optimization does the same with items, trying to identify which items are best for your research goal (e.g., finding items that are comparable items across age). The method creates virtual pheromones for each item and then it'll start picking items randomly and looking which ones yield the best solution. It will then increase the pheromone levels for these items and select them more frequently in later iterations until a optimal solution is found. I think it's a really interesting direction and this method actually does surprisingly well compared to if you’d try to identify the best items “by hand”.
The second part part of the article was on the moderator analysis method, which is called "Local Structural Equation Modeling". It allows you to look at your personality model – or any type of model you are interested in – across a continuous variable like age and it will estimate the model at each age point. What you get from that is very nice plots on which you can see how the personality model changes across age. So for example, it is very interesting to examine the mean-level and structure of personality across age, because the method shows you that personality doesn't change linearly across life, but mostly during young age. Using this technique allows you to see these trajectories.
In a way, Local Structural Equation Modeling examines measurement invariance. Using Local Structural Equation Modeling to do so is nice because you can see how your factor loadings, item intercepts, variance, or whatever you want to look at changes with age (which they shouldn’t, if your measure is invariant). Therefore, one can use the item-selection method to create measurement invariant scales, and then use Local Structural Equation Modeling to examine how the model changes across age, which is very relevant for personality development research. But you can apply them to anything else! For example, right now we are using them to see how knowledge acquisition at school changes depending on the socioeconomic status of the family. This is quite a big problem in Germany, as children go to different schools, depending on the socioeconomic status of the family – not necessarily the one that is best suited for them.
Another major benefit of these methods are the sample size requirements. In classical measurement invariance testing you create groups (e.g., you compare young to old individuals) and then you can only look at one difference, which is a linear difference. What Local Structural Equation Modeling does is that it doesn't form a group at each age, but instead it uses a normally distributed weighting function. It will weight all people of the target age at 1, but it will also pick people of age 29 and 31, but then with lower weights, and so on. So, in the end, even though you may had only 20 people who are 30-years-old, you can obtain a sample size of 300 by including all people from other ages, just with decreased weights. The idea behind this is that it doesn't really matter if someone is 30- or 29-years-old. Of course, it does matter whether someone is 20- or 30-years-old, so someone of 20 years-old would get a weight near 0.
Additionally, the required sample size also depends on the number of points on your moderator. If you're looking at education level, for example, it typically has about 13 levels if you don't count university. For that, 300 people should be enough because of the weighting. But if you want to look at age from 16- to 90-years-old, then 300 is probably not going to be enough. So, this method has ways to overcome low sample size, but at the end of the day, it's still a Structural Equation Model, so you shouldn't use it if you have only a very small number of people.
I think the main message that we wanted to get across with this article – in particular with the item selection method – is that we need to lose the "respect" we have for existing measurements. Just because someone wrote it 20 years ago doesn't mean you can use it for everything, and there is no such thing as the perfect scale. There will be good items in there, but there will also be bad items, depending on what you want to do. The story we try to tell is that researchers should be mindful about which items you are going to use, because that can change the entire outcome of your study. A perfect example of this is that in one particularly interesting application of the Ant Colony Optimization method, we had data from a national science knowledge test. With this data, you typically find gender differences, with men performing better on natural science questions and women performing better on humanities-related questions. But, by using these item-selection algorithms, we could actually turn the whole thing around and make women better at the natural science questions and men better at the humanities-related questions. I'm not saying "try to change gender differences in one direction or the other", but it shows how the findings depend on which items you use. I do think that we as researchers should be aware of that. We should try to figure out which items are the most appropriate for the research question we want to answer.
Q: You almost finished your PhD — congratulations! What’s next?
Three things are very important to me. First, I want to keep trying to find new methods to help us solve problems in psychology. And then, I probably would like to do something similar to the tutorial that we wrote to spread the word. I like the item selection method a lot, but I also want to do and learn something different, for example, machine learning. Ant Colony Optimization was developed by computer scientists and it has been roaming around that field for a while before someone discovered it for psychology. Something I would like to do is look at what computer scientists are doing and see if we can use more of their methods in psychology. I enjoy looking for something new, to keep things interesting.
Second, I also want to travel more, not just in my free time but also for work. I would love to do a research visit in combination with traveling -- that would be great. I was planning to go to Oregon, because I heard it's really beautiful with the Rocky Mountains, and the forests, and the Pacific Ocean!
As a third point, I would also like to continue teaching, because I really like that a lot.
Q: What kind of topics do you teach?
I teach about qualitative measures, so for example, how to conduct interviews. This is absolutely not my topic, but I'm still having fun with it. At first, I wasn't very fond of the topic, but it turned out really nice.
I also taught Personality and Intelligence Assessment last semester, which was a lot of fun. We started building our own personality test, which everybody completed, and then we analyzed the data.
Q: Do you have any tips or advice for young scholars?
I'm still trying to figure out stuff myself, but I can share some things that helped me. I mentioned that I had to teach qualitative methods and that at first, I wasn't too fond of that. But it actually inspired me to use qualitative methods in an article. In this paper, we picked the items that were least measurement invariant across age; so the ones that differed most. We had a total of 240 items, so we couldn't look at each item, which might have led to a hindsight bias (e.g., "oh in hindsight it makes sense that this item only works for old people"). So, I used qualitative classification techniques to structure these items. The advice I want to give is that sometimes you can get inspiration from places you wouldn't expect it to come from. So, try to also look at what other people are doing, talk to other people – be open.
Another thing I liked was having a small side project. In your PhD, all of your projects are going to be related and if you have a small side project it will allow you to change things up a little and make your PhD experience more varied. There shouldn't be too many side projects of course, or else you won't get your main articles done. But having something else on the side is really nice, especially if you're waiting for reviews and there is downtime. For me this was developing personality Situational Judgment Tests; this was not something I generally do and also not related to my PhD, but I found it very enjoyable.
A more practical tip is that academic writing was very hard for me at first. It didn't come naturally and I struggled a lot until I found this book, "Write it Up" by Paul Silvia, which gives some great advice on how to make your writing more varied and more enjoyable to read. But also things like how to change your grammar, how to use punctuation marks to make it more varied, tec. This helped me a lot; after I read the book, my advisor’s first feedback was, "What happened to your writing style; it's totally different." So I guess my advice would be to read a book on academic writing or take a writing workshop.
That's all the advice I can give, based on my personal experiences.