Select Projects
Modeling Social Media habits
This project introduces a novel approach to studying social media habits through predictive modeling of sequential smartphone user behaviors. While much of the literature on media and technology habits has relied on self-report questionnaires and simple behavioral frequency measures, we examine an important yet understudied aspect of media and technology habits: their embeddedness in repetitive behavioral sequences. Leveraging Long Short-Term Memory (LSTM) and transformer neural networks, we show that (i) social media use is predictable at the within and between-person level and that (ii) there are robust individual differences in the predictability of social media use. We examine the performance of several modeling approaches, including (i) global models trained on the pooled data from all participants, (ii) idiographic person-specific models, and (iii) global models fine-tuned on person-specific data. Neither person-specific modeling nor fine-tuning on person-specific data substantially outperformed the global models, indicating that the global models were able to represent a variety of idiosyncratic behavioral patterns. Additionally, our analyses reveal that the person-level predictability of social media use is not substantially related to the frequency of smartphone use in general or the frequency of social media use, indicating that our approach captures an aspect of habits that is distinct from behavioral frequency. Implications for habit modeling and theoretical development are discussed.
You can find the published paper here.
Collaborators: Joe Bayer, Sandra Matz, Yikun Chi, Sumer Vaid, Gabriella Harari
LLMs Can Infer Personality form Digital Footprints
Large language models (LLMs) demonstrate increasingly human-like abilities across a wide variety of tasks. In this paper, we investigate whether LLMs like ChatGPT can accurately infer the psychological dispositions of social media users and whether their ability to do so varies across socio-demographic groups. Specifically, we test whether GPT-3.5 and GPT-4 can derive the Big Five personality traits from users’ Facebook status updates in a zero-shot learning scenario. Our results show an average correlation of r = .29 (range = [.22, .33]) between LLM-inferred and self-reported trait scores—a level of accuracy that is similar to that of supervised machine learning models specifically trained to infer personality. Our findings also highlight heterogeneity in the accuracy of personality inferences across different age groups and gender categories: predictions were found to be more accurate for women and younger individuals on several traits, suggesting a potential bias stemming from the underlying training data or differences in online self-expression. The ability of LLMs to infer psychological dispositions from user-generated text has the potential to democratize access to cheap and scalable psychometric assessments for both researchers and practitioners. On the one hand, this democratization might facilitate large-scale research of high ecological validity and spark innovation in personalized services. On the other hand, it also raises ethical concerns regarding user privacy and self-determination, highlighting the need for stringent ethical frameworks and regulation.
Collaborators: Sandra Matz
Context-Aware Prediction of User Engagement
The success of online social platforms hinges on their ability to predict and understand user behavior at scale. Here, we present data suggesting that context-aware modeling approaches may offer a holistic yet lightweight and potentially privacy-preserving representation of user engagement on online social platforms. Leveraging deep LSTM neural networks to analyze more than 100 million Snapchat sessions from almost 80.000 users, we demonstrate that patterns of active and passive use are predictable from past behavior (R2=0.345) and that the integration of context features substantially improves predictive performance compared to the behavioral baseline model (R2=0.522). Features related to smartphone connectivity status, location, temporal context, and weather were found to capture non-redundant variance in user engagement relative to features derived from histories of in-app behaviors. Further, we show that a large proportion of variance can be accounted for with minimal behavioral histories if momentary context is considered (R2=0.442). These results indicate the potential of context-aware approaches for making models more efficient and privacy-preserving by reducing the need for long data histories. Finally, we employ model explainability techniques to glean preliminary insights into the underlying behavioral mechanisms. Our findings are consistent with the notion of context-contingent, habit-driven patterns of active and passive use, highlighting the value of contextualized representations of user behavior for predicting user engagement on online social platforms.
You can check out our paper here.
Collaborators: Yozen Liu, Francesco Barbieri, Raiyan Abdul Baten, Sandra Matz, Maarten Bos
Generalizable Error Modeling
Machine learning (ML) and artificial intelligence (AI) systems rely heavily on human-annotated data for training and evaluation. A major challenge in this context is the occurrence of annotation errors, as their effects can degrade model performance. This paper presents a predictive error model trained to detect potential errors in search relevance annotation tasks for three industry-scale ML applications (music streaming, video streaming, and mobile apps). Drawing on data from an extensive search relevance annotation program, we demonstrate that errors can be predicted with moderate model performance (AUC=0.65-0.75) and that model performance generalizes well across applications (i.e., a global, task-agnostic model performs on par with task-specific models). In contrast to past research, which has often focused on predicting annotation labels from task-specific features, our model is trained to predict errors directly from a combination of task features and behavioral features derived from the annotation process, in order to achieve a high degree of generalizability. We demonstrate the usefulness of the model in the context of auditing, where prioritizing tasks with high predicted error probabilities considerably increases the amount of corrected annotation errors (e.g., 40% efficiency gains for the music streaming application). These results highlight that behavioral error detection models can yield considerable improvements in the efficiency and quality of data annotation processes. Our findings reveal critical insights into effective error management in the data annotation process, thereby contributing to the broader field of human-in-the-loop ML.
Collaborators: James Rae, Alireza Hashemi
Model Share AI
Machine learning (ML) is revolutionizing a wide range of research areas and industries, but many ML projects never progress past the proof-of-concept stage. To address this problem, we introduce Model Share AI (AIMS), a platform designed to streamline collaborative model development, model provenance tracking, and model deployment, as well as a host of other functions aiming to maximize the real-world impact of ML research. AIMS features collaborative project spaces and a standardized model evaluation process that ranks model submissions based on their performance on holdout evaluation data, enabling users to run experiments and competitions. In addition, various model metadata are automatically captured to facilitate provenance tracking and allow users to learn from and build on previous submissions. Furthermore, AIMS allows users to deploy ML models built in Scikit-Learn, TensorFlow Keras, or PyTorch into live REST APIs and automatically generated web apps with minimal code. The ability to collaboratively develop and rapidly deploy models, making them accessible to non-technical end-users through automatically generated web apps, ensures that ML projects can transition smoothly from concept to real-world application.
Collaborators: Michael Parrott
Message Response Behaviors In Context
Instant messaging plays a significant role in people's social and professional lives, but little is known about the factors shaping message response behaviors. In the present study, we investigate the determinants of message response behaviors from a predictive-explanatory perspective. Using a large and diverse sample of Snapchat users, we first show that message response times are highly predictable (AUC=0.97). Second, we employ ablation techniques to examine the contributions of several important groups of predictors: message attributes (such as message length, time sent, and location of sender, but not message content), user attributes, network communication patterns, and dyad-level communication patterns, as well as spatial and temporal context. The results indicate that dyad-specific communication patterns in conjunction with spatial and temporal context account for the largest share of explained target variance. Our findings are consistent with the idea of dyad-specific, context-contingent messaging habits, and a state-based view of message response behaviors. Our work has implications for the development of new systems and UX design. For example our models could facilitate context-aware delivery timing, message rankings, and availability status displays.
Collaborators: Ron Dotsch, Yozen Liu, Sandra Matz, Maarten Bos
Organizational Fit AND JOB Tenure
Job tenure is an important organizational outcome. It is usually in the interest of both employees and employers that individuals stay in the organization for an extended time in order to be productive and contribute in meaningful ways. A potentially important factor influencing job tenure is organizational fit, the alignment of employees’ values and beliefs with those of the organization they work for. We analyze the effects of organizational fit on job tenure using a very large corpus of user data scraped from LinkedIn. The project will help to better understand how organizational fit affects job tenure and which variables moderate this effect. For example: Is organizational fit particularly important in certain companies or industries, and who are the people that profit the most/least from organizational fit?
Collaborators: Daniel Stein, Seung-Jae Bang, Huang Ke-Wei, Sandra Matz
Predicting The SPread of COVID-19
Social behaviors and compliance behaviors play a critical role in the transmission of COVID-19. Consequently, regional variation in personality traits that capture individual differences in these behaviors may offer new insight into the spread of COVID-19. We combine self-reported personality data (3.5M people), COVID-19 prevalence and death rates, and behavioral mobility observations (29M people) to show that regional personality differences in the US and Germany predict COVID-19-related outcomes and behaviors incremental to a conservative set of socio-demographic, economic, and pandemic-related control variables. Earlier onsets of COVID-19 and steeper initial growth rates were related to higher levels of Openness and lower levels of Neuroticism. We also show that (i) regional personality is associated with objective indicators of social distancing, (ii) the effects of regional personality can change over time (Openness), and that (iii) the effects of regional personality do not always converge with those observed at the individual level (Agreeableness and Conscientiousness).
You can find the published paper here.
Collaborators: Friedrich Götz, Tobias Ebert, Sandrine Müller, Jason Rentfrow, Sam Gosling, Martin Obschonka, Jeff Potter, Sandra Matz
Investigating the Relationships between Mobility and WellBeing
People interact with their physical environments every day by visiting different places and moving between them. Such mobility behaviours likely influence and are influenced by people's subjective well‐being. However, past research examining the links between mobility behaviours and well‐being has been inconclusive. Here, we provide a comprehensive investigation of these relationships by examining individual differences in two types of mobility behaviours (movement patterns and places visited) and their relationship to six indicators of subjective well‐being (depression, loneliness, anxiety, stress, affect, and energy) at two different temporal levels of analysis (two‐week tendencies and daily level). Using data from a large smartphone‐based longitudinal study (N = 1765), we show that (i) movement patterns assessed via GPS data (distance travelled, entropy, and irregularity) and (ii) places visited assessed via experience sampling reports (home, work, and social places) are associated with subjective well‐being at the between and within person levels. Our findings suggest that distance travelled is related to anxiety, affect, and stress, irregularity is related to depression and loneliness, and spending time in social places is negatively associated with loneliness. We discuss the implications of our work and highlight directions for future research on the generalizability to other populations as well as the characteristics of places.
You can find the published paper here.
Collaborators: Sandrine Muller, Weichen Wang, Sandra Matz, Gabriella Harari
Using Sensory Substitution to Improve decision Making
Our brains are able to juggle vast amounts of information unconsciously, but we often fail to make optimal decisions when faced with conscious decisions that require us, for example, to memorize and manipulate numbers or estimate probabilities. Here, we explore whether we can overcome some of these cognitive limitations in everyday decision making by enabling people to feel data in a more holistic way and rely on intuitive rather than deliberative cognitive processes. Specifically, we propose that sensory substitution - a concept used in neuroscience to describe the process of encoding a particular type of sensory information in a way that makes this information accessible to a different sensory modality - can improve the quality of decision making. To test this idea, we are (1) developing a tactile interface that enables us to convey complex information through vibrations, and (2) running a series of rigorous experiments to demonstrate that sensory substitution can not only expand our sensory horizon, but also improve decision making in the lab and in the real world.
You can find the published paper here.
Collaborators: Moran Cerf, Sandra Matz
Measuring Spatial Reasoning Ability with Minecraft
Video games are a promising tool for the psychometric assessment of cognitive abilities. They can present novel task types and answer formats, they can record process data, and they can be highly motivating for test takers. This paper introduces the first game-based intelligence assessment implemented in Minecraft, an exceptionally popular video game with 176m copies sold. We found that abilities measured with Minecraft and conventional tests were highly correlated at the latent level (r = .72). Furthermore we found that behavioral log-data collected from the game environment was highly predictive of performance in the Minecraft test and, to a lesser extent, also predicted scores in conventional tests. We identify a number of behavioral features associated with spatial reasoning ability, demonstrating the utility of analyzing granular behavioral data in addition to traditional response formats. Overall, our findings indicate that Minecraft is a suitable platform for game-based intelligence assessment and encourage future work aiming to explore game-based problem solving tasks that would not be feasible on paper or in conventional computer-based tests.
The published paper is available here.
Collaborators: Andrew Kyngdon, David Stillwell
The Big Data Toolkit for Psychologists
I have summed up my approach to data analysis for a chapter of the APA handbook, The Psychology of Technology. It serves as a practical introduction for psychologists who want to use large data sets and data sets from nontraditional data sources in their research. First, the chapter discusses the concept of Big Data and reviews some of the theoretical challenges and opportunities that arise with the availability of ever larger amounts of data. Second, it discusses practical implications and best practices with respect to data collection, data storage, data processing, and data modeling for psychological research in the age of Big Data.
You can find the published chapter here.
Collaborators: Sam Gosling, Zachariah Marrero
