Outline

1. Part 1: The Phisolophical Multiverse approach
   

2. Part 2: MB2-Pupillometry Spin-Off Case Study

Thanks so much to these incredibly collaborative people

Co-authors:
Melanie S. Schreiner • Alvin Tan • Marlena Mayer • Robert Hepach and the MB2-P Team

What is Multiverse Analysis?

• “A systematic exploration of all reasonable analytical paths” (Steegen et al., 2016)
  
• Enhances transparency and systematically organizes researcher degrees of freedom
  
• A garden of analytical forking paths: grounded in theory

Why It Matters for Developmental Science

• Universal Applicability: works across contexts and methods
  
• Flexible Implementation: supports exploratory and confirmatory designs
  
• Credibility Enhancement: addresses replication challenges
  
• ManyBabies Alignment: fits with goals of transparency and rigor

Guiding Principles

• Think Before Testing
  Evaluate choices before collecting data.

• Theoretical Grounding
  Base decisions on prior evidence.

• Complete Transparency
  Present all plausible paths.

• Computational Implementation
  Use tools to explore the full analytic space.

Types of Multiverse Approaches

Exploratory Methods

• Vibration of Effects (VoE)
  
• Summary indices
  
• Graphical representations

Focus: Identify patterns and generate hypotheses.

Inferential Methods

• PIMA (Post-selection Inference)
• Sign-flipping score test

Focus: Maintain statistical rigor while exploring variation.

Addressing the Credibility Crisis

Problem: P-hacking

• Selective analysis for significance undermines integrity.

Problem: HARKing

• Hypothesizing after results are known (post hoc as a priori).

Solutions via Multiverse Analysis

• Multiverse Transparency
  Makes all choices visible to prevent selective reporting.

• “Legal P-hacking”
  Exploring multiple paths is valid if systematic and transparent.

The Multiverse Landscape

• Data Inclusion/Exclusion: who/what to include?
  
• Preprocessing Methods: cleaning, transforming
  
• Variable Operationalization: defining constructs
  
• Statistical Models: which methods and why

Visualizing the Multiverse

Specification Curve

Volcano Plot

Volcano Plot

Study Design Overview

• Cross-lab pupillometry with over 30 labs worldwide
• Age cohorts: toddlers and adults
• Condition × Outcome × Time modeling
• From: Calignano, Girardi, Altoè., 2024 First steps into the pupillometry multiverse of developmental science. Behav Res Methods.

ManyBabies 2: Theory of Mind in Infancy

MB2 – Do toddlers and adults engage in spontaneous Theory of Mind (ToM)?

MB2 – ToM: Paradigm

MB2 – ToM