Search Papers | Poster Sessions | All Posters
Poster B109 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Climbing the Ladder of Causation with Counterfactual World Modeling
Rahul Mysore Vekatesh1, Honglin Chen1, Klemen Kotar1, Kevin Feigelis1, Wanhee Lee1, Daniel Bear1, Daniel Yamins1; 1Stanford University
While language models have begun to show signs of understanding causal relationships, vision models seem to lag behind. We introduce Counterfactual World Modeling (CWM) — a visual world model trained for future prediction that demonstrates capabilities analogous to various levels of Pearl's "Ladder of Causation". A key finding of this paper is that mid-level vision structures can be formulated as counterfactual queries to CWM, enabling their extraction under a unified, self-supervised architecture. This not only moves closer to a human-like learning process, but also reduces the reliance on expensive annotated datasets for training task-specific models — a long-standing predicament in computer vision.
Keywords: Mid-level vision counterfactuals deep learning transformers