“Capturing Higher-order Relationships through Information Decomposition”
Thesis lab: Andrew Clark (WashU Electrical & Systems Engineering)
Abstract: Mutual information between two random variables is a well-studied notion. But for the relationship between one random variable and a pair of other random variables, mutual information cannot capture their fine-grained interactions, resulting in limited insights into complex systems. To solve this problem, Williams and Beer proposed a conceptual framework called Partial Information Decomposition (PID) with several operational axioms to decompose this mutual information to fine-grained information atoms. To explore a wider range of higher-order relationships based on this conceptual framework, we provide the first explicit formula for calculating the information atoms so that PID axioms and additional properties from subsequent studies are satisfied. In this work, we introduced a do-operation, an operation over the variable system which sets a certain marginal to a desired value, which is distinct from any existing approaches. In addition, we also expanded PID from decomposing the directional mutual information to the system’s whole entropy, which can efficiently capture all interactions among variables, termed System Information Decomposition (SID). These works extend current information decomposition techniques and provide a promising framework for understanding complex higher-order relationships within interdisciplinary systems.
For inquiries contact Aaron Beagle at abeagle@wustl.edu.