Projects: Statistical Relational Learning (SRL)

Much has been achieved in the field of AI, yet much remains to be done if we are to reach the goals we all imagine. One of the key challenges with moving ahead is closing the gap between logical and statistical AI. Logical AI has mainly focused on complex representations and statistical AI on uncertainty. However, intelligent agents must be able to handle both the complexity and the uncertainty of the real world. Relational Probabilistic approaches have been developed that seek to avoid explicit state enumeration as is traditionally done in statistical learning through a symbolic representation of states. There have been several previous SRL workshops on these representations, and the most recent one was a workshop on Statistical Relational AI at AAAI 2010. Our group pursues this research through the two different aspects of SRL models - inference and learning.

Anytime Lifted Belief Propagation (ALBP)

Lifted inference on first-order probabilistic models has been receiving increasing attention recently. It is inference that manipulates and maintains the first-order structure which avoids extensive propositionalization. To date, all lifted inference methods require a model to be shattered against itself and evidence before inference starts. The reason shattering is needed in advance is because the algorithms that have been lifted (belief propagation and variable elimination) require the entire model in order to compute a query's belief; using the entire model requires it to be completely shattered beforehand. In this work, we develop an algorithm in which reasoning only considers sub- or individual cases on an as-needed basis. This parallels what is done in theorem proving where unification and resolution are gradually used.
  • Members involved:
  • Rick Freedman and Sriraam Natarajan
  • Other collaborators:
  • Dr. Rodrigo de Salvo Braz (SRI), Dr. Hung Bui (SRI), and Dr. Jude Shavlik (University of Wisconsin, Madison)
  • Publications:
  • Rodrigo De Salvo Braz, Sriraam Natarajan, Hung Bui, Jude Shavlik, and Stuart Russell. Anytime Lifted Belief Propagation. International Workshop in SRL 2009.

Boosting SRL Models

While SRL models are highly attractive due to their compactness and comprehensibility, the problem of learning in these models is computationally intensive. Structured learning is an active research area in SRL. Most approaches first learn a few rules and then learn the parameters (or weights) for these rules. We take a slightly different approach based on Friedman's functional gradient boosting algorithm by learning the parameters and structure of SRL models simultaneously. The key idea is to consider the target potential function as a series of relational regression trees learned in a stage-wise manner. We have successfully applied this algorithm to learning Relational Dependency Networks and Markov Logic Networks.
  • Members involved:
  • Kristian Kersting, Tushar Khot, and Sriraam Natarajan
  • Other collaborators:
  • Dr. Jude Shavlik (University of Wisconsin, Madison)
  • Publications:
  • Sriraam Natarajan, Tushar Khot, Kristian Kersting, Bernd Gutmann and Jude Shavlik. Gradient-based Boosting for Statistical Relational Learning: The Relational Dependency Network Case, Invited contribution to special issue of Machine Learning Journal (MLJ), 2011.
  • Sriraam Natarajan, Tushar Khot, Kristian Kersting, Bernd Gutmann and Jude Shavlik. Boosting Relational Dependency Networks, International Conference on Inductive Logic Programming (ILP) 2010.

Machine Reading

Our team is involved in the Machine Reading project where the goal is to automatically read text, construct a knowledge base and reason about specific queries. We are part of the FAUST group that is led by SRI international. We work closely with Prof. Jude Shavlik's group at Wisconsin in building SRL models for NLP reasoning.
  • Members involved:
  • Ryan Barnard, Jose Picado, and Sriraam Natarajan
  • Publications to be updated soon.