IADIS International Journal on Computer Science and Information Systems

Published by IADIS (International Association for Development of the Information Society) • ISSN (Online): 1646-3692 • ISSN (Print): 1646-3692
100% Open Access
Double-Blind Peer Review
Crossref DOI Persistent IDs
Open Access Peer-Reviewed Original Research

Calculating the Normalized Maximum Likelihood Distribution for Bayesian Forests

Hannes Wettig *
Petri Kontkanen *
Petri Myllymäki Complex Systems Computation *
* Group (CoSCo), Helsinki Institute for Information Technology (HIIT), University of Helsinki and Helsinki University of Technology, P.O.Box 68 (Department of Computer Science), FIN-00014 University of Helsinki, Finland {Firstname}.{Lastname}@hiit.fi (Portugal)
* Group (CoSCo), Helsinki Institute for Information Technology (HIIT), University of Helsinki and Helsinki University of Technology, P.O.Box 68 (Department of Computer Science), FIN-00014 University of Helsinki, Finland {Firstname}.{Lastname}@hiit.fi (Portugal)
* Group (CoSCo), Helsinki Institute for Information Technology (HIIT), University of Helsinki and Helsinki University of Technology, P.O.Box 68 (Department of Computer Science), FIN-00014 University of Helsinki, Finland {Firstname}.{Lastname}@hiit.fi (Portugal)

Abstract

When learning Bayesian network structures from sample data, an important issue is how to evaluate the goodness of alternative network structures. Perhaps the most commonly used model (class) selection criterion is the marginal likelihood, which is obtai ned by integrating over a prior distribution for the model parameters. However, the problem of determining a reasonable prior for the parameters is a highly controversial issue, and no comple tely satisfying Bayesian solution has yet been presented in the non- informative setting. The normalized maximum lik elihood (NML), based on Rissanen's information- theoretic MDL methodology, offers an alternative, th eoretically solid criterion that is objective and non- informative, while no parameter prior is required. It has been previously shown that for discrete data, this criterion can be computed in linear time for Bayesian networks with no arcs, and in quadratic time for the so called Naive Bayes network structure. Here we extend the previous results by showing how to compute the NML criterion in polynomial time for tree- structured Bayesian networks. The order of the polynomial depends on the number of values of the variables, but neither on the number of variables itself, nor on the sample size1.

Keywords

Machine Learning Bayesian Networks Minimu m Description Length Normalized Maximum Likelihood.
Full-Text PDF Available

Read Complete Peer-Reviewed Manuscript

Includes full econometric models, data tables, policy recommendations, declarations, and citations.

Declarations & Ethics

Funding: This research received academic dissemination support through ESCAP / JournalsHub publishing programs.
Conflicts of Interest: The authors declare no competing financial or institutional interests.
Peer Review: Double-blind peer reviewed by international subject specialists.
License: Creative Commons Attribution 4.0 International (CC BY 4.0).
How to Cite This Article
APA / MLA / BibTeX
Wettig, et al. (2007). Calculating the Normalized Maximum Likelihood Distribution for Bayesian Forests. IADIS International Journal on Computer Science and Information Systems, 2(2). https://doi.org/10.33965/ijcsis_2007_v2i2_02
Wettig, et al. "Calculating the Normalized Maximum Likelihood Distribution for Bayesian Forests." IADIS International Journal on Computer Science and Information Systems, vol. 2, no. 2, 2007. https://doi.org/10.33965/ijcsis_2007_v2i2_02
Wettig, et al. "Calculating the Normalized Maximum Likelihood Distribution for Bayesian Forests." IADIS International Journal on Computer Science and Information Systems 2, no. 2 (2007). https://doi.org/10.33965/ijcsis_2007_v2i2_02