Home About us Contact
|
Home About us Contact | ||
|
|
||
Leibler Divergence (leibler + divergence)
Selected AbstractsUse of Kullback,Leibler divergence for forgettingINTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING, Issue 10 2009Miroslav Kárný Abstract Non-symmetric Kullback,Leibler divergence (KLD) measures proximity of probability density functions (pdfs). Bernardo (Ann. Stat. 1979; 7(3):686,690) had shown its unique role in approximation of pdfs. The order of the KLD arguments is also implied by his methodological result. Functional approximation of estimation and stabilized forgetting, serving for tracking of slowly varying parameters, use the reversed order. This choice has the pragmatic motivation: recursive estimator often approximates the parametric model by a member of exponential family (EF) as it maps prior pdfs from the set of conjugate pdfs (CEF) back to the CEF. Approximations based on the KLD with the reversed order of arguments preserves this property. In the paper, the approximation performed within the CEF but with the proper order of arguments of the KLD is advocated. It is applied to the parameter tracking and performance improvements are demonstrated. This practical result is of importance for adaptive systems and opens a way for improving the functional approximation. Copyright © 2008 John Wiley & Sons, Ltd. [source] Combining inflation density forecastsJOURNAL OF FORECASTING, Issue 1-2 2010Christian Kascha Abstract In this paper, we empirically evaluate competing approaches for combining inflation density forecasts in terms of Kullback,Leibler divergence. In particular, we apply a similar suite of models to four different datasets and aim at identifying combination methods that perform well throughout different series and variations of the model suite. We pool individual densities using linear and logarithmic combination methods. The suite consists of linear forecasting models with moving estimation windows to account for structural change. We find that combining densities is a much better strategy than selecting a particular model ex ante. While combinations do not always perform better than the best individual model, combinations always yield accurate forecasts and, as we show analytically, provide insurance against selecting inappropriate models. Logarithmic combinations can be advantageous, in particular if symmetric densities are preferred. Copyright © 2010 John Wiley & Sons, Ltd. [source] On-line expectation,maximization algorithm for latent data modelsJOURNAL OF THE ROYAL STATISTICAL SOCIETY: SERIES B (STATISTICAL METHODOLOGY), Issue 3 2009Olivier Cappé Summary., We propose a generic on-line (also sometimes called adaptive or recursive) version of the expectation,maximization (EM) algorithm applicable to latent variable models of independent observations. Compared with the algorithm of Titterington, this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete-data distribution. The resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the Kullback,Leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate, i.e. that of the maximum likelihood estimator. In addition, the approach proposed is also suitable for conditional (or regression) models, as illustrated in the case of the mixture of linear regressions model. [source] A superharmonic prior for the autoregressive process of the second-orderJOURNAL OF TIME SERIES ANALYSIS, Issue 3 2008Fuyuhiko Tanaka Abstract., The Bayesian estimation of the spectral density of the AR(2) process is considered. We propose a superharmonic prior on the model as a non-informative prior rather than the Jeffreys prior. Theoretically, the Bayesian spectral density estimator based on it dominates asymptotically the one based on the Jeffreys prior under the Kullback,Leibler divergence. In the present article, an explicit form of a superharmonic prior for the AR(2) process is presented and compared with the Jeffreys prior in computer simulation. [source] Bayesian Case Influence Diagnostics for Survival ModelsBIOMETRICS, Issue 1 2009Hyunsoon Cho Summary We propose Bayesian case influence diagnostics for complex survival models. We develop case deletion influence diagnostics for both the joint and marginal posterior distributions based on the Kullback,Leibler divergence (K,L divergence). We present a simplified expression for computing the K,L divergence between the posterior with the full data and the posterior based on single case deletion, as well as investigate its relationships to the conditional predictive ordinate. All the computations for the proposed diagnostic measures can be easily done using Markov chain Monte Carlo samples from the full data posterior distribution. We consider the Cox model with a gamma process prior on the cumulative baseline hazard. We also present a theoretical relationship between our case-deletion diagnostics and diagnostics based on Cox's partial likelihood. A simulated data example and two real data examples are given to demonstrate the methodology. [source] | ||||||||||