Bayesian or democratic voting: which one wins?

The theory

Bayesian model weighting is a well know and well analysed technique used for making decisions based on data D. A number of models M₁...M_n are considered and for each model M_i, P(M_i|D) the probability of the model given the data is calculated. Each of the models are then used for making a prediction f₁...f_n and each prediction is weighted with the probability P(M_i|D) giving as final prediction F=C å_i P(M_i|D).f_i where C is a normalizing constant C = å_i P(M_i|D).

(See the Bayesian Model Averaging page for more details.)

The practice

It was recently reported in an article Bayesian Averaging of Classifiers and the Overfitting Problem by Pedro Domingos that Bayesian model weighting may be outperformed in cross validation experiments by democratic voting. In democratic voting, the final prediction does not take the probability P(M_i|D) into account but instead gives each model a single vote: one model one vote! So, democratic voting gives a prediction based on F=(1/n) å_i f_i.

The problem

Can this emperical result be verified in other areas of model selection?

Can this emperical result be explained by looking at emperical results in other areas of model selection?

webmaster