Skip to main content

2024 | OriginalPaper | Buchkapitel

5. Bayesian Inference

verfasst von : Eduardo Souza de Cursi

Erschienen in: Uncertainty Quantification with R

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This chapter presents the Bayesian approach for practical tasks, such as estimation, hypothesis testing, model or variable selection, and regression. The choice of priors is analyzed, by using Jeffreys approach and uncertainty quantification techniques. The Expectation-Maximization Algorithm is presented in this chapter. Implementations in R are given for all the topics, with examples of use.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Ash, R. (1965, reedited 2003). Information Theory. New York, NY, USA: Dover. Ash, R. (1965, reedited 2003). Information Theory. New York, NY, USA: Dover.
Zurück zum Zitat Batou, A., & Soize, C. (2013). Calculation of Lagrange Multipliers in the Construction of Maximum Entropy Distributions in High Stochastic Dimension. SIAM/ASA Journal on Uncertainty Quantification, 1(1). doi:https://doi.org/10.1137/120901386 Batou, A., & Soize, C. (2013). Calculation of Lagrange Multipliers in the Construction of Maximum Entropy Distributions in High Stochastic Dimension. SIAM/ASA Journal on Uncertainty Quantification, 1(1). doi:https://​doi.​org/​10.​1137/​120901386
Zurück zum Zitat Berger, J. O., & Bernardo, J. M. (1989, March). Estimating a Product of Means: Bayesian Analysis with Reference Priors. Journal of the American Statistical Association, 84(405), pp. 200–207. doi:https://doi.org/10.2307/2289864 Berger, J. O., & Bernardo, J. M. (1989, March). Estimating a Product of Means: Bayesian Analysis with Reference Priors. Journal of the American Statistical Association, 84(405), pp. 200–207. doi:https://​doi.​org/​10.​2307/​2289864
Zurück zum Zitat Berger, J. O., & Bernardo, J. M. (1992). On the development of reference priors. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. Smith (Ed.), Bayesian Statistics 4. Clarendon Press. Berger, J. O., & Bernardo, J. M. (1992). On the development of reference priors. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. Smith (Ed.), Bayesian Statistics 4. Clarendon Press.
Zurück zum Zitat Boltzmann, L. E. (1877 traduction 2002). On the relationship between the second main theorem of mechanical heat theory and the probability calculation with respect to the results about the heat equilibrium. Akademie der Wissenschaften in Wien Mathematisch-naturwissenschaftliche Klasse Sitzungsberichte, 2(76), 373–435. Retrieved February 14, 2023, from http://users.polytech.unice.fr/~leroux/boltztrad.pdf Boltzmann, L. E. (1877 traduction 2002). On the relationship between the second main theorem of mechanical heat theory and the probability calculation with respect to the results about the heat equilibrium. Akademie der Wissenschaften in Wien Mathematisch-naturwissenschaftliche Klasse Sitzungsberichte, 2(76), 373–435. Retrieved February 14, 2023, from http://​users.​polytech.​unice.​fr/​~leroux/​boltztrad.​pdf
Zurück zum Zitat Boltzmann, L. E. (1896). Vorlesungen ̈uber Gastheorie (Vol. 1). Leipzig: Barth. Boltzmann, L. E. (1896). Vorlesungen ̈uber Gastheorie (Vol. 1). Leipzig: Barth.
Zurück zum Zitat Boltzmann, L. E. (1995). Lectures on Gas Theory. (S. G. Brush, Trans.) New York, NY, USA: Dover. Boltzmann, L. E. (1995). Lectures on Gas Theory. (S. G. Brush, Trans.) New York, NY, USA: Dover.
Zurück zum Zitat Brillouin, L. (1956). Science and Information Theory. New York, NY, USA: Dover.CrossRef Brillouin, L. (1956). Science and Information Theory. New York, NY, USA: Dover.CrossRef
Zurück zum Zitat Clausius, R. J. (1850). Ueber die bewegende Kraft der Wärme und die Gesetze, welche sich daraus für die Wärmelehre selbst ableiten lassen. Annalen der Physik, 368–397 , 500–524.CrossRef Clausius, R. J. (1850). Ueber die bewegende Kraft der Wärme und die Gesetze, welche sich daraus für die Wärmelehre selbst ableiten lassen. Annalen der Physik, 368–397 , 500–524.CrossRef
Zurück zum Zitat Dale, A. I. (1982). Bayes or Laplace? An Examination of the Origin and Early Applications of Bayes’ Theorem. Archive for History of Exact Sciences, 27(1), pp. 23–47.CrossRef Dale, A. I. (1982). Bayes or Laplace? An Examination of the Origin and Early Applications of Bayes’ Theorem. Archive for History of Exact Sciences, 27(1), pp. 23–47.CrossRef
Zurück zum Zitat Dale, A. I. (1999). A History of Inverse Probability—From Thomas Bayes to Karl Pearson (2nd. ed.). New York: Springer.CrossRef Dale, A. I. (1999). A History of Inverse Probability—From Thomas Bayes to Karl Pearson (2nd. ed.). New York: Springer.CrossRef
Zurück zum Zitat De Finetti, B. (2017). Theory of Probability—A Critical Introductory Treatment. (A. M. Smith, Trans.) UK: John Wiley and Sons. De Finetti, B. (2017). Theory of Probability—A Critical Introductory Treatment. (A. M. Smith, Trans.) UK: John Wiley and Sons.
Zurück zum Zitat DeGroot, M. H. (2004). Optimal Statistical Decisions. New Jersey: John Wiley & Sons.CrossRef DeGroot, M. H. (2004). Optimal Statistical Decisions. New Jersey: John Wiley & Sons.CrossRef
Zurück zum Zitat Dieudonné, J. (1969). Foundations of Modern Analysis. London: Academic Press. Dieudonné, J. (1969). Foundations of Modern Analysis. London: Academic Press.
Zurück zum Zitat Doré, P., Martin, A., Abi-Zeid, I., Jousselme, A., & Maupin, P. (2011a, January). Belief functions induced by multimodalprobability density functions, an application to the search and rescue problem. RAIRO—Operations Research, 44(4), 323–343. doi:https://doi.org/10.1051/ro/2011001 Doré, P., Martin, A., Abi-Zeid, I., Jousselme, A., & Maupin, P. (2011a, January). Belief functions induced by multimodalprobability density functions, an application to the search and rescue problem. RAIRO—Operations Research, 44(4), 323–343. doi:https://​doi.​org/​10.​1051/​ro/​2011001
Zurück zum Zitat Doré, P., Osswald, C., Martin, A., Jousselme, A., & Maupin, P. (2011b). Continuous belief functions to qualify sensors performances. In W. Liu (Ed.), Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2011 (pp. 350–361). Belfast, Ireland: Springer. doi:https://doi.org/10.1007/978-3-642-22152-1_30CrossRef Doré, P., Osswald, C., Martin, A., Jousselme, A., & Maupin, P. (2011b). Continuous belief functions to qualify sensors performances. In W. Liu (Ed.), Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2011 (pp. 350–361). Belfast, Ireland: Springer. doi:https://​doi.​org/​10.​1007/​978-3-642-22152-1_​30CrossRef
Zurück zum Zitat Fort, H. (2022). Forecasting with Maximum Entropy. IOP Publishing Ltd.CrossRef Fort, H. (2022). Forecasting with Maximum Entropy. IOP Publishing Ltd.CrossRef
Zurück zum Zitat Gibbs, J. W. (1901 reprinted 1960). Elementary Principles of Statistical Mechanics. Dover. Gibbs, J. W. (1901 reprinted 1960). Elementary Principles of Statistical Mechanics. Dover.
Zurück zum Zitat Halmos, P. (1978). Measure theory. Berlin, Heidelberg, New York: : Springer Verlag. Halmos, P. (1978). Measure theory. Berlin, Heidelberg, New York: : Springer Verlag.
Zurück zum Zitat Halpern, J., & Fagin, R. (1992). Two views of belief: belief as generalized probability and belief as evidence. Artificial Intelligence,, 54, pp. 275–317.CrossRef Halpern, J., & Fagin, R. (1992). Two views of belief: belief as generalized probability and belief as evidence. Artificial Intelligence,, 54, pp. 275–317.CrossRef
Zurück zum Zitat Hartley, R. (1928, July). Transmission of Information. Bell System Technical Journal, 7(3), 535–563. Hartley, R. (1928, July). Transmission of Information. Bell System Technical Journal, 7(3), 535–563.
Zurück zum Zitat He, Y. (2013). Uncertainty Quantification and Data Fusion based on Dempster-Shafer Theory. Florida State University, Mathematics. Florida State University Libraries. He, Y. (2013). Uncertainty Quantification and Data Fusion based on Dempster-Shafer Theory. Florida State University, Mathematics. Florida State University Libraries.
Zurück zum Zitat Howson, C., & Urbach, P. (2006). Scientific reasoning : the Bayesian approach . Chicago: Open Court. Howson, C., & Urbach, P. (2006). Scientific reasoning : the Bayesian approach . Chicago: Open Court.
Zurück zum Zitat Hüllermeier, E., Kruse, R., & Hoffmann, F. (Eds.). (2010). Consonant continuous belief functions conflicts calculation. IPMU’10: Proceedings of the Computational intelligence for knowledge-based systems design, and 13th international conference on Information processing and management of uncertainty (pp. 706–715). Dortmund, Germany: Springer. Hüllermeier, E., Kruse, R., & Hoffmann, F. (Eds.). (2010). Consonant continuous belief functions conflicts calculation. IPMU’10: Proceedings of the Computational intelligence for knowledge-based systems design, and 13th international conference on Information processing and management of uncertainty (pp. 706–715). Dortmund, Germany: Springer.
Zurück zum Zitat Jaynes, E. (2003). Probability Theory: The Logic of Science. Cambridge: Cambridge University Press.CrossRef Jaynes, E. (2003). Probability Theory: The Logic of Science. Cambridge: Cambridge University Press.CrossRef
Zurück zum Zitat Jaynes, E. T. (1965). Gibbs vs. Boltzmann Entropies. American Journal of Physics, 33(5), 391–398. Jaynes, E. T. (1965). Gibbs vs. Boltzmann Entropies. American Journal of Physics, 33(5), 391–398.
Zurück zum Zitat Jeffreys, H. (1939). Theory of probability. Oxford: University Press. Jeffreys, H. (1939). Theory of probability. Oxford: University Press.
Zurück zum Zitat Kapur, J. N. (1989 revised 1993). Maximum Entropy Models in Science and Technology. New Delhi, India: Wiley Eastern. Kapur, J. N. (1989 revised 1993). Maximum Entropy Models in Science and Technology. New Delhi, India: Wiley Eastern.
Zurück zum Zitat Khinchin, A. Y. (1957). Mathematical Foundations of Information Theory. New York, NY, USA: Dover. Khinchin, A. Y. (1957). Mathematical Foundations of Information Theory. New York, NY, USA: Dover.
Zurück zum Zitat Klopotek, M. A., & Wierzchon, S. T. (1998). A New Qualitative Rough-Set Approach to Modeling Belief Functions. In L. Polkowski, & A. Skowron (Ed.), Rough Sets and Current Trends in Computing, First International Conference, RSCTC’98 (pp. 346–354). Warzsaw, Poland: Springer. doi:https://doi.org/10.1007/3-540-69115-4_47CrossRef Klopotek, M. A., & Wierzchon, S. T. (1998). A New Qualitative Rough-Set Approach to Modeling Belief Functions. In L. Polkowski, & A. Skowron (Ed.), Rough Sets and Current Trends in Computing, First International Conference, RSCTC’98 (pp. 346–354). Warzsaw, Poland: Springer. doi:https://​doi.​org/​10.​1007/​3-540-69115-4_​47CrossRef
Zurück zum Zitat Kullback, S. (1951 reed. 1969 reprint 1979). Information Theory and Statistics. New York: Wiley, reed. Dover. Kullback, S. (1951 reed. 1969 reprint 1979). Information Theory and Statistics. New York: Wiley, reed. Dover.
Zurück zum Zitat Laplace, P.-S. (1774). Memoire sur la probabilité des causes par les événements. Memoires de Mathématique et de Physique, Presentés à l’Académie Royale des Sciences par divers Savans & lus dans ses Assemblées, pp. 621–656. Laplace, P.-S. (1774). Memoire sur la probabilité des causes par les événements. Memoires de Mathématique et de Physique, Presentés à l’Académie Royale des Sciences par divers Savans & lus dans ses Assemblées, pp. 621–656.
Zurück zum Zitat Laplace, P.-S. (1891). Oeuvres complètes (Vol. 8). (A. d. Paris, Ed.) Paris: Gauthiers-Villars. Laplace, P.-S. (1891). Oeuvres complètes (Vol. 8). (A. d. Paris, Ed.) Paris: Gauthiers-Villars.
Zurück zum Zitat Lian, C. (2017). Information Fusion and Decision Making using Belief Functions. Compiègne, France: Université de Technologie de Compiègne. Lian, C. (2017). Information Fusion and Decision Making using Belief Functions. Compiègne, France: Université de Technologie de Compiègne.
Zurück zum Zitat Muñoz-Cobo, J.-L., Mendizábal, R., Miquel, A., Berna, C., & Escrivá, A. (2017). Use of the Principles of Maximum Entropy and Maximum Relative Entropy for the Determination of Uncertain Parameter Distributions in Engineering Applications. Entropy, 19(9). doi:https://doi.org/10.3390/e19090486 Muñoz-Cobo, J.-L., Mendizábal, R., Miquel, A., Berna, C., & Escrivá, A. (2017). Use of the Principles of Maximum Entropy and Maximum Relative Entropy for the Determination of Uncertain Parameter Distributions in Engineering Applications. Entropy, 19(9). doi:https://​doi.​org/​10.​3390/​e19090486
Zurück zum Zitat Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine Series 5, 50(302), pp. 157–175. doi:https://doi.org/10.1080/14786440009463897 Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine Series 5, 50(302), pp. 157–175. doi:https://​doi.​org/​10.​1080/​1478644000946389​7
Zurück zum Zitat Planck, M. K. (1914). The Theory of Heat Radiation. (M. Masius, Trans.) Philadelphia: P. Blakiston’s Son & Co. Planck, M. K. (1914). The Theory of Heat Radiation. (M. Masius, Trans.) Philadelphia: P. Blakiston’s Son & Co.
Zurück zum Zitat Press, S. J., & Tanur, J. M. (2001). The Subjectivity of Scientists and the Bayesian Approach. New York: John Wiley & Sons.CrossRef Press, S. J., & Tanur, J. M. (2001). The Subjectivity of Scientists and the Bayesian Approach. New York: John Wiley & Sons.CrossRef
Zurück zum Zitat Rahman, A., Gao, J., D’Este, C., & Ahmed, S. E. (2016, September). An Assessment of the Effects of Prior Distributions on the Bayesian Predictive Inference. International Journal of Statistics and Probability, 5(5), 31–42. doi:https://doi.org/10.5539/ijsp.v5n5p31 Rahman, A., Gao, J., D’Este, C., & Ahmed, S. E. (2016, September). An Assessment of the Effects of Prior Distributions on the Bayesian Predictive Inference. International Journal of Statistics and Probability, 5(5), 31–42. doi:https://​doi.​org/​10.​5539/​ijsp.​v5n5p31
Zurück zum Zitat Rao, C. R. (1948, January). Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 44(1), pp. 50–57 . doi:https://doi.org/10.1017/S0305004100023987 Rao, C. R. (1948, January). Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 44(1), pp. 50–57 . doi:https://​doi.​org/​10.​1017/​S030500410002398​7
Zurück zum Zitat Rényi, A. (1961). On Measures of Entropy and Information. In J. Neyman (Ed.), 4th Berkeley Symposium on Mathematics, Statistics and Probability,. 1, pp. 547–561. University of California Press. Rényi, A. (1961). On Measures of Entropy and Information. In J. Neyman (Ed.), 4th Berkeley Symposium on Mathematics, Statistics and Probability,. 1, pp. 547–561. University of California Press.
Zurück zum Zitat Risti, B., & Smets, P. (2006). Belief function theory on the continuous space with an application to model based classification. In B. Bouchon-Meunier, G. Coletti, & R. R. Yager (Ed.), Modern Information Processing: From Theory to Applications. IPMU’04 (pp. 11–24). Perugia, Italy.: Elsevier. doi:https://doi.org/10.1016/B978-044452075-3/50002-9CrossRef Risti, B., & Smets, P. (2006). Belief function theory on the continuous space with an application to model based classification. In B. Bouchon-Meunier, G. Coletti, & R. R. Yager (Ed.), Modern Information Processing: From Theory to Applications. IPMU’04 (pp. 11–24). Perugia, Italy.: Elsevier. doi:https://​doi.​org/​10.​1016/​B978-044452075-3/​50002-9CrossRef
Zurück zum Zitat Savchuk, V. P., & Tsokos, C. P. (2011). Bayesian Theory and Methods with Applications. Paris: Atlantis Press.CrossRef Savchuk, V. P., & Tsokos, C. P. (2011). Bayesian Theory and Methods with Applications. Paris: Atlantis Press.CrossRef
Zurück zum Zitat Shafer, G. (1976). A Mathematical Theory of Evidence . New Jersey: Princeton University Press.CrossRef Shafer, G. (1976). A Mathematical Theory of Evidence . New Jersey: Princeton University Press.CrossRef
Zurück zum Zitat Soize, C. (2008). Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices. Int. J. Numer. Meth. Engng, 76, 1583–1611. doi:https://doi.org/10.1002/nme.2385CrossRef Soize, C. (2008). Construction of probability distributions in high dimension using the maximum entropy principle: Applications to stochastic processes, random fields and random matrices. Int. J. Numer. Meth. Engng, 76, 1583–1611. doi:https://​doi.​org/​10.​1002/​nme.​2385CrossRef
Zurück zum Zitat Souza de Cursi, E. (2023). Uncertainty Quantification using R. Springer Cham.CrossRef Souza de Cursi, E. (2023). Uncertainty Quantification using R. Springer Cham.CrossRef
Zurück zum Zitat Stigler, S. M. (1982). Thomas Bayes’s Bayesian Inference. Journal of the Royal Statistical Society. Series A (General), 145(2), pp. .250–258. Stigler, S. M. (1982). Thomas Bayes’s Bayesian Inference. Journal of the Royal Statistical Society. Series A (General), 145(2), pp. .250–258.
Zurück zum Zitat Taillandier, P., & Therond, O. (2011). Use of the Belief Theory to formalize Agent DecisionMaking Processes : Application to cropping Plan Decision Making. European Simulation and Modelling Conference, (pp. 138–142). Guimaraes, Portugal. Retrieved 12 17, 2022, from https://hal.archives-ouvertes.fr/hal-00688405 Taillandier, P., & Therond, O. (2011). Use of the Belief Theory to formalize Agent DecisionMaking Processes : Application to cropping Plan Decision Making. European Simulation and Modelling Conference, (pp. 138–142). Guimaraes, Portugal. Retrieved 12 17, 2022, from https://​hal.​archives-ouvertes.​fr/​hal-00688405
Zurück zum Zitat Wald, A. (1943, November). Tests of Statistical Hypotheses concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54(3), pp. 426–482. doi:https://doi.org/10.2307/1990256 Wald, A. (1943, November). Tests of Statistical Hypotheses concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54(3), pp. 426–482. doi:https://​doi.​org/​10.​2307/​1990256
Zurück zum Zitat Winkler, R. L. (1966). The assessment of prior distributions in Bayesian analysis. Chicago, IL, USA: University of Chicago. Winkler, R. L. (1966). The assessment of prior distributions in Bayesian analysis. Chicago, IL, USA: University of Chicago.
Zurück zum Zitat Zanchini, E., & Beretta, G. P. (2008). Rigorous Axiomatic Definition of Entropy Valid Also for Non-Equilibrium States. In G. P. Beretta, A. Ghoniem, & G. Hatsopoulos (Ed.), MEETING THE ENTROPY CHALLENGE: An International Thermodynamics Symposium in Honor and Memory of Professor Joseph H. Keenan. 1033. Cambridge, MA, USA: AIP Conference Proceedings. doi:https://doi.org/10.1063/1.2979048 Zanchini, E., & Beretta, G. P. (2008). Rigorous Axiomatic Definition of Entropy Valid Also for Non-Equilibrium States. In G. P. Beretta, A. Ghoniem, & G. Hatsopoulos (Ed.), MEETING THE ENTROPY CHALLENGE: An International Thermodynamics Symposium in Honor and Memory of Professor Joseph H. Keenan. 1033. Cambridge, MA, USA: AIP Conference Proceedings. doi:https://​doi.​org/​10.​1063/​1.​2979048
Zurück zum Zitat Zellner, A. (1986b). On Assessing Prior Distributions and Bayesian Regression Analysis with g Prior Distributions. In P. Goel, & A. Zellner (Eds.), Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti. (pp. 233–243). Elsevier. Zellner, A. (1986b). On Assessing Prior Distributions and Bayesian Regression Analysis with g Prior Distributions. In P. Goel, & A. Zellner (Eds.), Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti. (pp. 233–243). Elsevier.
Metadaten
Titel
Bayesian Inference
verfasst von
Eduardo Souza de Cursi
Copyright-Jahr
2024
DOI
https://doi.org/10.1007/978-3-031-48208-3_5

Premium Partner