Bayesian Data Analysis for Ecology

67 %
33 %
Information about Bayesian Data Analysis for Ecology
Education

Published on July 20, 2009

Author: xianblog

Source: slideshare.net

Description

Those are the slides made for the course given in the Gran Paradiso National Park on July 13-17, based on the original slides for the three first chapters of Bayesian Core. The specific datasets used to illustrate those slides are not publicly available.

Bayesian data analysis for Ecologists Bayesian data analysis for Ecologists Christian P. Robert, Universit´ Paris Dauphine e Parco Nazionale Gran Paradiso, 13-17 Luglio 2009 1 / 427

Bayesian data analysis for Ecologists Outline 1 The normal model 2 Regression and variable selection 3 Generalized linear models 2 / 427

Bayesian data analysis for Ecologists The normal model The normal model 1 The normal model Normal problems The Bayesian toolbox Prior selection Bayesian estimation Confidence regions Testing Monte Carlo integration Prediction 3 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Normal model Sample x1 , . . . , xn from a normal N (µ, σ 2 ) distribution normal sample 0.6 0.5 0.4 Density 0.3 0.2 0.1 0.0 −2 −1 0 1 2 4 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) 5 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) Confidence region [interval] on (µ, σ) 6 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Inference on (µ, σ) based on this sample Estimation of [transforms of] (µ, σ) Confidence region [interval] on (µ, σ) Test on (µ, σ) and comparison with other samples 7 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Datasets Larcenies =normaldata 4 Relative changes in reported 3 larcenies between 1991 and 1995 (relative to 1991) for 2 the 90 most populous US 1 counties (Source: FBI) 0 −0.4 −0.2 0.0 0.2 0.4 0.6 8 / 427

Bayesian data analysis for Ecologists The normal model Normal problems Marmot weights 0.0015 0.0010 Marmots (72) separated by age Density between adults and others, with normal estimation 0.0005 0.0000 0 1000 2000 3000 4000 5000 Weight 9 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox The Bayesian toolbox Bayes theorem = Inversion of probabilities 10 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox The Bayesian toolbox Bayes theorem = Inversion of probabilities If A and E are events such that P (E) = 0, P (A|E) and P (E|A) are related by P (E|A)P (A) P (A|E) = P (E|A)P (A) + P (E|Ac )P (Ac ) P (E|A)P (A) = P (E) 11 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Who’s Bayes? Reverend Thomas Bayes (ca. 1702–1761) Presbyterian minister in Tunbridge Wells (Kent) from 1731, son of Joshua Bayes, nonconformist minister. Election to the Royal Society based on a tract of 1736 where he defended the views and philosophy of Newton. 12 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Who’s Bayes? Reverend Thomas Bayes (ca. 1702–1761) Presbyterian minister in Tunbridge Wells (Kent) from 1731, son of Joshua Bayes, nonconformist minister. Election to the Royal Society based on a tract of 1736 where he defended the views and philosophy of Newton. Sole probability paper, “Essay Towards Solving a Problem in the Doctrine of Chances”, published posthumously in 1763 by Pierce and containing the seeds of Bayes’ Theorem. 13 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox New perspective Uncertainty on the parameters θ of a model modeled through a probability distribution π on Θ, called prior distribution 14 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox New perspective Uncertainty on the parameters θ of a model modeled through a probability distribution π on Θ, called prior distribution Inference based on the distribution of θ conditional on x, π(θ|x), called posterior distribution f (x|θ)π(θ) π(θ|x) = . f (x|θ)π(θ) dθ 15 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Bayesian model A Bayesian statistical model is made of 1 a likelihood f (x|θ), 16 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Bayesian model A Bayesian statistical model is made of 1 a likelihood f (x|θ), and of 2 a prior distribution on the parameters, π(θ) . 17 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ 18 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x 19 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x Allows incorporation of imperfect/imprecise information in the decision process 20 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Justifications Semantic drift from unknown θ to random θ Actualization of information/knowledge on θ by extracting information/knowledge on θ contained in the observation x Allows incorporation of imperfect/imprecise information in the decision process Unique mathematical way to condition upon the observations (conditional perspective) 21 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (σ 2 = 1)) Assume π(θ) = exp{−θ} Iθ>0 22 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (σ 2 = 1)) Assume π(θ) = exp{−θ} Iθ>0 Then π(θ|x1 , . . . , xn ) ∝ exp{−θ} exp{−n(θ − x)2 /2} Iθ>0 ∝ exp −nθ2 /2 + θ(nx − 1) Iθ>0 ∝ exp −n(θ − (x − 1/n))2 /2 Iθ>0 23 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal illustration (2)) Truncated normal distribution N + ((x − 1/n), 1/n) mu=−0.08935864 2.5 2.0 1.5 1.0 0.5 0.0 0.0 0.1 0.2 0.3 0.4 0.5 24 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Prior and posterior distributions Given f (x|θ) and π(θ), several distributions of interest: 1 the joint distribution of (θ, x), ϕ(θ, x) = f (x|θ)π(θ) ; 25 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Prior and posterior distributions Given f (x|θ) and π(θ), several distributions of interest: 1 the joint distribution of (θ, x), ϕ(θ, x) = f (x|θ)π(θ) ; 2 the marginal distribution of x, m(x) = ϕ(θ, x) dθ = f (x|θ)π(θ) dθ ; 26 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox 3 the posterior distribution of θ, f (x|θ)π(θ) π(θ|x) = f (x|θ)π(θ) dθ f (x|θ)π(θ) = ; m(x) 4 the predictive distribution of y, when y ∼ g(y|θ, x), g(y|x) = g(y|θ, x)π(θ|x)dθ . 27 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations 28 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x 29 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x 30 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x Coherent updating of the information available on θ, independent of the order in which i.i.d. observations are collected 31 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Posterior distribution center of Bayesian inference π(θ|x) ∝ f (x|θ) π(θ) Operates conditional upon the observations Integrate simultaneously prior information/knowledge and information brought by x Avoids averaging over the unobserved values of x Coherent updating of the information available on θ, independent of the order in which i.i.d. observations are collected Provides a complete inferential scope and an unique motor of inference 32 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal-normal case) Consider x|θ ∼ N (θ, 1) and θ ∼ N (a, 10). (x − θ)2 (θ − a)2 π(θ|x) ∝ f (x|θ)π(θ) ∝ exp − − 2 20 11θ2 ∝ exp − + θ(x + a/10) 20 11 ∝ exp − {θ − ((10x + a)/11)}2 20 33 / 427

Bayesian data analysis for Ecologists The normal model The Bayesian toolbox Example (Normal-normal case) Consider x|θ ∼ N (θ, 1) and θ ∼ N (a, 10). (x − θ)2 (θ − a)2 π(θ|x) ∝ f (x|θ)π(θ) ∝ exp − − 2 20 11θ2 ∝ exp − + θ(x + a/10) 20 11 ∝ exp − {θ − ((10x + a)/11)}2 20 and θ|x ∼ N (10x + a) 11, 10 11 34 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference 35 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference But... In practice, it seldom occurs that the available prior information is precise enough to lead to an exact determination of the prior distribution 36 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Prior selection The prior distribution is the key to Bayesian inference But... In practice, it seldom occurs that the available prior information is precise enough to lead to an exact determination of the prior distribution There is no such thing as the prior distribution! 37 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 38 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram 39 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π 40 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π Use the marginal distribution of x, m(x) = f (x|θ)π(θ) dθ Θ 41 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Strategies for prior determination Ungrounded prior distributions produce unjustified posterior inference. —Anonymous, ca. 2006 Use a partition of Θ in sets (e.g., intervals), determine the probability of each set, and approach π by an histogram Select significant elements of Θ, evaluate their respective likelihoods and deduce a likelihood curve proportional to π Use the marginal distribution of x, m(x) = f (x|θ)π(θ) dθ Θ Empirical and hierarchical Bayes techniques 42 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Conjugate priors Specific parametric family with analytical properties Conjugate prior A family F of probability distributions on Θ is conjugate for a likelihood function f (x|θ) if, for every π ∈ F, the posterior distribution π(θ|x) also belongs to F. 43 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Conjugate priors Specific parametric family with analytical properties Conjugate prior A family F of probability distributions on Θ is conjugate for a likelihood function f (x|θ) if, for every π ∈ F, the posterior distribution π(θ|x) also belongs to F. Only of interest when F is parameterised : switching from prior to posterior distribution is reduced to an updating of the corresponding parameters. 44 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) 45 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations 46 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... 47 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... tractability and simplicity 48 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications Limited/finite information conveyed by x Preservation of the structure of π(θ) Exchangeability motivations Device of virtual past observations Linearity of some estimators But mostly... tractability and simplicity First approximations to adequate priors, backed up by robustness analysis 49 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Exponential families Sampling models of interest Exponential family The family of distributions f (x|θ) = C(θ)h(x) exp{R(θ) · T (x)} is called an exponential family of dimension k. When Θ ⊂ Rk , X ⊂ Rk and f (x|θ) = h(x) exp{θ · x − Ψ(θ)}, the family is said to be natural. 50 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) 51 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) 52 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) Analyticity (E[x] = ∇Ψ(θ), ...) 53 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Analytical properties of exponential families Sufficient statistics (Pitman–Koopman Lemma) Common enough structure (normal, Poisson, &tc...) Analyticity (E[x] = ∇Ψ(θ), ...) Allow for conjugate priors π(θ|µ, λ) = K(µ, λ) eθ.µ−λΨ(θ) λ>0 54 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Standard exponential families f (x|θ) π(θ) π(θ|x) Normal Normal N (θ, σ 2 ) N (µ, τ 2 ) N (ρ(σ 2 µ + τ 2 x), ρσ 2 τ 2 ) ρ−1 = σ 2 + τ 2 Poisson Gamma P(θ) G(α, β) G(α + x, β + 1) Gamma Gamma G(ν, θ) G(α, β) G(α + ν, β + x) Binomial Beta B(n, θ) Be(α, β) Be(α + x, β + n − x) 55 / 427

Bayesian data analysis for Ecologists The normal model Prior selection More... f (x|θ) π(θ) π(θ|x) Negative Binomial Beta N eg(m, θ) Be(α, β) Be(α + m, β + x) Multinomial Dirichlet Mk (θ1 , . . . , θk ) D(α1 , . . . , αk ) D(α1 + x1 , . . . , αk + xk ) Normal Gamma N (µ, 1/θ) Ga(α, β) G(α + 0.5, β + (µ − x)2 /2) 56 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Linearity of the posterior mean If θ ∼ πλ,µ (θ) ∝ eθ·µ−λΨ(θ) with µ ∈ X , then µ Eπ [∇Ψ(θ)] = . λ where ∇Ψ(θ) = (∂Ψ(θ)/∂θ1 , . . . , ∂Ψ(θ)/∂θp ) 57 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Linearity of the posterior mean If θ ∼ πλ,µ (θ) ∝ eθ·µ−λΨ(θ) with µ ∈ X , then µ Eπ [∇Ψ(θ)] = . λ where ∇Ψ(θ) = (∂Ψ(θ)/∂θ1 , . . . , ∂Ψ(θ)/∂θp ) Therefore, if x1 , . . . , xn are i.i.d. f (x|θ), µ + n¯ x Eπ [∇Ψ(θ)|x1 , . . . , xn ] = . λ+n 58 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Example (Normal-normal) In the normal N (θ, σ 2 ) case, conjugate also normal N (µ, τ 2 ) and Eπ [∇Ψ(θ)|x] = Eπ [θ|x] = ρ(σ 2 µ + τ 2 x) where ρ−1 = σ 2 + τ 2 59 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Example (Full normal) In the normal N (µ, σ 2 ) case, when both µ and σ are unknown, there still is a conjugate prior on θ = (µ, σ 2 ), of the form (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 60 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Example (Full normal) In the normal N (µ, σ 2 ) case, when both µ and σ are unknown, there still is a conjugate prior on θ = (µ, σ 2 ), of the form (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 since π(µ, σ 2 |x1 , . . . , xn ) ∝ (σ 2 )−λσ exp − λµ (µ − ξ)2 + α /2σ 2 ×(σ 2 )−n/2 exp − n(µ − x)2 + s2 /2σ 2 x ∝ (σ 2 )−λσ −n/2 exp − (λµ + n)(µ − ξx )2 nλµ (x − ξ)2 +α + s2 + x /2σ 2 n + λµ 61 / 427

Bayesian data analysis for Ecologists The normal model Prior selection parameters (0,1,1,1) 2.0 1.5 σ 1.0 0.5 −1.0 −0.5 0.0 0.5 1.0 µ 62 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Improper prior distribution Extension from a prior distribution to a prior σ-finite measure π such that π(θ) dθ = +∞ Θ 63 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Improper prior distribution Extension from a prior distribution to a prior σ-finite measure π such that π(θ) dθ = +∞ Θ Formal extension: π cannot be interpreted as a probability any longer 64 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 65 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 66 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 67 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 4 More robust answer against possible misspecifications of the prior 68 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Justifications 1 Often only way to derive a prior in noninformative/automatic settings 2 Performances of associated estimators usually good 3 Often occur as limits of proper distributions 4 More robust answer against possible misspecifications of the prior 5 Improper priors (infinitely!) preferable to vague proper priors such as a N (0, 1002 ) distribution [e.g., BUGS] 69 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Validation Extension of the posterior distribution π(θ|x) associated with an improper prior π given by Bayes’s formula f (x|θ)π(θ) π(θ|x) = , Θ f (x|θ)π(θ) dθ when f (x|θ)π(θ) dθ < ∞ Θ 70 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Example (Normal+improper) If x ∼ N (θ, 1) and π(θ) = ̟, constant, the pseudo marginal distribution is +∞ 1 m(x) = ̟ √ exp −(x − θ)2 /2 dθ = ̟ −∞ 2π and the posterior distribution of θ is 1 (x − θ)2 π(θ | x) = √ exp − , 2π 2 i.e., corresponds to N (x, 1). [independent of ̟] 71 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Meaningless as probability distribution The mistake is to think of them [the non-informative priors] as representing ignorance —Lindley, 1990— 72 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Meaningless as probability distribution The mistake is to think of them [the non-informative priors] as representing ignorance —Lindley, 1990— Example Consider a θ ∼ N (0, τ 2 ) prior. Then P π (θ ∈ [a, b]) −→ 0 when τ → ∞ for any (a, b) 73 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! 74 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! In the absence of prior information, prior distributions solely derived from the sample distribution f (x|θ) 75 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Noninformative prior distributions What if all we know is that we know “nothing” ?! In the absence of prior information, prior distributions solely derived from the sample distribution f (x|θ) Noninformative priors cannot be expected to represent exactly total ignorance about the problem at hand, but should rather be taken as reference or default priors, upon which everyone could fall back when the prior information is missing. —Kass and Wasserman, 1996— 76 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s prior Principle of Insufficient Reason (Laplace) Θ = {θ1 , · · · , θp } π(θi ) = 1/p 77 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s prior Principle of Insufficient Reason (Laplace) Θ = {θ1 , · · · , θp } π(θi ) = 1/p Extension to continuous spaces π(θ) ∝ 1 [Lebesgue measure] 78 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Who’s Laplace? Pierre Simon de Laplace (1749–1827) French mathematician and astronomer born in Beaumont en Auge (Normandie) who formalised mathematical astronomy in M´canique C´leste. Survived the e e French revolution, the Napoleon Empire (as a comte!), and the Bourbon restauration (as a marquis!!). 79 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Who’s Laplace? Pierre Simon de Laplace (1749–1827) French mathematician and astronomer born in Beaumont en Auge (Normandie) who formalised mathematical astronomy in M´canique C´leste. Survived the e e French revolution, the Napoleon Empire (as a comte!), and the Bourbon restauration (as a marquis!!). In Essai Philosophique sur les Probabilit´s, Laplace set out a e mathematical system of inductive reasoning based on probability, precursor to Bayesian Statistics. 80 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s problem Lack of reparameterization invariance/coherence 1 π(θ) ∝ 1, and ψ = eθ π(ψ) = = 1 (!!) ψ 81 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Laplace’s problem Lack of reparameterization invariance/coherence 1 π(θ) ∝ 1, and ψ = eθ π(ψ) = = 1 (!!) ψ Problems of properness x ∼ N (µ, σ 2 ), π(µ, σ) = 1 2 2 π(µ, σ|x) ∝ e−(x−µ) /2σ σ −1 ⇒ π(σ|x) ∝ 1 (!!!) 82 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Jeffreys’ prior Based on Fisher information ∂ log ℓ ∂ log ℓ I F (θ) = Eθ ∂θt ∂θ Ron Fisher (1890–1962) 83 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Jeffreys’ prior Based on Fisher information ∂ log ℓ ∂ log ℓ I F (θ) = Eθ ∂θt ∂θ Ron Fisher (1890–1962) the Jeffreys prior distribution is π J (θ) ∝ |I F (θ)|1/2 84 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Who’s Jeffreys? Sir Harold Jeffreys (1891–1989) English mathematician, statistician, geophysicist, and astronomer. Founder of English Geophysics & originator of the theory that the Earth core is liquid. 85 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Who’s Jeffreys? Sir Harold Jeffreys (1891–1989) English mathematician, statistician, geophysicist, and astronomer. Founder of English Geophysics & originator of the theory that the Earth core is liquid. Formalised Bayesian methods for the analysis of geophysical data and ended up writing Theory of Probability 86 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory 87 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors 88 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors Parameterization invariant 89 / 427

Bayesian data analysis for Ecologists The normal model Prior selection Pros & Cons Relates to information theory Agrees with most invariant priors Parameterization invariant Suffers from dimensionality curse 90 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D 91 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D Requires an evaluation criterion/loss function for decisions and estimators L(θ, d) 92 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Evaluating estimators Purpose of most inferential studies: to provide the statistician/client with a decision d ∈ D Requires an evaluation criterion/loss function for decisions and estimators L(θ, d) There exists an axiomatic derivation of the existence of a loss function. [DeGroot, 1970] 93 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Loss functions Decision procedure δ π usually called estimator (while its value δ π (x) is called estimate of θ) 94 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Loss functions Decision procedure δ π usually called estimator (while its value δ π (x) is called estimate of θ) Impossible to uniformly minimize (in d) the loss function L(θ, d) when θ is unknown 95 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Bayesian estimation Principle Integrate over the space Θ to get the posterior expected loss = Eπ [L(θ, d)|x] = L(θ, d)π(θ|x) dθ, Θ and minimise in d 96 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Bayes estimates Bayes estimator A Bayes estimate associated with a prior distribution π and a loss function L is arg min Eπ [L(θ, d)|x] . d 97 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation The quadratic loss Historically, first loss function (Legendre, Gauss, Laplace) L(θ, d) = (θ − d)2 98 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation The quadratic loss Historically, first loss function (Legendre, Gauss, Laplace) L(θ, d) = (θ − d)2 The Bayes estimate δ π (x) associated with the prior π and with the quadratic loss is the posterior expectation Θ θf (x|θ)π(θ) dθ δ π (x) = Eπ [θ|x] = . Θ f (x|θ)π(θ) dθ 99 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation The absolute error loss Alternatives to the quadratic loss: L(θ, d) =| θ − d |, or k2 (θ − d) if θ > d, Lk1 ,k2 (θ, d) = k1 (d − θ) otherwise. 100 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation The absolute error loss Alternatives to the quadratic loss: L(θ, d) =| θ − d |, or k2 (θ − d) if θ > d, Lk1 ,k2 (θ, d) = k1 (d − θ) otherwise. Associated Bayes estimate is (k2 /(k1 + k2 )) fractile of π(θ|x) 101 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ 102 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ Penalized likelihood estimator 103 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation MAP estimator With no loss function, consider using the maximum a posteriori (MAP) estimator arg max ℓ(θ|x)π(θ) θ Penalized likelihood estimator Further appeal in restricted parameter spaces 104 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Example (Binomial probability) Consider x|θ ∼ B(n, θ). Possible priors: 1 π J (θ) = θ−1/2 (1 − θ)−1/2 , B(1/2, 1/2) π1 (θ) = 1 and π2 (θ) = θ−1 (1 − θ)−1 . 105 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Example (Binomial probability) Consider x|θ ∼ B(n, θ). Possible priors: 1 π J (θ) = θ−1/2 (1 − θ)−1/2 , B(1/2, 1/2) π1 (θ) = 1 and π2 (θ) = θ−1 (1 − θ)−1 . Corresponding MAP estimators: x − 1/2 δ πJ (x) = max ,0 , n−1 δ π1 (x) = x/n, x−1 δ π2 (x) = max ,0 . n−2 106 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Not always appropriate Example (Fixed MAP) Consider 1 −1 f (x|θ) = 1 + (x − θ)2 , π 1 and π(θ) = 2 e−|θ| . 107 / 427

Bayesian data analysis for Ecologists The normal model Bayesian estimation Not always appropriate Example (Fixed MAP) Consider 1 −1 f (x|θ) = 1 + (x − θ)2 , π 1 and π(θ) = 2 e−|θ| . Then the MAP estimate of θ is always δ π (x) = 0 108 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Credible regions Natural confidence region: Highest posterior density (HPD) region π Cα = {θ; π(θ|x) > kα } 109 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Credible regions Natural confidence region: Highest posterior density (HPD) region π Cα = {θ; π(θ|x) > kα } Optimality 0.15 The HPD regions give the highest 0.10 probabilities of containing θ for a given volume 0.05 0.00 −10 −5 0 5 10 µ 110 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Example If the posterior distribution of θ is N (µ(x), ω −2 ) with ω 2 = τ −2 + σ −2 and µ(x) = τ 2 x/(τ 2 + σ 2 ), then Cα = µ(x) − kα ω −1 , µ(x) + kα ω −1 , π where kα is the α/2-quantile of N (0, 1). 111 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Example If the posterior distribution of θ is N (µ(x), ω −2 ) with ω 2 = τ −2 + σ −2 and µ(x) = τ 2 x/(τ 2 + σ 2 ), then Cα = µ(x) − kα ω −1 , µ(x) + kα ω −1 , π where kα is the α/2-quantile of N (0, 1). If τ goes to +∞, π Cα = [x − kα σ, x + kα σ] , the “usual” (classical) confidence interval 112 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Full normal Under [almost!] Jeffreys prior prior π(µ, σ 2 ) = 1/σ 2 , posterior distribution of (µ, σ) σ2 µ|σ, x, s2 ¯ x ∼ N x, ¯ , n n − 1 s2 σ 2 |¯, sx x 2 ∼ IG , x . 2 2 113 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Full normal Under [almost!] Jeffreys prior prior π(µ, σ 2 ) = 1/σ 2 , posterior distribution of (µ, σ) σ2 µ|σ, x, s2 ¯ x ∼ N x, ¯ , n n − 1 s2 σ 2 |¯, sx x 2 ∼ IG , x . 2 2 Then n(¯ − µ)2 (n−3)/2 x π(µ|¯, s2 ) ∝ x x ω 1/2 exp −ω ω exp{−ωs2 /2} dω x 2 −n/2 ∝ s2 + n(¯ − µ)2 x x [Tn−1 distribution] 114 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Normal credible interval Derived credible interval on µ [¯ − tα/2,n−1 sx x n(n − 1), x + tα/2,n−1 sx ¯ n(n − 1)] 115 / 427

Bayesian data analysis for Ecologists The normal model Confidence regions Normal credible interval Derived credible interval on µ [¯ − tα/2,n−1 sx x n(n − 1), x + tα/2,n−1 sx ¯ n(n − 1)] marmotdata Corresponding 95% confidence region for µ the mean for adults [3484.435, 3546.815] [Warning!] It is not because AUDREY has a weight of 2375g that she can be excluded from the adult group! 116 / 427

Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 117 / 427

Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 Binary outcome of the decision process: accept [coded by 1] or reject [coded by 0] D = {0, 1} 118 / 427

Bayesian data analysis for Ecologists The normal model Testing Testing hypotheses Deciding about validity of assumptions or restrictions on the parameter θ from the data, represented as H0 : θ ∈ Θ0 versus H1 : θ ∈ Θ0 Binary outcome of the decision process: accept [coded by 1] or reject [coded by 0] D = {0, 1} Bayesian solution formally very close from a likelihood ratio test statistic, but numerical values often strongly differ from classical solutions 119 / 427

Bayesian data analysis for Ecologists The normal model Testing The 0 − 1 loss Rudimentary loss function 1−d if θ ∈ Θ0 L(θ, d) = d otherwise, Associated Bayes estimate  1 if P π (θ ∈ Θ |x) > 1 , π 0 δ (x) = 2 0 otherwise. 120 / 427

Bayesian data analysis for Ecologists The normal model Testing The 0 − 1 loss Rudimentary loss function 1−d if θ ∈ Θ0 L(θ, d) = d otherwise, Associated Bayes estimate  1 if P π (θ ∈ Θ |x) > 1 , π 0 δ (x) = 2 0 otherwise. Intuitive structure 121 / 427

Bayesian data analysis for Ecologists The normal model Testing Extension Weighted 0 − 1 (or a0 − a1 ) loss  0 if d = IΘ0 (θ),  L(θ, d) = a0 if θ ∈ Θ0 and d = 0,   a1 if θ ∈ Θ0 and d = 1,

Add a comment

Related presentations

Related pages

Bayesian Data Analysis in Ecology Using Linear Models with ...

Franzi - Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN: Including jetzt kaufen. ISBN: 9780128013700, Fremdsprachige Bücher ...
Read more

Bayesian Data Analysis in Ecology Using Linear Models with ...

Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN examines the Bayesian and frequentist methods of conducting data analyses.
Read more

Bayesian Data Analysis in Ecology Using Linear Models with ...

Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and Stan eBook: Franzi Korner-Nievergelt, Tobias Roth, Stefanie von Felten, Jérôme ...
Read more

Bayesian Analysis for Population Ecology - CRC Press Book

... Bayesian Analysis for Population Ecology ... the authors develop the Bayesian approach for fitting models to data. They also compare Bayesian ...
Read more

Bayesian Data Analysis in Ecology Using Linear Models with ...

eBook Shop: Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and Stan als Download. Jetzt eBook herunterladen & mit Ihrem Tablet oder ...
Read more

Bayesian Data Analysis in Ecology Using ... - Elsevier Store

Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and Stan, 1st Edition
Read more

Bayesian data analysis in population ecology: motivations ...

During the 20th century ecologists largely relied on the frequentist system of inference for the analysis of their data. However, in the past few decades ...
Read more

Bayesian inference in ecology - Auburn University

Bayesian inference in ecology ... In a Bayesian analysis, ... or not to use Bayesian methods to analyse ecological data.
Read more

Bayesian Data Analysis in Ecology Using Linear Models with ...

Bücher bei Weltbild: Jetzt Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN portofrei bestellen bei Weltbild, Ihrem Bücher ...
Read more

- Bayesian Data Analysis in Ecology Using Linear Models ...

Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN. Including Comparisons to Frequentist Statistics
Read more