Ma(1), n=2

A.I. McLeod
Department of Statistical and Actuarial Sciences,
University of Western Ontario,
e-mail: aim@uwo.ca

B. Quenneville,
Statistics Canada
e-mail: quenne@statcan.ca

Introduction

Given a Gaussian time series of length 2, (Subscript[Z, 1]) ,  (Subscript[Z, 2]) generated from the first-order moving average equation (Subscript[Z, t]) =  (Subscript[A, t])  -  θ (Subscript[A, t - 1]) , 
       where (Subscript[A, t]) are independent normal random variables with mean zero and variance (Subscript[σ, A]^2). Let \(TraditionalForm
      \`W\  = \ \(-\ Z\_1\) Z\_2/\((Z\_1\%2\  + \ Z\_2\%2)\)\). Then given data,  (Subscript[Z, 1]) ,  (Subscript[Z, 2]), the exact concentrated likelihood function for θ is (Cryer and Ledholter, 1982; Quenneville, 1993),

\(TraditionalForm
    \`L(θ\  | \ W\ )\  = 
      \@\(1 + θ\^2 + θ\^4\)\/\(1 + θ\^2 - 
          2\ θ\ W\)\)

and

\(θ\&^\)={({{      -1,  
                W ϵ [-0.5,  -0. ]}, {((1 - (Sqrt[1 - 16  (w^2)]))/(4 w)), 
                W ϵ (-0.25,  0.25),  W ≠ 0}, {0,                  W = 0}, {           -1,  
                W ϵ [-0.5,  -0.25]}})

The mele for θ, \(TraditionalForm\`θ\&_\), cannot be evaluated symbolically. However using NIntegrate we can obtain it numerically. Numerical evaluation suggests that  \(θ\&_\) is either a linear or close to a linear function of W.  To speed up our computations for the mean-square error of  \(θ\&_\), we use the FunctionInterpolation in Mathematica to construct  \(θ\&_\)= \(θ\&_\)(W).  The mean-square error for  \(θ\&_\) and  \(θ\&_\) are now easily evaluated using the following result.

Theorem:  W has probability density function,

\(TraditionalForm
      \`\(\(\(f\_W\)(x)\  = \ 
        \(2\ \@\(1 + θ\^2 + \ θ\^4\)\)\/\(π 
            \(\@\( 1 - 4  x\^2\)\) \((1 + θ\^2 - 2  θ\ x)\)\), 
      \)\ \)\)  | x |   ≤ (1/2) .

Proof: Quenneville (1993) has derived this result.

Derivations of mele and mse

First we define the pdf of  W  =  - (Subscript[Z, 1]) (Subscript[Z, 2])/((Subscript[Z, 1]^2)  +  (Subscript[Z, 2]^2)) and concentrated log likelihood function for θ given W.

SetOptions[Integrate,  GenerateConditions -> False];

PDFMA1W[w_,  θ_] := 
      2 (Sqrt[1 + (θ^2) + (θ^4)])/
          (π (Sqrt[ 1 - 4 (w^2)]) 
              (1 + (θ^2) + 2 θ w))

Lmax[θ_, w_]  := 
      Sqrt[1 + (θ^2) + (θ^4)]/
        (1 + (θ^2) - 2  θ w);

The mle corresponding to an observed W,

\(mlew[w_ /; w == 0] := 0; 

    mlew[w_ /; \((Abs[w] < 1/4)\) && \((w != 0)\)] := 
      \((1 - Sqrt[1 - 16\ w\^2])\)/\((4\ w)\); 

    mlew[w_ /; \((w < 1/2)\) && \((w ≥ 1/4)\)]\  := \ 1; 

    mlew[w_ /; \((w > \(-1\)/2)\) && \((w <= \(-1\)/4)\)] := \(-1\); \)

Since Mathematica cannot handle the necessary integration for the mele symbolically we will have to work numerically. We construct an interpolation function which for a given value of W gives \(TraditionalForm\`θ\&_\).  The interpolation function, meanlew[w], provides a much faster method of computing the mean-square error and PMC for  \(θ\&_\). We check that meanlew[w] is working by comparing plots of it with the defining mele function Nθbar.

Nθbar[w_] := 
      NIntegrate[θ Lmax[θ, w],  {θ, -1, 1}]/
        NIntegrate[Lmax[θ, w],  {θ, -1, 1}];

meanlew  = 
      FunctionInterpolation[Nθbar[w], {w, -0.5, 0.5}];

Plot[{meanlew[w],  mlew[w]},  {w, -0.5, 0.5}, 
      PlotStyle -> {RGBColor[1, 0, 0], RGBColor[0, 1, 0]}, 
      AxesLabel -> {"W", "θ"}];

Plot[{Nθbar[w],  mlew[w]},  {w, -0.5, 0.5}, 
      PlotStyle -> {RGBColor[1, 0, 0], RGBColor[0, 1, 0]}, 
      AxesLabel -> {"W", "θ"}];

Derivation of the relative efficiency, mele vs mle

Now define the mean-square error functions and plot them.

\(mseθhat[θ_] := 
      NIntegrate[\((mlew[w] - θ)\)^2\ PDFMA1W[w, θ], 
        \ {w, \(-1\)/2, \ 1/2}, MaxRecursion -> 10, MinRecursion -> 3, 
        AccuracyGoal -> 5, \ PrecisionGoal -> 5]; 

    mseθbar[θ_] := 
      NIntegrate[\((meanlew[w] - θ)\)^2\ PDFMA1W[w, θ], 
        \ {w, \(-1\)/2, \ 1/2}, MaxRecursion -> 10, MinRecursion -> 3, 
        AccuracyGoal -> 5, \ PrecisionGoal -> 5]; \)

eff[θ_] := mseθhat[θ]/mseθbar[θ]

Plot[eff[θ], {θ, -1, 1}, 
      FrameLabel -> {"theta", "R"}, PlotRange -> All, 
      AxesOrigin -> 0.0, Frame -> True];

\(\(Plot[1/eff[θ], {θ, \(-1\), 1}, 
      AxesLabel -> {"θ", 
          \*""R(\!\(θ\&^\),\!\(θ\&_\))""}, 
      PlotRange -> All, AxesOrigin -> 0.0, Frame -> True]; \)\)

Derivation of PMC, MLE and MELE

PMC[θ_] := 
      NIntegrate[
        If[Abs[mlew[w] - θ] > Abs[meanlew[w] - θ], 1, 0] 
          PDFMA1W[w, θ], {w, -1/2, 1/2}, MaxRecursion -> 10, 
        MinRecursion -> 3, AccuracyGoal -> 5,  PrecisionGoal -> 5]

Plot[PMC[θ], {θ, -1, 1}, 
      FrameLabel -> {"theta", "PMC"}, PlotRange -> All, 
      AxesOrigin -> 0.0, Frame -> True];

Bayesian with Jeffrey's Prior: Efficiency, PMC

Notice that it is a proper prior,

Integrate[1/Sqrt[1 - (θ^2)], {θ, -1, 1}]

Nθtilde[w_] := 
      NIntegrate[θ Lmax[θ, w]/Sqrt[1 - (θ^2)], 
           {θ, -1, 1}]/
        NIntegrate[Lmax[θ, w]/Sqrt[1 - (θ^2)], 
           {θ, -1, 1}];

Plot[Nθtilde[w], {w, -0.5, 0.5}]

Needs["Graphics`Legend`"]

Plot[{mlew[w], Nθbar[w],  Nθtilde[w]}, {w, -0.5, 
        0.5}, PlotLegend -> {"MLE", "MELE", "Bayes"}, 
      Axes -> False, Frame -> True, FrameLabel -> {"W", "Estimate"}, 
      PlotStyle -> {{Dashing[{0.01, 0.02}]}, {}, {Dashing[{0.04, 0.05}]}}, 
      RotateLabel -> False, LegendShadow -> False, 
      LegendPosition -> {1.1, -0.4}];

Bayesw  = 
      FunctionInterpolation[Nθtilde[w], {w, -0.5, 0.5}];

\(mseθtilde[θ_] := 
      NIntegrate[\((Bayesw[w] - θ)\)^2\ PDFMA1W[w, θ], 
        \ {w, \(-1\)/2, \ 1/2}, MaxRecursion -> 10, MinRecursion -> 3, 
        AccuracyGoal -> 5, \ PrecisionGoal -> 5]; 

    eff2[θ_] := mseθhat[θ]/mseθtilde[θ]; 
    \)

Plot[eff2[θ], {θ, -1, 1}, 
      FrameLabel -> {"theta", "R"}, PlotRange -> All, 
      AxesOrigin -> 0.0, Frame -> True];

eff3[θ_] := 
      mseθtilde[θ]/mseθbar[θ];

Plot[eff3[θ], {θ, -1, 1}, 
      FrameLabel -> {"theta", "R"}, PlotRange -> All, 
      AxesOrigin -> 0.0, Frame -> True];

PMC2[θ_] := 
      NIntegrate[
        If[Abs[mlew[w] - θ] > Abs[Bayesw[w] - θ], 1, 0] 
          PDFMA1W[w, θ], {w, -1/2, 1/2}, MaxRecursion -> 10, 
        MinRecursion -> 3, AccuracyGoal -> 5,  PrecisionGoal -> 5]

Plot[PMC2[θ], {θ, -1, 1}, 
      FrameLabel -> {"theta", "PMC"}, PlotRange -> All, 
      AxesOrigin -> {0, 0}, Frame -> True];

Combined Plots: Efficiency

Needs["Graphics`Legend`"]

Plot[{eff[θ], eff2[θ]}, {θ, -1, 1}, 
      PlotRange -> {Automatic, All}, Axes -> False, Frame -> True, 
      RotateLabel -> False, FrameLabel -> {"theta", "R"}, 
      PlotStyle -> {{}, {Dashing[{0.005, 0.01}]}}, 
      PlotLegend -> {"mele", "Bayes"}, LegendShadow -> None, 
      LegendPosition -> {0.3, 0.0}];

Plot[{PMC[θ], PMC2[θ]}, {θ, -1, 1}, 
      PlotRange -> {Automatic, All}, Axes -> False, Frame -> True, 
      RotateLabel -> False, FrameLabel -> {"theta", "PMC"}, 
      PlotStyle -> {{}, {Dashing[{0.005, 0.01}]}}, 
      PlotLegend -> {"mele", "Bayes"}, LegendSize -> {0.4, 0.4}, 
      LegendShadow -> None, LegendPosition -> {0.35, 0.2}];