Transformation to Symmetry of Absolute Error Distributions

Introduction and Motivation

Spread-location (S-L) plots can be used to check the adequacy of many types of statistical models. In the S-L plot, an estimate of spread or variability is plotted against the expected value in order to detect a particular type of herteroscedasticity often called monotone spread. In checking the adequacy of a fitted statistical model, the absolute residual provides an estimate of spread and the fitted value an estimate of location.
 Cleveland (1993) suggested the use of a square root transformation to remove skewness from the absolute residual. More generally, in this notebook we derive the skewness coefficient for transformed random variables, \(TraditionalForm
      \`\(B = \ \(A\^\((λ)\) = | E  |\^\((λ)\)\), \)\) where

B=(A^((λ))) ={({{(A^λ)}, {ln(A) . }})

The square root transformation is then shown to be effective in removing skewness for the normal case. When E has some other distribution, other transformations are shown to be more suitable.
Improved S-L plots are obtained by plotting a suitable transformation of the absolute residual against the fitted value. If model inadequacy in the form of monotone spread is present, it will be revealed as a monotonic trend.  Often a smooth curve such as a robust loess smooth is used to help visualize the trend. However such curve fitting and visual trend assessment required the distribution of spread estimate be symmetric.

Skewness of Transformed Variables

Given a probability density function with a given interval of support, we symbolically define the expecation, variance and skewness.

Ex[pdf_,  support_] := Integrate[x*pdf[x], support]

Var[ pdf_,  support_] := 
      Integrate[(x - Ex[ pdf,  support])^2 * pdf[x],  support]

g1[pdf_,  support_] :=  
      Integrate[(x - Ex[ pdf,  support])^3*pdf[x],  support]/
        (Var[ pdf,  support])^(3/2)

Given the distribution of the absolute residual, g(a), we define the distribution function, h(b), of the transformed absolute value first for the case p  =  1/λ and λ ≠  0 and second for the case λ=0.

h[b_]  :=  
      If[p ≠ ∞,   p g[b^p] b^(p - 1),   
        E^b  g[E^b]]

In some cases, such as with the Γ-distribution, symbolic or exact solutions are not available. In this case we can use numerical integration. This also provides a check.

NEx[pdf_,  support_] := NIntegrate[x*pdf[x], support]

NVar[pdf_,  support_] := 
      Module[{mean = NEx[pdf,  support]}, 
        NIntegrate[(x - mean)^2 * pdf[x],  support]]

Ng1[pdf_,  support_] := 
      Module[{mean = NEx[pdf,  support]},  
        NIntegrate[(x - mean)^3*pdf[x],  support]/
          (NVar[ pdf,  support])^(3/2)]

Detailed Derivations

Normal Density Case

f[x_]  :=  1/Sqrt[2 Pi]*Exp[-x^2/2]

The absolute value distribution is given by

g[x_]  :=  2*f[x]

g1[ g,  {x, 0,  Infinity}]

% // N

Next, skewness of the square root of the absolute value.

p = 2;

g1[ h,  {x, 0, Infinity}]

% // N

p = 3 ;

g1[h,  {x, 0, Infinity}] // N

p = 4;

g1[h,  {x, 0, Infinity}] // N

p = 5/2;

g1[h,  {x, 0, Infinity}] // N

Next when λ = 0,  log of the absolute value. Note that the range of support is (-∞,  ∞) .

p = Infinity;

g1[h,  {x, -Infinity,  Infinity}]

%  // N

Student t on 5 df

Needs["Statistics`ContinuousDistributions`"];

Clear[f,  g];

f[x_]  := PDF[StudentTDistribution[5],  x]

The absolute value distribution is given by

g[x_]  :=  2*f[x]

g1[g,  {x, 0, Infinity}]

% // N

p = 2;

g1[h,  {x, 0, Infinity}]

% // N

p = 3;

g1[h,  {x, 0, Infinity}]

% // N

p = 4;

g1[h,  {x, 0, Infinity}]

% // N

p = 5/2;

g1[h,  {x, 0, Infinity}] // N

Next the log of the absolute value. Note that the range of support is (-∞,  ∞) .

p = Infinity;

When I tried g1[h,{x,-Infinity, Infinity}], I had to terminate it after running for several hours. So we use numerical integration here.

Ng1[h, {x, -Infinity,  Infinity}]

Laplace Distribution

Clear[f,  g];

LaplacePDF[x_,  beta_]  :=  
      (1/(2 beta))  E^(-Abs[x]/beta ); 
    f[x_]  =  LaplacePDF[x,  1/Sqrt[2]];

Check that this density integrates to 1.

Integrate[f[x], {x, -Infinity,  Infinity}]

The absolute value distribution is given by

g[x_]  :=  2 f[x]

Skewness of absolute value distribution.

g1[g,  {x, 0, Infinity}]

p = 2   ;

g1[h,  {x, 0, Infinity}] // N

p = 3   ;

g1[h,  {x, 0, Infinity}] // N

p = 4   ;

g1[h,  {x, 0, Infinity}] // N

p = 5/2;

g1[h,  {x, 0, Infinity}] // N

p = Infinity   ;

g1[h,  {x, -Infinity, Infinity}] // N

Let's check this result with the numerical integration routines.

Ng1[h,  {x, -Infinity, Infinity}] // N

Contaminated Normal Distribution, p=5%

Clear[f,  g];

TukeyPDF[x_,  p_,  c_]  := 
      Sqrt[1 - p + p*c^2]*
        ((1 - p)* 
              PDF[NormalDistribution[0, 1],  x*Sqrt[1 - p + p*c^2]] + 
            p*PDF[NormalDistribution[0,  c],  x*Sqrt[1 - p + p*c^2]])

f[x_]  :=  TukeyPDF[x,  0.05,  3]

g[x_]  :=  2*f[x]

g1[g,  {x, 0, Infinity}]

p = 2   ;

g1[h,  {x, 0, Infinity}]

p = 3;

g1[h,  {x, 0, Infinity}]

p = 4;

g1[h,  {x, 0, Infinity}]

p = 5/2;

g1[h, {x, 0, Infinity}]

p = Infinity;

Ng1[h,  {x, -Infinity,  Infinity}]

Centered Gamma Distribution

Clear[f,  g];

f[x_]  :=  PDF[GammaDistribution[4,  1], x + 4]

f[x]

Let's look at a plot of this density.

Plot[f[x], {x, -4, 10}];

Verify that it has mean zero.

Ex[f,  {x, -4, Infinity}]

Define the distribution of the absolute value.

g[x_ /; x > 0]  :=  f[x]  +  f[-x]

g[x_ /; x <= 0] := 0

Check that it integrates to 1.

NIntegrate[g[x], {x, 0, Infinity}]

Ng1[g,  {x, 0, Infinity}]

p = 2;

Ng1[h,  {x, 0, Infinity}]

p = 3;

Ng1[h,  {x, 0, Infinity}]

p = 5/2;

Ng1[h, {x, 0, Infinity}]

p = 4;  ;

Ng1[h, {x, 0, Infinity}]

p = Infinity   ;

Ng1[h,  {x, -Infinity, Infinity}]

Student t on 4 df

Needs["Statistics`ContinuousDistributions`"];

Clear[f,  g];

f[x_]  := PDF[StudentTDistribution[4],  x]

The absolute value distribution is given by

g[x_]  :=  2*f[x]

g1[g,  {x, 0, Infinity}] // N

p = 2;

g1[h,  {x, 0, Infinity}] // N

Ng1[h,  {x, 0, Infinity}]

p = 3;

Ng1[h,  {x, 0, Infinity}]

p = 4;

Ng1[h,  {x, 0, Infinity}]

p = 5/2;

Ng1[h,  {x, 0, Infinity}]

Next the log of the absolute value. Note that the range of support is (-∞,  ∞) .

p = Infinity;

Ng1[h, {x, -Infinity,  Infinity}]

Contaminated Normal Distribution, p=10%

Clear[f,  g];

TukeyPDF[x_,  p_,  c_]  := 
      Sqrt[1 - p + p*c^2]*
        ((1 - p)* 
              PDF[NormalDistribution[0, 1],  x*Sqrt[1 - p + p*c^2]] + 
            p*PDF[NormalDistribution[0,  c],  x*Sqrt[1 - p + p*c^2]])

f[x_]  :=  TukeyPDF[x,  0.1,  3]

g[x_]  :=  2*f[x]

g1[g,  {x, 0, Infinity}]

p = 2   ;

g1[h,  {x, 0, Infinity}]

p = 3;

g1[h,  {x, 0, Infinity}]

p = 4;

g1[h,  {x, 0, Infinity}]

p = 5/2;

g1[h, {x, 0, Infinity}]

p = Infinity;

Ng1[h,  {x, -Infinity,  Infinity}]