Discrete CDF Probability:

\[P(x \leq b) = n^{-1}\sum_{i=1}^n I(x_i \leq b) \] where $I$ is the indicator function.

This is represented by the degree 0 LPM ratio for target $(b)$:

\[ LPM(\color{red}{0}, b, x) = \frac{1}{n} \sum_{i=1}^n [max(0, b - x_i)]^\color{red}{0} \] Whereby the 0 exponent converts this to a discrete indicator function. It is already normalized [0,1].

Continuous PDF Probability:

\[P(x \leq b) = \frac{\int_{a}^b f(x) dx}{(\int_{a}^b f(x) dx +\int_{b}^c f(x) dx) } \quad where \quad a<b<c\] This is represented by the degree 1 LPM ratio for target $(b)$:

\[ LPM(\color{red}{1}, b, x) = \frac{1}{n} \sum_{i=1}^n [max(0, b - x_i)]^\color{red}{1} \] And to normalize this value [0,1], we divide by the total area (lower partial moment + upper partial moment).

\[ LPM_{norm}(1, b, x) = \frac{LPM(1, b, x)}{LPM(1, b, x) + UPM(1, b, x)} \]

In NNS, this is the LPM.ratio(degree, target, variable) function.

Key Difference

It is different from the discrete because in order to go from the CDF (discrete, $F(x)$), the PDF (continuous, $f(x)$) contains the magnitude of the observations through the derivative of the CDF whereby $f(x) = \frac{dF(x)}{dx}$.

Even if you have 1mm observations in the CDF, it is still discrete, but by using the PDF, you have captured the area between the bins of the discrete and are truly continuous.

Elementary Example

Here we will illustrate two very important features, the difference from using the PDF vs. CDF in bias reduction and the ability to compensate for area between values.

First, we will show that the degree 0 partial moment ratio is equal to the empirical CDF.

library(NNS)
set.seed(12345)
x = rnorm(100, mean = 5, sd = 1)

Empirical CDF

P = ecdf(x)
P(mean(x))

## [1] 0.44

Discrete LPM degree 0

LPM.ratio(0, mean(x), x)

## [1] 0.44

Visualization

We can see the red LPM degree 0 points overlayed on the empirical CDF…same values.

plot(ecdf(x))
points(sort(x), LPM.ratio(0,sort(x),x), col = 'red')
legend('left', legend = c('ecdf','LPM.CDF'), fill=c('black','red'), border=NA, bty='n')

Bias

In this example, 44% of the observations lie below the mean. We know that this should approach 50% in the limit, but it will never equal 50% for any finite number of observations.

Continuous LPM degree 1

Let’s see how the continuous area based probability treats this…

LPM.ratio(1, mean(x), x)

## [1] 0.5

50% exactly. Wow, good job! Maybe it’s a lucky outcome, let’s increase the number of observations and see what happens…

set.seed(12345)
x_2 = rnorm(500, mean = 5, sd = 1)

P = ecdf(x_2)
P(mean(x_2))

## [1] 0.496

LPM(0, mean(x_2), x_2)

## [1] 0.496

LPM.ratio(1, mean(x_2), x_2)

## [1] 0.5

Still not there for the discrete CDF based probabilities, but the area based probability is consistent.

Let’s test this out for a range of observations:

# Generate data
set.seed(12345); x = rnorm(500)

# Compute statistics for each observation
LPM.mean_target.CDF = numeric()
LPM.1.mean_target.CDF = numeric()

for(i in 1:length(x)){
    LPM.mean_target.CDF[i] = LPM.ratio(0, mean(x[1:i]), x[1:i]);
    LPM.1.mean_target.CDF[i] = LPM.ratio(1, mean(x[1:i]), x[1:i])
}

# Plot values
plot(LPM.mean_target.CDF, col='red', type = 'l', lwd=3)
lines((1:500), LPM.1.mean_target.CDF, col='blue', lwd=3)
legend('topright',legend = c('LPM.CDF','LPM.1.CDF'),fill=c('red','blue'),
       border=NA,bty='n')

For every observation of every type of distribution the continuous probability of LPM degree 1 from the $(\hat{\mu}_x)$ will equal 0.5, without exception, while the empirical CDF based probability will asymptotically approach this known value.

There is no bias associated with this measure. Once we realize this salient point, we can apply it to other points in the distribution to ascertain confidence intervals.

Confidence Intervals

Now that the bias (lack thereof) of the metric has been established, we can move to confidence intervals.

Let’s look at an example described in Efron and Tibshirani’s (1993) text on bootstrapping (page 19), available in the R-package bootstrap.

library(bootstrap)
data("law")

law

Using the other bootstrapping package in R, boot we can easily generate several confidence intervals for the correlation statistic of the sample.¹

library(boot)

# Define Correlation Function
get_r <- function(data, indices, x, y) {

  d <- data[indices, ]
  r <- round(as.numeric(cor(d[x], d[y])), 3)

  r
}

set.seed(12345)

boot_out <- boot(
  law,
  x = "LSAT", 
  y = "GPA", 
  R = 500,
  statistic = get_r
)

# Visualization of the distribution of bootstrapped correlation statistics
hist(boot_out$t)

# Confidence Intervals and their methods
boot.ci(boot_out)

## Warning in boot.ci(boot_out): bootstrap variances needed for studentized
## intervals

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 500 bootstrap replicates
## 
## CALL : 
## boot.ci(boot.out = boot_out)
## 
## Intervals : 
## Level      Normal              Basic         
## 95%   ( 0.5247,  1.0368 )   ( 0.5900,  1.0911 )  
## 
## Level     Percentile            BCa          
## 95%   ( 0.4609,  0.9620 )   ( 0.3948,  0.9443 )  
## Calculations and Intervals on Original Scale
## Some BCa intervals may be unstable

It looks very asymmetrical, we can try to correct for that with a double bootstrap by keeping the variance of the statistic for each bootstrap and studentizing…

library(purrr)

## 
## Attaching package: 'purrr'

## The following objects are masked from 'package:foreach':
## 
##     accumulate, when

get_r_var <- function(x, y, data, indices, its) {

  d <- data[indices, ]
  r <- cor(d[x], d[y]) %>%
    as.numeric() %>%
    round(3)
  n <- nrow(d)

  v <- boot(
    x = x, 
    y = y,
    R = its,
    data = d,
    statistic = get_r
  ) %>%
    pluck("t") %>%
    var(na.rm = TRUE)
  c(r, v)
}

boot_t_out <- boot(
  x = "LSAT", y = "GPA", its = 200,
  R = 1000, data = law, statistic = get_r_var
)


# CI
boot.ci(boot_t_out)

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 1000 bootstrap replicates
## 
## CALL : 
## boot.ci(boot.out = boot_t_out)
## 
## Intervals : 
## Level      Normal              Basic             Studentized     
## 95%   ( 0.5187,  1.0485 )   ( 0.5880,  1.0830 )   (-0.2543,  0.9728 )  
## 
## Level     Percentile            BCa          
## 95%   ( 0.4690,  0.9640 )   ( 0.3210,  0.9408 )  
## Calculations and Intervals on Original Scale
## Some BCa intervals may be unstable

Negative values for the studentized version!?!?!

Next, we can compare these results with the partial moments solutions.

Discrete LPM degree 0

Notice this corresponds with the percentile based CI. We call this from the double bootstrap output to note the correspondence. It is also true for the regular bootstrap using LPM.VaR(0..25, 0, boot_out$t) and that percentile output.

# Discrete Lower and Upper CI
LPM.VaR(.025, 0, boot_t_out$t[,1]); UPM.VaR(.025, 0, boot_t_out$t[,1])

## [1] 0.4688333

## [1] 0.9632222

Continuous LPM degree 1

Much more sensible treatment, especially noticeable in the left-tail of our original bootstrap. There is no need for the double bootstrap as the area based probabilities compensate our original single bootstrap output boot_out$t.

LPM.VaR(.025, 1, boot_out$t); UPM.VaR(0.25, 1, boot_out$t)

## [1] 0.5612749

## [1] 0.8263255

There is an excellent description with code explaining all of the derivations for each component of the bootstrap confidence interval output at the following link: https://blog.methodsconsultants.com/posts/understanding-bootstrap-confidence-interval-output-from-the-r-boot-package/↩

Continuous vs. Discrete Confidence Intervals

Fred Viole

5/22/2020

Discrete CDF Probability:

Continuous PDF Probability:

Key Difference

Elementary Example

Empirical CDF

Discrete LPM degree 0

Visualization

Bias

Continuous LPM degree 1

Confidence Intervals

Discrete LPM degree 0

Continuous LPM degree 1