fairlens.metrics.significance#

Collection of methods which can be used to numerically or analytically compute p-values and confidence intervals.

This module provides three functions to sample and generate distributions required for estimating p_values:
  • permutation_statistic

  • bootstrap_statistic

  • bootstrap_binned_statistic

The functions, resampling_p_value, resampling_interval can be use these distributions to carry out p-value tests or obtain a confidence interval.

Functions

binominal_proportion_interval

Calculate an approximate confidence interval for a binomial proportion of a sample.

binominal_proportion_p_value

Calculate an exact p-value for an observed binomial proportion of a sample.

bootstrap_binned_statistic

Compute the samples of a binned statistic estimate using the bootstrap method.

bootstrap_statistic

Compute the samples of a statistic estimate using the bootstrap method.

brunner_munzel_test

Compute the non-parametric Brunner-Munzel test of the hypothesis that the probability of getting large values in the target attribute distributions (determined by the input groups of interest) is equal, without requiring equivariance.

permutation_statistic

Performs the sampling for a two sample permutation test.

resampling_interval

Compute a confidence interval using a distribution of the test statistic on resampled data.

resampling_p_value

Compute a p-value using a resampled test statistic distribution.