criterion-1.1.1.0: Robust, reliable performance measurement and analysis

Copyright(c) 2009-2014 Bryan O'Sullivan
LicenseBSD-style
Maintainerbos@serpentine.com
Stabilityexperimental
PortabilityGHC
Safe HaskellNone
LanguageHaskell98

Criterion.Analysis

Description

Analysis code for benchmarks.

Synopsis

Documentation

data Outliers

Outliers from sample data, calculated using the boxplot technique.

Constructors

Outliers 

Fields

samplesSeen :: !Int64
 
lowSevere :: !Int64

More than 3 times the interquartile range (IQR) below the first quartile.

lowMild :: !Int64

Between 1.5 and 3 times the IQR below the first quartile.

highMild :: !Int64

Between 1.5 and 3 times the IQR above the third quartile.

highSevere :: !Int64

More than 3 times the IQR above the third quartile.

data OutlierEffect

A description of the extent to which outliers in the sample data affect the sample mean and standard deviation.

Constructors

Unaffected

Less than 1% effect.

Slight

Between 1% and 10%.

Moderate

Between 10% and 50%.

Severe

Above 50% (i.e. measurements are useless).

data OutlierVariance

Analysis of the extent to which outliers in a sample affect its standard deviation (and to some extent, its mean).

Constructors

OutlierVariance 

Fields

ovEffect :: OutlierEffect

Qualitative description of effect.

ovDesc :: String

Brief textual description of effect.

ovFraction :: Double

Quantitative description of effect (a fraction between 0 and 1).

data SampleAnalysis

Result of a bootstrap analysis of a non-parametric sample.

Constructors

SampleAnalysis 

Fields

anRegress :: [Regression]

Estimates calculated via linear regression.

anOverhead :: Double

Estimated measurement overhead, in seconds. Estimation is performed via linear regression.

anMean :: Estimate

Estimated mean.

anStdDev :: Estimate

Estimated standard deviation.

anOutlierVar :: OutlierVariance

Description of the effects of outliers on the estimated variance.

analyseSample

Arguments

:: Int

Experiment number.

-> String

Experiment name.

-> Vector Measured

Sample data.

-> ExceptT String Criterion Report 

Perform an analysis of a measurement.

scale

Arguments

:: Double

Value to multiply by.

-> SampleAnalysis 
-> SampleAnalysis 

Multiply the Estimates in an analysis by the given value, using scale.

analyseMean

Arguments

:: Sample 
-> Int

Number of iterations used to compute the sample.

-> Criterion Double 

Display the mean of a Sample, and characterise the outliers present in the sample.

countOutliers :: Outliers -> Int64

Count the total number of outliers in a sample.

classifyOutliers :: Sample -> Outliers

Classify outliers in a data set, using the boxplot technique.

noteOutliers :: Outliers -> Criterion ()

Display a report of the Outliers present in a Sample.

outlierVariance

Arguments

:: Estimate

Bootstrap estimate of sample mean.

-> Estimate

Bootstrap estimate of sample standard deviation.

-> Double

Number of original iterations.

-> OutlierVariance 

Compute the extent to which outliers in the sample data affect the sample mean and standard deviation.

resolveAccessors :: [String] -> Either String [(String, Measured -> Maybe Double)]

Given a list of accessor names (see measureKeys), return either a mapping from accessor name to function or an error message if any names are wrong.

validateAccessors

Arguments

:: [String]

Predictor names.

-> String

Responder name.

-> Either String [(String, Measured -> Maybe Double)] 

Given predictor and responder names, do some basic validation, then hand back the relevant accessors.

regress

Arguments

:: GenIO 
-> [String]

Predictor names.

-> String

Responder name.

-> Vector Measured 
-> ExceptT String Criterion Regression 

Regress the given predictors against the responder.

Errors may be returned under various circumstances, such as invalid names or lack of needed data.

See olsRegress for details of the regression performed.