Validation

GeoStats.jl was designed to, among other things, facilitate rigorous scientific comparison of different geostatistical solvers in the literature. As a user of geostatistics, you may be interested in applying various solvers on a given data set and pick the ones with best performance. As a researcher in the field, you may be interested in benchmarking your new method against other established methods.

Errors of geostatistical solvers can be estimated on given geostatistical problems:

Base.errorMethod
error(solver, problem, method)

Estimate error of solver in a given problem with error estimation method.

Below is the list of currently implemented error estimation methods.

Leave-one-out validation

Leave-ball-out validation

Cross-validation

Block cross-validation

GeoStatsBase.BlockCrossValidationType
BlockCrossValidation(sides; loss=Dict())

Cross-validation with blocks of given sides. Optionally, specify loss function from LossFunctions.jl for some of the variables. If only one side is provided, then blocks become cubes.

References

Weighted cross-validation

GeoStatsBase.WeightedCrossValidationType
WeightedCrossValidation(weigthing, folding; lambda=1.0, loss=Dict())

An error estimation method which samples are weighted with weighting method and split into folds with folding method. Weights are raised to lambda power in [0,1]. Optionally, specify loss function from LossFunctions.jl for some of the variables.

References

Density-ratio validation

GeoStatsBase.DensityRatioValidationType
DensityRatioValidation(k; [parameters])

Desntity ratio validation where weights are first obtained with density ratio estimation, and then used in k-fold weighted cross-validation.

Parameters

  • shuffle - Shuffle the data before folding (default to true)
  • estimator - Density ratio estimator (default to LSIF())
  • optlib - Optimization library (default to default_optlib(estimator))
  • lambda - Power of density ratios (default to 1.0)

Please see DensityRatioEstimation.jl for a list of supported estimators.

References