Validation
GeoStats.jl was designed to, among other things, facilitate rigorous scientific comparison of different geostatistical solvers in the literature. As a user of geostatistics, you may be interested in applying various solvers on a given data set and pick the ones with best performance. As a researcher in the field, you may be interested in benchmarking your new method against other established methods.
Errors of geostatistical solvers can be estimated on given geostatistical problems:
Base.error
— Methoderror(solver, problem, method)
Estimate error of solver
in a given problem
with error estimation method
.
Below is the list of currently implemented error estimation methods.
Leave-one-out validation
GeoStatsBase.LeaveOneOut
— TypeLeaveOneOut(; loss=Dict())
Leave-one-out validation. Optionally, specify loss
function from LossFunctions.jl
for some of the variables.
References
Leave-ball-out validation
GeoStatsBase.LeaveBallOut
— TypeLeaveBallOut(ball; loss=Dict())
Leave-ball
-out (a.k.a. spatial leave-one-out) validation. Optionally, specify loss
function from the LossFunctions.jl package for some of the variables.
LeaveBallOut(radius; loss=Dict())
By default, use Euclidean ball of given radius
in space.
References
Cross-validation
GeoStatsBase.CrossValidation
— TypeCrossValidation(k; shuffle=true, loss=Dict())
k
-fold cross-validation. Optionally, shuffle
the data, and specify loss
function from LossFunctions.jl
for some of the variables.
References
Block cross-validation
GeoStatsBase.BlockCrossValidation
— TypeBlockCrossValidation(sides; loss=Dict())
Cross-validation with blocks of given sides
. Optionally, specify loss
function from LossFunctions.jl
for some of the variables. If only one side is provided, then blocks become cubes.
References
Weighted cross-validation
GeoStatsBase.WeightedCrossValidation
— TypeWeightedCrossValidation(weigthing, folding; lambda=1.0, loss=Dict())
An error estimation method which samples are weighted with weighting
method and split into folds with folding
method. Weights are raised to lambda
power in [0,1]
. Optionally, specify loss
function from LossFunctions.jl
for some of the variables.
References
- Sugiyama et al. 2006. Importance-weighted cross-validation for covariate shift
- Sugiyama et al. 2007. Covariate shift adaptation by importance weighted cross validation
Density-ratio validation
GeoStatsBase.DensityRatioValidation
— TypeDensityRatioValidation(k; [parameters])
Desntity ratio validation where weights are first obtained with density ratio estimation, and then used in k
-fold weighted cross-validation.
Parameters
shuffle
- Shuffle the data before folding (default totrue
)estimator
- Density ratio estimator (default toLSIF()
)optlib
- Optimization library (default todefault_optlib(estimator)
)lambda
- Power of density ratios (default to1.0
)
Please see DensityRatioEstimation.jl for a list of supported estimators.
References
- Hoffimann et al. 2020. Geostatistical Learning: Challenges and Opportunities