Geospatial transforms

Select(col₁, col₂, ..., colₙ)
Select([col₁, col₂, ..., colₙ])
Select((col₁, col₂, ..., colₙ))

The transform that selects columns col₁, col₂, ..., colₙ.

Select(col₁ => newcol₁, col₂ => newcol₂, ..., colₙ => newcolₙ)

Selects the columns col₁, col₂, ..., colₙ and rename them to newcol₁, newcol₂, ..., newcolₙ.

Select(regex)

Selects the columns that match with regex.

Examples

Select(1, 3, 5)
Select([:a, :c, :e])
Select(("a", "c", "e"))
Select(1 => :x, 3 => :y)
Select(:a => :x, :b => :y)
Select("a" => "x", "b" => "y")
Select(r"[ace]")

FeatureTransform

A transform that operates on the columns of the table containing features, i.e., simple attributes such as numbers, strings, etc.

GeometricTransform

A method to transform the geometry (e.g. coordinates) of objects. See https://en.wikipedia.org/wiki/Geometric_transformation.

isrevertible(transform)

Tells whether or not the transform is revertible, i.e. supports a revert function. Defaults to false for new transform types.

Transforms can be revertible and yet don't be invertible. Invertibility is a mathematical concept, whereas revertibility is a computational concept.

See also isinvertible.

isinvertible(transform)

Tells whether or not the transform is invertible, i.e. whether it implements the inverse function. Defaults to false for new transform types.

Transforms can be invertible in the mathematical sense, i.e., there exists a one-to-one mapping between input and output spaces.

See also inverse, isrevertible.

inverse(transform)

Returns the inverse transform of the transform.

See also isinvertible.

newobject, cache = apply(transform, object)

Apply transform on the object. Return the newobject and a cache to revert the transform later.

object = revert(transform, newobject, cache)

Revert the transform on the newobject using the cache from the corresponding apply call and return the original object. Only defined when the transform isrevertible.

newobject = reapply(transform, object, cache)

Reapply the transform to (a possibly different) object using a cache that was created with a previous apply call. Fallback to apply without using the cache.

Learn(train, model, incols => outcols)

Fits the statistical learning model using the input columns, selected by incols, and the output columns, selected by outcols, from the train table.

The column selection can be a single column identifier (index or name), a collection of identifiers or a regular expression (regex).

Examples

Learn(train, model, [1, 2, 3] => "d")
Learn(train, model, [:a, :b, :c] => :d)
Learn(train, model, ["a", "b", "c"] => 4)
Learn(train, model, [1, 2, 3] => [:d, :e])
Learn(train, model, r"[abc]" => ["d", "e"])

UniqueCoords(var₁ => agg₁, var₂ => agg₂, ..., varₙ => aggₙ)

Retain locations in data with unique coordinates.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

UniqueCoords(1 => last, 2 => maximum)
UniqueCoords(:a => first, :b => minimum)
UniqueCoords("a" => last, "b" => maximum)

Detrend(col₁, col₂, ..., colₙ; degree=1)
Detrend([col₁, col₂, ..., colₙ]; degree=1)
Detrend((col₁, col₂, ..., colₙ); degree=1)

The transform that detrends columns col₁, col₂, ..., colₙ with a polynomial of given degree.

Detrend(regex; degree=1)

Detrends the columns that match with regex.

Examples

Detrend(1, 3, 5)
Detrend([:a, :c, :e])
Detrend(("a", "c", "e"))
Detrend(r"[ace]", degree=2)
Detrend(:)

References

• Menafoglio, A., Secchi, P. 2013. A Universal Kriging predictor for spatially dependent functional data of a Hilbert Space

Potrace(mask; [ϵ])
Potrace(mask, var₁ => agg₁, ..., varₙ => aggₙ; [ϵ])

Trace polygons on 2D image data with Selinger's Potrace algorithm.

The categories stored in column mask are converted into binary masks, which are then traced into multi-polygons. When provided, the option ϵ is forwarded to Selinger's simplification algorithm.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

Potrace(:mask, ϵ=0.1)
Potrace(1, 1 => last, 2 => maximum)
Potrace(:mask, :a => first, :b => minimum)
Potrace("mask", "a" => last, "b" => maximum)

References

• Selinger, P. 2003. Potrace: A polygon-based tracing algorithm

Rasterize(grid)
Rasterize(grid, var₁ => agg₁, ..., varₙ => aggₙ)

Rasterize geometries within specified grid.

Rasterize(nx, ny)
Rasterize(nx, ny, var₁ => agg₁, ..., varₙ => aggₙ)

Alternatively, use the grid with size nx by ny obtained with discretization of the bounding box.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

grid = CartesianGrid(10, 10)
Rasterize(grid)
Rasterize(10, 10)
Rasterize(grid, 1 => last, 2 => maximum)
Rasterize(10, 10, 1 => last, 2 => maximum)
Rasterize(grid, :a => first, :b => minimum)
Rasterize(10, 10, :a => first, :b => minimum)
Rasterize(grid, "a" => last, "b" => maximum)
Rasterize(10, 10, "a" => last, "b" => maximum)

GHC(k, λ; kern=:epanechnikov, link=:ward, as=:cluster)

A transform for partitioning geospatial data into k clusters according to a range λ using Geostatistical Hierarchical Clustering (GHC). The larger the range the more connected are nearby samples.

Parameters

• k - Approximate number of clusters
• λ - Approximate range of kernel function in length units
• kern - Kernel function (:uniform, :triangular or :epanechnikov)
• link - Linkage function (:single, :average, :complete, :ward or :ward_presquared)
• as - Variable name used to store clustering results

References

• Fouedjio, F. 2016. A hierarchical clustering method for multivariate geostatistical data

Notes

• The range parameter controls the sparsity pattern of the pairwise distances, which can greatly affect the computational performance of the GHC algorithm. We recommend choosing a range that is small enough to connect nearby samples. For example, clustering data over a 100x100 Cartesian grid with unit spacing is possible with λ=1.0 or λ=2.0 but the problem starts to become computationally unfeasible around λ=10.0 due to the density of points.

GSC(k, m; σ=1.0, tol=1e-4, maxiter=10, weights=nothing, as=:cluster)

A transform for partitioning geospatial data into k clusters using Geostatistical Spectral Clustering (GSC).

Parameters

• k - Desired number of clusters
• m - Multiplicative factor for adjacent weights
• σ - Standard deviation for exponential model (default to 1.0)
• tol - Tolerance of k-means algorithm (default to 1e-4)
• maxiter - Maximum number of iterations (default to 10)
• weights - Dictionary with weights for each attribute (default to nothing)
• as - Variable name used to store clustering results

References

• Romary et al. 2015. Unsupervised classification of multivariate geostatistical data: Two algorithms

Notes

• The algorithm implemented here is slightly different than the algorithm

described in Romary et al. 2015. Instead of setting Wᵢⱼ = 0 when i <-/-> j, we simply magnify the weight by a multiplicative factor Wᵢⱼ *= m when i <–> j. This leads to dense matrices but also better results in practice.

SLIC(k, m; tol=1e-4, maxiter=10, weights=nothing, as=:cluster)

A transform for clustering geospatial data into approximately k clusters using Simple Linear Iterative Clustering (SLIC). The transform produces clusters of samples that are spatially connected based on a distance dₛ and that, at the same time, are similar in terms of vars with distance dᵥ. The tradeoff is controlled with a hyperparameter parameter m in an additive model dₜ = √(dᵥ² + m²(dₛ/s)²).

Parameters

• k - Approximate number of clusters
• m - Hyperparameter of SLIC model
• tol - Tolerance of k-means algorithm (default to 1e-4)
• maxiter - Maximum number of iterations (default to 10)
• weights - Dictionary with weights for each attribute (default to nothing)
• as - Variable name used to store clustering results

References

• Achanta et al. 2011. SLIC superpixels compared to state-of-the-art superpixel methods

Interpolate(domain, vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])
Interpolate([g₁, g₂, ..., gₙ], vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])

Interpolate geospatial data on given domain or vector of geometries [g₁, g₂, ..., gₙ], using geostatistical models model₁, ..., modelₙ for variables vars₁, ..., varsₙ.

Interpolate(domain, model=NN(); [parameters])
Interpolate([g₁, g₂, ..., gₙ], model=NN(); [parameters])

Interpolate geospatial data on given domain or vector of geometries [g₁, g₂, ..., gₙ], using geostatistical model for all variables.

Parameters

• point - Perform interpolation on point support (default to true)
• prob - Perform probabilistic interpolation (default to false)

See also InterpolateNeighbors, InterpolateMissing, InterpolateNaN.

InterpolateNeighbors(domain, vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])
InterpolateNeighbors([g₁, g₂, ..., gₙ], vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])

Interpolate geospatial data on given domain or set of geometries g₁, g₂, ..., gₙ, using geostatistical models model₁, ..., modelₙ for variables vars₁, ..., varsₙ.

InterpolateNeighbors(domain, model=NN(); [parameters])
InterpolateNeighbors([g₁, g₂, ..., gₙ], model=NN(); [parameters])

Interpolate geospatial data on given domain or set of geometries g₁, g₂, ..., gₙ, using geostatistical model for all variables.

Unlike Interpolate, this transform uses neighbor search methods to fit geostatistical models at each interpolation location with a reduced number of measurements.

Parameters

• minneighbors - Minimum number of neighbors (default to 1)
• maxneighbors - Maximum number of neighbors (default to 10)
• neighborhood - Search neighborhood (default to nothing)
• distance - A distance defined in Distances.jl (default to Euclidean())
• point - Perform interpolation on point support (default to true)
• prob - Perform probabilistic interpolation (default to false)

The maxneighbors parameter can be used to perform interpolation with a subset of measurements per prediction location. If maxneighbors is not provided, then all measurements are used.

Two neighborhood search methods are available:

• If a neighborhood is provided, local prediction is performed by sliding the neighborhood in the domain.

• If a neighborhood is not provided, the prediction is performed using maxneighbors nearest neighbors according to distance.

See also Interpolate, InterpolateMissing, InterpolateNaN.

InterpolateMissing(vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])
InterpolateMissing(vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])

Interpolate geospatial data on its own domain, using geostatistical models model₁, ..., modelₙ and non-missing values of the variables vars₁, ..., varsₙ.

InterpolateMissing(model=NN(); [parameters])
InterpolateMissing(model=NN(); [parameters])

Interpolate geospatial data on its own domain, using geostatistical model and non-missing values of all variables.

Just like InterpolateNeighbors, this transform uses neighbor search methods to fit geostatistical models at each interpolation location with a reduced number of measurements.

Parameters

• minneighbors - Minimum number of neighbors (default to 1)
• maxneighbors - Maximum number of neighbors (default to 10)
• neighborhood - Search neighborhood (default to nothing)
• distance - A distance defined in Distances.jl (default to Euclidean())
• point - Perform interpolation on point support (default to true)
• prob - Perform probabilistic interpolation (default to false)

The maxneighbors parameter can be used to perform interpolation with a subset of measurements per prediction location. If maxneighbors is not provided, then all measurements are used.

Two neighborhood search methods are available:

• If a neighborhood is provided, local prediction is performed by sliding the neighborhood in the domain.

• If a neighborhood is not provided, the prediction is performed using maxneighbors nearest neighbors according to distance.

See also InterpolateNaN, InterpolateNeighbors, Interpolate.

InterpolateNaN(vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])
InterpolateNaN(vars₁ => model₁, ..., varsₙ => modelₙ; [parameters])

Interpolate geospatial data on its own domain, using geostatistical models model₁, ..., modelₙ and non-NaN values of the variables vars₁, ..., varsₙ.

InterpolateNaN(model=NN(); [parameters])
InterpolateNaN(model=NN(); [parameters])

Interpolate geospatial data on its own domain, using geostatistical model and non-NaN values of all variables.

Just like InterpolateNeighbors, this transform uses neighbor search methods to fit geostatistical models at each interpolation location with a reduced number of measurements.

Parameters

• minneighbors - Minimum number of neighbors (default to 1)
• maxneighbors - Maximum number of neighbors (default to 10)
• neighborhood - Search neighborhood (default to nothing)
• distance - A distance defined in Distances.jl (default to Euclidean())
• point - Perform interpolation on point support (default to true)
• prob - Perform probabilistic interpolation (default to false)

The maxneighbors parameter can be used to perform interpolation with a subset of measurements per prediction location. If maxneighbors is not provided, then all measurements are used.

Two neighborhood search methods are available:

• If a neighborhood is provided, local prediction is performed by sliding the neighborhood in the domain.

• If a neighborhood is not provided, the prediction is performed using maxneighbors nearest neighbors according to distance.

See also InterpolateMissing, InterpolateNeighbors, Interpolate.

Simulate(domain, vars₁ => process₁, ..., varsₙ => processₙ; [parameters])
Simulate(domain, nreals, vars₁ => process₁, ..., varsₙ => processₙ; [parameters])
Simulate([g₁, g₂, ..., gₙ], vars₁ => process₁, ..., varsₙ => processₙ; [parameters])
Simulate([g₁, g₂, ..., gₙ], nreals, vars₁ => process₁, ..., varsₙ => processₙ; [parameters])

Simulate nreals realizations of variables varsᵢ with geostatistical process processᵢ over given domain or vector of geometries [g₁, g₂, ..., gₙ].

The parameters are forwarded to the rand method of the geostatistical processes.

CookieCutter(domain, parent => process, var₁ => procmap₁, ..., varₙ => procmapₙ; [parameters])
CookieCutter(domain, nreals, parent => process, var₁ => procmap₁, ..., varₙ => procmapₙ; [parameters])

Simulate nreals realizations of variable parent with geostatistical process process, and each child variable varsᵢ with process map procmapᵢ, over given domain.

The process map must be an iterable of pairs of the form: value => process. Each process in the map is related to a value of the parent realization, therefore the values of the child variables will be chosen according to the values of the corresponding parent realization.

The parameters are forwarded to the rand method of the geostatistical processes.

Examples

parent = QuiltingProcess(trainimg, (30, 30))
child0 = GaussianProcess(SphericalVariogram(range=20.0, sill=0.2))
child1 = GaussianProcess(SphericalVariogram(MetricBall((200.0, 20.0))))
transform = CookieCutter(domain, :parent => parent, :child => [0 => child0, 1 => child1])