GitHub

Geospatial transforms

We provide a very powerful list of transforms that were designed to work seamlessly with geospatial data. These transforms are implemented in different packages depending on how they interact with geometries.

The list of supported transforms is continuously growing. The following code can be used to print an updated list in any project environment:

# packages to print type tree
using InteractiveUtils
using AbstractTrees
using TransformsBase

# packages with transforms
using GeoStats

# define the tree of types
AbstractTrees.children(T::Type) = subtypes(T)

# print all currently available transforms
AbstractTrees.print_tree(TransformsBase.Transform)
Transform
├─ GeometricTransform
│  ├─ Bridge
│  ├─ CoordinateTransform
│  │  ├─ Affine
│  │  ├─ LengthUnit
│  │  ├─ Morphological
│  │  ├─ Proj
│  │  ├─ Rotate
│  │  ├─ Scale
│  │  ├─ Stretch
│  │  └─ Translate
│  ├─ LambdaMuSmoothing
│  ├─ Repair
│  ├─ Shadow
│  ├─ Slice
│  ├─ StdCoords
│  └─ ValidCoords
├─ Identity
├─ TableTransform
│  ├─ Aggregate
│  ├─ Detrend
│  ├─ Downscale
│  ├─ FeatureTransform
│  │  ├─ EigenAnalysis
│  │  ├─ Remainder
│  │  ├─ ColwiseFeatureTransform
│  │  │  ├─ Center
│  │  │  ├─ Coalesce
│  │  │  ├─ LowHigh
│  │  │  ├─ Quantile
│  │  │  └─ ZScore
│  │  └─ StatelessFeatureTransform
│  │     ├─ AbsoluteUnits
│  │     ├─ Assert
│  │     ├─ Closure
│  │     ├─ Coerce
│  │     ├─ ColTable
│  │     ├─ Compose
│  │     ├─ DropConstant
│  │     ├─ DropExtrema
│  │     ├─ DropMissing
│  │     ├─ DropNaN
│  │     ├─ DropUnits
│  │     ├─ Filter
│  │     ├─ Functional
│  │     ├─ Indicator
│  │     ├─ Learn
│  │     ├─ Levels
│  │     ├─ Map
│  │     ├─ OneHot
│  │     ├─ ProjectionPursuit
│  │     ├─ Reject
│  │     ├─ Rename
│  │     ├─ Replace
│  │     ├─ RowTable
│  │     ├─ Sample
│  │     ├─ Satisfies
│  │     ├─ Select
│  │     ├─ Sort
│  │     ├─ StdFeats
│  │     ├─ StdNames
│  │     ├─ LogRatio
│  │     │  ├─ ALR
│  │     │  ├─ CLR
│  │     │  └─ ILR
│  │     ├─ Unit
│  │     └─ Unitify
│  ├─ ClusteringTransform
│  │  ├─ GHC
│  │  ├─ GSC
│  │  └─ SLIC
│  ├─ Interpolate
│  ├─ InterpolateNeighbors
│  ├─ Potrace
│  ├─ Rasterize
│  ├─ ParallelTableTransform
│  ├─ Transfer
│  ├─ UniqueCoords
│  └─ Upscale
└─ SequentialTransform

Transforms at the leaves of the tree above should have a docstring with more information on the available options. For example, the documentation of the Select transform is shown below:

TableTransforms.Select — Type
Select(col₁, col₂, ..., colₙ)
Select([col₁, col₂, ..., colₙ])
Select((col₁, col₂, ..., colₙ))

The transform that selects columns col₁, col₂, ..., colₙ.

Select(col₁ => newcol₁, col₂ => newcol₂, ..., colₙ => newcolₙ)

Selects the columns col₁, col₂, ..., colₙ and rename them to newcol₁, newcol₂, ..., newcolₙ.

Select(regex)

Selects the columns that match with regex.

Examples

Select(1, 3, 5)
Select([:a, :c, :e])
Select(("a", "c", "e"))
Select(1 => :x, 3 => :y)
Select(:a => :x, :b => :y)
Select("a" => "x", "b" => "y")
Select(r"[ace]")
source

Transforms of type FeatureTransform operate on the attribute table, whereas transforms of type GeometricTransform operate on the underlying geospatial domain:

TableTransforms.FeatureTransform — Type
FeatureTransform

A transform that operates on the columns of the table containing features, i.e., simple attributes such as numbers, strings, etc.

source

Other transforms such as Detrend are defined in terms of both the geospatial domain and the attribute table. All transforms and pipelines implement the following functions:

TransformsBase.isrevertible — Function
isrevertible(transform)

Tells whether or not the transform is revertible, i.e. supports a revert function. Defaults to false for new transform types.

Transforms can be revertible and yet don't be invertible. Invertibility is a mathematical concept, whereas revertibility is a computational concept.

See also isinvertible.

source
TransformsBase.isinvertible — Function
isinvertible(transform)

Tells whether or not the transform is invertible, i.e. whether it implements the inverse function. Defaults to false for new transform types.

Transforms can be invertible in the mathematical sense, i.e., there exists a one-to-one mapping between input and output spaces.

See also inverse, isrevertible.

source
TransformsBase.apply — Function
newobject, cache = apply(transform, object)

Apply transform on the object. Return the newobject and a cache to revert the transform later.

source
TransformsBase.revert — Function
object = revert(transform, newobject, cache)

Revert the transform on the newobject using the cache from the corresponding apply call and return the original object. Only defined when the transform isrevertible.

source
TransformsBase.reapply — Function
newobject = reapply(transform, object, cache)

Reapply the transform to (a possibly different) object using a cache that was created with a previous apply call. Fallback to apply without using the cache.

source

Feature transforms

Please check the TableTransforms.jl documentation for an updated list of feature transforms. As an example consider the following features over a Cartesian grid and their statistics:

using DataFrames

# table of features and domain
tab = DataFrame(a=rand(1000), b=randn(1000), c=rand(1000))
dom = CartesianGrid(100, 100)

# georeference table onto domain
Ω = georef(tab, dom)

# describe features
describe(values(Ω))
3×7 DataFrame
Rowvariablemeanminmedianmaxnmissingeltype
SymbolFloat64Float64Float64Float64Int64DataType
1a0.50565.09529e-50.5041340.9977010Float64
2b0.000376204-3.003260.03134272.811830Float64
3c0.4950240.001393660.4931610.999190Float64

We can create a pipeline that transforms the features to their normal quantile (or scores):

pipe = Quantile()

Ω̄, cache = apply(pipe, Ω)

describe(values(Ω̄))
3×7 DataFrame
Rowvariablemeanminmedianmaxnmissingeltype
SymbolFloat64Float64Float64Float64Int64DataType
1a0.00309023-3.090230.001253323.090230Float64
2b0.00309023-3.090230.001253323.090230Float64
3c0.00309023-3.090230.001253323.090230Float64

We can then revert the transform given any new geospatial data in the transformed sample space:

Ωₒ = revert(pipe, Ω̄, cache)

describe(values(Ωₒ))
3×7 DataFrame
Rowvariablemeanminmedianmaxnmissingeltype
SymbolFloat64Float64Float64Float64Int64DataType
1a0.5061050.0008309440.5044030.9976380Float64
2b0.00327679-2.995930.0322132.708780Float64
3c0.4955180.004530020.4940740.9987620Float64

Geometric transforms

Please check the Meshes.jl documentation for an updated list of geometric transforms. As an example consider the rotation of geospatial data over a Cartesian grid:

# geospatial domain
Ω = georef((Z=rand(10, 10),))

# apply geometric transform
Ωr = Ω |> Rotate(π/4)

fig = Mke.Figure(size = (800, 400))
viz(fig[1,1], Ω.geometry, color = Ω.Z)
viz(fig[1,2], Ωr.geometry, color = Ωr.Z)
fig
Example block output

Geostatistical transforms

Bellow is the current list of transforms that operate on both the geometries and features of geospatial data. They are implemented in the GeoStatsBase.jl package.

UniqueCoords

GeoStatsTransforms.UniqueCoords — Type
UniqueCoords(var₁ => agg₁, var₂ => agg₂, ..., varₙ => aggₙ)

Retain locations in data with unique coordinates.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

UniqueCoords(1 => last, 2 => maximum)
UniqueCoords(:a => first, :b => minimum)
UniqueCoords("a" => last, "b" => maximum)
source
# point set with repeated points
p = rand(Point, 50)
Ω = georef((Z=rand(100),), [p; p])
100×2 GeoTable over 100 PointSet
Z geometry
Continuous Point
[NoUnits] 🖈 Cartesian{NoDatum}
0.478016 (x: 0.815521 m, y: 0.434568 m, z: 0.703501 m)
0.56295 (x: 0.124533 m, y: 0.907696 m, z: 0.36937 m)
0.352043 (x: 0.792637 m, y: 0.821407 m, z: 0.975597 m)
0.791411 (x: 0.447635 m, y: 0.790791 m, z: 0.0183913 m)
0.0595895 (x: 0.0826312 m, y: 0.0798377 m, z: 0.525962 m)
0.251043 (x: 0.0487548 m, y: 0.980991 m, z: 0.739716 m)
0.212703 (x: 0.243145 m, y: 0.28123 m, z: 0.518337 m)
0.327558 (x: 0.47038 m, y: 0.735805 m, z: 0.733122 m)
0.34889 (x: 0.782202 m, y: 0.0962442 m, z: 0.0657916 m)
0.714239 (x: 0.241544 m, y: 0.881138 m, z: 0.878898 m)
⋮ ⋮ 
# discard repeated points
𝒰 = Ω |> UniqueCoords()
50×2 GeoTable over 50 view(::PointSet, [1, 2, 3, 4, ..., 47, 48, 49, 50])
Z geometry
Continuous Point
[NoUnits] 🖈 Cartesian{NoDatum}
0.536171 (x: 0.815521 m, y: 0.434568 m, z: 0.703501 m)
0.707335 (x: 0.124533 m, y: 0.907696 m, z: 0.36937 m)
0.437504 (x: 0.792637 m, y: 0.821407 m, z: 0.975597 m)
0.677722 (x: 0.447635 m, y: 0.790791 m, z: 0.0183913 m)
0.386083 (x: 0.0826312 m, y: 0.0798377 m, z: 0.525962 m)
0.205564 (x: 0.0487548 m, y: 0.980991 m, z: 0.739716 m)
0.28043 (x: 0.243145 m, y: 0.28123 m, z: 0.518337 m)
0.649145 (x: 0.47038 m, y: 0.735805 m, z: 0.733122 m)
0.360961 (x: 0.782202 m, y: 0.0962442 m, z: 0.0657916 m)
0.731694 (x: 0.241544 m, y: 0.881138 m, z: 0.878898 m)
⋮ ⋮

Detrend

GeoStatsTransforms.Detrend — Type
Detrend(col₁, col₂, ..., colₙ; degree=1)
Detrend([col₁, col₂, ..., colₙ]; degree=1)
Detrend((col₁, col₂, ..., colₙ); degree=1)

The transform that detrends columns col₁, col₂, ..., colₙ with a polynomial of given degree.

Detrend(regex; degree=1)

Detrends the columns that match with regex.

Examples

Detrend(1, 3, 5)
Detrend([:a, :c, :e])
Detrend(("a", "c", "e"))
Detrend(r"[ace]", degree=2)
Detrend(:)

References

source
# quadratic trend + random noise
r = range(-1, stop=1, length=100)
μ = [x^2 + y^2 for x in r, y in r]
ϵ = 0.1rand(100, 100)
Ω = georef((Z=μ+ϵ,))

# detrend and obtain noise component
𝒩 = Ω |> Detrend(:Z, degree=2)

fig = Mke.Figure(size = (800, 400))
viz(fig[1,1], Ω.geometry, color = Ω.Z)
viz(fig[1,2], 𝒩.geometry, color = 𝒩.Z)
fig
Example block output

Potrace

GeoStatsTransforms.Potrace — Type
Potrace(mask; [ϵ])
Potrace(mask, var₁ => agg₁, ..., varₙ => aggₙ; [ϵ])

Trace polygons on 2D image data with Selinger's Potrace algorithm.

The categories stored in column mask are converted into binary masks, which are then traced into multi-polygons. When provided, the option ϵ is forwarded to Selinger's simplification algorithm.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

Potrace(:mask, ϵ=0.1)
Potrace(1, 1 => last, 2 => maximum)
Potrace(:mask, :a => first, :b => minimum)
Potrace("mask", "a" => last, "b" => maximum)

References

source
# continuous feature
Z = [sin(i/10) + sin(j/10) for i in 1:100, j in 1:100]

# binary mask
M = Z .> 0

# georeference data
Ω = georef((Z=Z, M=M))

# trace polygons using mask
𝒯 = Ω |> Potrace(:M)

fig = Mke.Figure(size = (800, 400))
viz(fig[1,1], Ω.geometry, color = Ω.Z)
viz(fig[1,2], 𝒯.geometry, color = 𝒯.Z)
fig
Example block output
𝒯.geometry
2 GeometrySet
├─ Multi(4×PolyArea)
└─ Multi(4×PolyArea)

Rasterize

GeoStatsTransforms.Rasterize — Type
Rasterize(grid)
Rasterize(grid, var₁ => agg₁, ..., varₙ => aggₙ)

Rasterize geometries within specified grid.

Rasterize(nx, ny)
Rasterize(nx, ny, var₁ => agg₁, ..., varₙ => aggₙ)

Alternatively, use the grid with size nx by ny obtained with discretization of the bounding box.

Duplicates of a variable varᵢ are aggregated with aggregation function aggᵢ. If an aggregation function is not defined for variable varᵢ, the default aggregation function will be used. Default aggregation function is mean for continuous variables and first otherwise.

Examples

grid = CartesianGrid(10, 10)
Rasterize(grid)
Rasterize(10, 10)
Rasterize(grid, 1 => last, 2 => maximum)
Rasterize(10, 10, 1 => last, 2 => maximum)
Rasterize(grid, :a => first, :b => minimum)
Rasterize(10, 10, :a => first, :b => minimum)
Rasterize(grid, "a" => last, "b" => maximum)
Rasterize(10, 10, "a" => last, "b" => maximum)
source
A = [1, 2, 3, 4, 5]
B = [1.1, 2.2, 3.3, 4.4, 5.5]
p1 = PolyArea((2, 0), (6, 2), (2, 2))
p2 = PolyArea((0, 6), (3, 8), (0, 10))
p3 = PolyArea((3, 6), (9, 6), (9, 9), (6, 9))
p4 = PolyArea((7, 0), (10, 0), (10, 4), (7, 4))
p5 = PolyArea((1, 3), (5, 3), (6, 6), (3, 8), (0, 6))
gt = georef((; A, B), [p1, p2, p3, p4, p5])

nt = gt |> Rasterize(20, 20)

viz(nt.geometry, color = nt.A)
Example block output