GitHub

Overview

A Julia package for fast 3D image quilting simulation.

Build Status Coverage Status Stable Documentation Latest Documentation

This package implements an extension to the famous Efros-Freeman algorithm for texture synthesis and transfer in computer vision. Unlike the original algorithm developed for 2D images, our method can also handle 3D masked grids and pre-existing point-data very efficiently (the fastest in the literature). For more details, please refer to our paper in Citation.

3D Quilting Animation

Features

  • Masked grids
  • Hard data conditioning
  • Soft data conditioning
  • Fast computation with GPUs

Installation

Get the latest stable release with Julia's package manager:

] add ImageQuilting

Talks

Below is a list of talks related to this project. For more material, please subscribe to the YouTube channel.

Usage

This package is part of the GeoStats.jl framework. See its documentation for examples of practical use.

Low-level API

If you are interested in using the package without GeoStats.jl, please use the following function:

ImageQuilting.iqsim — Function
iqsim(trainimg::AbstractArray{T,N}, tilesize::Dims{N},
      simsize::Dims{N}=size(trainimg);
      overlap::NTuple{N,<:Real}=ntuple(i->1/6,N),
      soft::AbstractVector=[], hard::Dict=Dict(), tol::Real=.1,
      path::Symbol=:raster, nreal::Integer=1,
      threads::Integer=cpucores(), debug::Bool=false,
      progress::Bool=true, rng::AbstractRNG=Random.default_rng())

Performs image quilting simulation as described in Hoffimann et al. 2017.

Parameters

Required

  • trainimg is any Julia array
  • tilesize is the tile size

Optional

  • simsize is the size of the simulation grid (default to training image size)
  • overlap is the percentage of overlap (default to 1/6 of tile size)
  • soft is a vector of (data,dataTI) pairs (default to none)
  • hard is a dictionary mapping coordinates to data values (default to none)
  • tol is the initial relaxation tolerance in (0,1] (default to .1)
  • path is the simulation path (:raster, :dilation or :random)
  • nreal is the number of realizations (default to 1)
  • threads is the number of threads for the FFT (default to all CPU cores)
  • debug informs whether to export or not the boundary cuts and voxel reuse
  • progress informs whether to show or not estimated time duration
  • rng is the random number generator (default to Random.default_rng())

The main output reals consists of a list of realizations that can be indexed with reals[1], reals[2], ..., reals[nreal]. If debug=true, additional output is generated:

reals, cuts, voxs = iqsim(..., debug=true)

cuts[i] is the boundary cut for reals[i] and voxs[i] is the associated voxel reuse.

source

The major difference compared to the high-level API is that the iqsim function has no notion of coordinate system, and you will have to pre/post-process the data manually to match it with the cells in the simulation grid.

GeoStats.jl takes the coordinate system into account and also enables parallel simulation on clusters of computers with distributed memory.