Experimental functions

conv

1D or 2D Convolution for input by filter, nice Convolution arithmetic explanation

(conv in filter {pad {stride {dilation}}})

parameter

• in: input vector or matrix (or timeserie of vector or matrix)
• filter: kernel tensor.
• pad: (optional default is 0), padding number, 1 integer or 2 4 integer array
• stride: (optional default is 1), stride number, 1 integer or [int int]
• dilation: (optional default is 1), dilation factor, 1 integer or [int int]

Sobel Horizontal

Sobel operator applied on 1st component (red) of an image.
Shows original tensor, filter kernel and result.

(def
;get 1st (red) component of image
gray (get (import "file://~/http_server/images/tix_color.png") 0)
sobel-x (tensor (shape 3 3) [
-1 0 1
-2 0 2
-1 0 1])
sobel-y (transpose sobel-x)
)
[
gray
sobel-x
(conv gray sobel-x)
]

Applies horizontal and veritcal Sobel operator on 1st component (red) of an image.
Combined together to find the absolute magnitude of the gradient at each point.

(def
;get 1st (red) component of image
gray (get (import "file://~/http_server/images/tix_color.png") 0)
sobel-x (tensor (shape 3 3) [-1 0 1 -2 0 2 -1 0 1])
sobel-y (transpose sobel-x)
)
[
gray
(sqrt
(+
(sqr (conv gray sobel-x))
(sqr (conv gray sobel-y))))
]

Sobel RGB combined magnitudes

Computes Sobel magnitude on each RGB channel.
Then combines in a 3D tensor (RGB x X x Y) to have a color image.

(def
tix (import "file://~/http_server/images/tix_color.png")
sobel-x (tensor (shape 3 3) [-1 0 1 -2 0 2 -1 0 1])
sobel-y (transpose sobel-x)
)
(defn one-channel[img filter c]
;apply filter on given channel
(conv (get img c) filter)
)
;compute sobel gradient on given channel
(sqrt
(+
(sqr (one-channel img sobel-x c))
(sqr (one-channel img sobel-y c))))
)
[
tix
;this creates an RGB image
(tensor
]

covariance

Covariance matrix (tensor of size 2) between timeseries. Input timerseries are synchronized, for missing point they are considered piecewise constant.

(covariance ts...)

parameter

• ts: as many timeserie or array of time serie as you want

example

Show covariance matrix between temperatures coming from 10 meteo stations. Latitude must be West (< 0) and timeserie length must be > 1000. Slice is used to compute covariance only for 10 first stations.

WARNING can be slow to compute (>10s): 30 Millions points explored.

(def start 2016-01-01 stop 2018-12-31)
(defn is-usable[code]
(cond
(< (vget (timeserie @"lon" "meteonet" code start stop)) 0) false
(< (len (timeserie @"t" "meteonet" code start stop)) 1000) false
true))
(def stations
(value-as-array (slice (keep (perimeter "meteonet" start) is-usable) 0 10)))
(defn ts[code]
(timeserie @"t" "meteonet" code start stop))
(covariance (map ts stations))

dropout

Dropout randomly replace input value by zero with given probability.

Only called during learning, does nothing if called directly.

(dropout in p)

parameter

• in: input vector or matrix (or timeserie of vector or matrix)
• p: probability to drop input, should be bewteen 0 and 1

import

Import resource from a URI(URL). Still in developpement, only PNG, JPEG and GIF images are imported as a RGBA tensor.

(import uri {checksum})

parameter

• uri: resource uri
• checksum: (optional) ensure resource SHA256 checksum is the expected one

examples

Import Strigi-Form small logo as a RGBA tensor and show it as an image.
URL describes local LispTick home images folder.

(import "file://~/http_server/images/logo_symbole_1x.png")

Same source image from official LispTick, but keep only channel 0 index, the RED.

(get (import "https://lisptick.org/images/logo_symbole_1x.png") 0)

maxpool

1D or 2D MaxPool replaces input by using maximum value in a neighborhood.
Used to reduce input dimensionality.

parameter

• in: input vector or matrix (or timeserie of vector or matrix)
• kernel: kernel size, 1 integer or [int int]
• stride: stride number, 1 integer or [int int]

Maximum

Replaces value by maximum in a 2x2 neighborhood.

(def
;get 1st (red) component of image
gray (get (import "file://~/http_server/images/tix_color.png") 0)
)
[
gray
(maxpool gray 2)
]

Reduce size

Reduces input dimensionality by 2 using maximum in a 2x2 neighborhood.
Zero padding, stride 2 so reduction is by 2.

(def
;get 1st (red) component of image
gray (get (import "file://~/http_server/images/tix_color.png") 0)
)
[
gray
(maxpool gray 2 0 2)
]

shape

Shape of tensor, its an array of int representating each dimension size.

See tensor for examples.

(shape arg1 {arg2 {arg3...}})

parameter

• arg1: size of first dimension, or tensor to get shape from
• arg2: size of second dimension
• argn: size of nth dimension

solve

Cost function optimizer by stochastic gradient descent.

Internally LispTick uses Gorgonia package. All optimizer models, their options and default values are mapped to Gorgonia models and default values.

(solve [learn1 ...] [ts1...] cost epochs [model {(option . value)...})

parameter

• [learn1 …]: array of symbol for learnables
• [ts1 …]: array of timeseries used as argument for cost function
• cost: cost function, inputs are [ts1 …] output is a scalar than will be minimized by solver
• epochs: number of epochs to run
• [model {(arg . value)…}] model name and its optional arguements
Available models, their optional arguments and default value
• (“rate” . 0.001) learn rate
• (“eps” . 1e-8) smoothing factor
• (“l1reg” . none) L1 regularization parameter
• (“l2reg” . none) L2 regularization parameter
• (“clip” . none) clip gradient at
• “adam”: Adaptive Moment Estimation solver (basically RMSProp on steroids) see paper.
• (“rate” . 0.001) learn rate
• (“eps” . 1e-8) smoothing factor
• (“beta1” . 0.9) modifier for means
• (“beta2” . 0.999) modifier for variances
• (“l1reg” . none) L1 regularization parameter
• (“l2reg” . none) L2 regularization parameter
• (“clip” . none) clip gradient at
• “barzilaiborwein”: Barzilai-Borwein performs Gradient Descent in steepest descend direction.
• (“rate” . 0.001) learn rate
• (“clip” . none) clip gradient at
• “momentum”: Momentum is the stochastic gradient descent optimizer with momentum item.
• (“rate” . 0.001) learn rate
• (“momentum” . 0.9) momentum
• (“l1reg” . none) L1 regularization parameter
• (“l2reg” . none) L2 regularization parameter
• (“clip” . none) clip gradient at
• “rmsprop”: RMSPropSolver is a solver that implements Geoffrey Hinton’s RMSProp gradient descent optimization algorithm see paper.
• (“rate” . 0.001) learn rate
• (“eps” . 1e-8) smoothing factor
• (“rho” . 0.999) decay rate/rho
• (“l2reg” . none) L2 regularization parameter
• (“clip” . none) clip gradient at
• “vanilla”: VanillaSolver is standard stochastic gradient descent optimizer.
• (“rate” . 0.001) learn rate
• (“l1reg” . none) L1 regularization parameter
• (“l2reg” . none) L2 regularization parameter
• (“clip” . none) clip gradient at

example

This example shows how to describe, lear and use a NN with one hidden layer to learn simple function like cosinus. You can play with it and change hidden layer size, target functions…

(def
pi      3.14159265359 ;π
hidden  8
size    10000

;randomly initialized weights
w0 (tensor (shape hidden 1) (fn[x] (rand-g 0 1)))
b0 (tensor (shape hidden 1) (fn[x] (rand-g 0 1)))
w1 (tensor (shape 1 hidden) (fn[x] (rand-g 0 1)))

start 2019-01-01
)
(def
;timeserie of size random value between -π and π
ts_angle
(timeserie
(range start (+ (* (- size 1) 1h) start) 1h)
(fn[t] (rand-u (* -1 pi) pi t)))
;target timeserie, simply input cosinus
ts_target (cos ts_angle)
)

;Neural Network transfert function with one hidden layer
(defn transfert[x]
(mat* w1 (sigmoid (+ b0 (mat* w0 x)))))
;cost function, square error
(defn cost[x ref]
(+ (sqr (- (transfert x) ref))))

;trick to call solver by looking at last value
(vget (last
(solve
["w0" "b0" "w1"]
[ts_angle ts_target]
cost
2 ;few epochs

;use learned NN to compute cosinus!
(transfert 0)

svd-s

Singular Value Decomposition, singular values.

Nice article on how to use it for Machine Leaning.

(svd-s matrix)

parameter

• matrix: a 2D tensor

svd-u

Singular Value Decomposition, U orthonormal base.

Nice article on how to use it for Machine Leaning.

(svd-s matrix)

parameter

• matrix: a 2D tensor

svd-v

Singular Value Decomposition, V orthonormal base.

Nice article on how to use it for Machine Leaning.

(svd-s matrix)

parameter

• matrix: a 2D tensor

temporal

Creates a timeserie of temporal vector from a timeserie. From a timeserie of scalar it will keep the n latest updates into a vector of size n. Typical usage is to transform a simple timeserie into an input for a Temporal Convolutional Network.

(temporal ts n)

parameter

• ts: input timeserie
• n: number of updates to keep, i.e. n latest upates

tensor

Creates a tensor, generaly form 1D to 4D.

(tensor shape {values|fn})

Or combine several tensors to create a higher dimension tensor.
Each tensor must have the same shape, result will be n x tensor_dim.

(tensor t1 .. tn)

parameter

• shape: shape of tensor
• values: array of values, sequential (1D)
• fn: a function called sequentially with single index argument in [0 size[
• t1: a tensor
• tn: a tensor with same shape as previous

examples

Hard coded 2D matrices:

(tensor (shape 2 3) [1 2 4 3 6 0])
(tensor (shape 3 2) [1 2 4 3 6 0])

Index as value with an anonymous identity function:

(tensor (shape 3 2) (fn[i] i))

Randomly generated values with rand-g, index unused:

(tensor (shape 8 16) (fn[i] (rand-g 0 1)))

transpose

Tensor tranposition.

(transpose tensor)

example

(def t
(tensor (shape 2 3) [1 2 4 3 6 0]))
(transpose t)