surrogate¶

How to use surrogate¶

class dynapsetorch.surrogate.FastSigmoid(*args, **kwargs)[source]¶

Bases: Function

Fast-sigmoid surrogated gradient Apply the fast-sigmoid gradient as a surrogated gradient for the heavyside step function.

\[\frac{\partial S}{\partial V} = \frac{1}{(\lambda \left|v\right| + 1.0)^2}\]

Where \(\lambda\) is a scale factor with default value 10.

static pseudo_derivative(v)[source]¶

Compute the gradient of the fast-sigmoid function.

class dynapsetorch.surrogate.STE(*args, **kwargs)[source]¶

Bases: Function

Straight Through Estimator

static pseudo_derivative(v)[source]¶

Compute the STE surrogate gradient.

class dynapsetorch.surrogate.Step(*args, **kwargs)[source]¶

Bases: Function

Step function surrogated gradient Use the step function as a surrogated gradient of itself

static pseudo_derivative(V)[source]¶

Compute the step function surrogate gradient.

class dynapsetorch.surrogate.Triangular(*args, **kwargs)[source]¶

Bases: Function

Triangular surrogated gradient Apply the triangular function as a surrogated gradient for the heavyside step function.

\[\frac{\partial S}{\partial V} = \lambda max(1 - \left|V\right|, 0)\]

Where \(\lambda\) is a scale factor with default value 0.3.

static pseudo_derivative(v)[source]¶

Compute the triangular surrogate gradient.

model Networks