snn.RLeaky

class snntorch._neurons.rleaky.RLeaky(beta, V=1.0, all_to_all=True, linear_features=None, conv2d_channels=None, kernel_size=None, threshold=1.0, spike_grad=None, surrogate_disable=False, init_hidden=False, inhibition=False, learn_beta=False, learn_threshold=False, learn_recurrent=True, reset_mechanism='subtract', state_quant=False, output=False, reset_delay=True)

Bases: LIF

First-order recurrent leaky integrate-and-fire neuron model. Input is assumed to be a current injection appended to the voltage spike output. Membrane potential decays exponentially with rate beta. For \(U[T] > U_{\rm thr} ⇒ S[T+1] = 1\).

If reset_mechanism = “subtract”, then \(U[t+1]\) will have threshold subtracted from it whenever the neuron emits a spike:

\[U[t+1] = βU[t] + I_{\rm in}[t+1] + V(S_{\rm out}[t]) - RU_{\rm thr}\]

Where \(V(\cdot)\) acts either as a linear layer, a convolutional operator, or elementwise product on \(S_{\rm out}\).

• If all_to_all = “True” and linear_features is specified, then \(V(\cdot)\) acts as a recurrent linear layer of the same size as \(S_{\rm out}\).

• If all_to_all = “True” and conv2d_channels and kernel_size are specified, then \(V(\cdot)\) acts as a recurrent convlutional layer with padding to ensure the output matches the size of the input.

• If all_to_all = “False”, then \(V(\cdot)\) acts as an elementwise multiplier with \(V\).

• If reset_mechanism = “zero”, then \(U[t+1]\) will be set to 0 whenever the neuron emits a spike:

\[U[t+1] = βU[t] + I_{\rm in}[t+1] + V(S_{\rm out}[t]) - R(βU[t] + I_{\rm in}[t+1] + V(S_{\rm out}[t]))\]

• \(I_{\rm in}\) - Input current

• \(U\) - Membrane potential

• \(U_{\rm thr}\) - Membrane threshold

• \(S_{\rm out}\) - Output spike

• \(R\) - Reset mechanism: if active, \(R = 1\), otherwise \(R = 0\)

• \(β\) - Membrane potential decay rate

• \(V\) - Explicit recurrent weight when all_to_all=False

Example:

import torch
import torch.nn as nn
import snntorch as snn

beta = 0.5 # decay rate
V1 = 0.5 # shared recurrent connection
V2 = torch.rand(num_outputs) # unshared recurrent connections

# Define Network
class Net(nn.Module):
    def __init__(self):
        super().__init__()

        # initialize layers
        self.fc1 = nn.Linear(num_inputs, num_hidden)

        # Default RLeaky Layer where recurrent connections
        # are initialized using PyTorch defaults in nn.Linear.
        self.lif1 = snn.RLeaky(beta=beta,
                    linear_features=num_hidden)

        self.fc2 = nn.Linear(num_hidden, num_outputs)

        # each neuron has a single connection back to itself
        # where the output spike is scaled by V.
        # For `all_to_all = False`, V can be shared between
        # neurons (e.g., V1) or unique / unshared between
        # neurons (e.g., V2).
        # V is learnable by default.
        self.lif2 = snn.RLeaky(beta=beta, all_to_all=False, V=V1)

    def forward(self, x):
        # Initialize hidden states at t=0
        spk1, mem1 = self.lif1.init_rleaky()
        spk2, mem2 = self.lif2.init_rleaky()

        # Record output layer spikes and membrane
        spk2_rec = []
        mem2_rec = []

        # time-loop
        for step in range(num_steps):
            cur1 = self.fc1(x)
            spk1, mem1 = self.lif1(cur1, spk1, mem1)
            cur2 = self.fc2(spk1)
            spk2, mem2 = self.lif2(cur2, spk2, mem2)

            spk2_rec.append(spk2)
            mem2_rec.append(mem2)

        # convert lists to tensors
        spk2_rec = torch.stack(spk2_rec)
        mem2_rec = torch.stack(mem2_rec)

        return spk2_rec, mem2_rec

Parameters:

• beta (float or torch.tensor) – membrane potential decay rate. Clipped between 0 and 1 during the forward-pass. May be a single-valued tensor (i.e., equal decay rate for all neurons in a layer), or multi-valued (one weight per neuron).

• V (float or torch.tensor) – Recurrent weights to scale output spikes, only used when all_to_all=False. Defaults to 1.

• all_to_all (bool, optional) – Enables output spikes to be connected in dense or convolutional recurrent structures instead of 1-to-1 connections. Defaults to True.

• linear_features (int, optional) – Size of each output sample. Must be specified if all_to_all=True and the input data is 1D. Defaults to None

• conv2d_channels (int, optional) – Number of channels in each output sample. Must be specified if all_to_all=True and the input data is 3D. Defaults to None

• kernel_size (int or tuple) – Size of the convolving kernel. Must be specified if all_to_all=True and the input data is 3D. Defaults to None

• threshold – Threshold for \(mem\) to reach in order to generate a spike S=1. Defaults to 1 :type threshold: float, optional

• spike_grad (surrogate gradient function from snntorch.surrogate, optional) – Surrogate gradient for the term dS/dU. Defaults to None (corresponds to ATan surrogate gradient. See snntorch.surrogate for more options)

• surrogate_disable (bool, Optional) – Disables surrogate gradients regardless of spike_grad argument. Useful for ONNX compatibility. Defaults to False

• init_hidden – Instantiates state variables as instance variables. Defaults to False :type init_hidden: bool, optional

• inhibition – If True, suppresses all spiking other than the neuron with the highest state. Defaults to False :type inhibition: bool, optional

• learn_beta (bool, optional) – Option to enable learnable beta. Defaults to False

• learn_recurrent (bool, optional) – Option to enable learnable recurrent weights. Defaults to True

• learn_threshold (bool, optional) – Option to enable learnable threshold. Defaults to False

• reset_mechanism (str, optional) – Defines the reset mechanism applied to \(mem\) each time the threshold is met. Reset-by-subtraction: “subtract”, reset-to-zero: “zero”, none: “none”. Defaults to “subtract”

• state_quant (quantization function from snntorch.quant, optional) – If specified, hidden state \(mem\) is quantized to a valid state for the forward pass. Defaults to False

• output – If True as well as init_hidden=True, states are returned when neuron is called. Defaults to False :type output: bool, optional

Inputs: input_, spk_0, mem_0

• input_ of shape (batch, input_size): tensor containing

input

features

• spk_0 of shape (batch, input_size): tensor containing

output

spike features

• mem_0 of shape (batch, input_size): tensor containing the initial membrane potential for each element in the batch.

Outputs: spk_1, mem_1

• spk_1 of shape (batch, input_size): tensor containing the

output

spikes.

• mem_1 of shape (batch, input_size): tensor containing

the next

membrane potential for each element in the batch

Learnable Parameters:

• RLeaky.beta (torch.Tensor) - optional learnable weights

must be

manually passed in, of shape 1 or (input_size).

• RLeaky.recurrent.weight (torch.Tensor) - optional learnable weights are automatically generated if all_to_all=True. RLeaky.recurrent stores a nn.Linear or nn.Conv2d layer depending on input arguments provided.

• RLeaky.V (torch.Tensor) - optional learnable weights must be manually passed in, of shape 1 or (input_size). It is only used where all_to_all=False for 1-to-1 recurrent connections.

• RLeaky.threshold (torch.Tensor) - optional learnable

  thresholds must be manually passed in, of shape 1 or`` (input_size).

classmethod detach_hidden()

Returns the hidden states, detached from the current graph. Intended for use in truncated backpropagation through time where hidden state variables are instance variables.

forward(input_, spk=None, mem=None)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_rleaky()

Deprecated, use RLeaky.reset_mem instead

classmethod reset_hidden()

Used to clear hidden state variables to zero. Intended for use where hidden state variables are instance variables. Assumes hidden states have a batch dimension already.

reset_mem()

class snntorch._neurons.rleaky.RecurrentOneToOne(V)

Bases: Module

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.