Delayed input effect

[LennardBo]

Input currently seems to affect the voltage at the 2nd timestep at the earliest.

I have noticed the reocurring pattern in my network that every LIF layer seemed to halt the forward progression of spikes in the network for a few timesteps. It didn't seem unnatural at first since the neuron's threshold may need several time steps to be overcome. However, even with very small thresholds the problem persisted so I dove depp into the code for the LIF step.

What I found is that in the very first timestep, the current input doesn't influence the voltage at all, as seen here -> it'll only affect the next time step's state.i. Now naturally, this effect will accumulate over several LIF layers; a reasonably sized network of 5 LIF activation layers will, at the very least, need 6 time steps to produce any meaningful output at all.

Now since this is a hopefully intentional design decision I didn't open an issue but wanted to discuss it here. I can see that purely from the underlying ODE's point of view this might be fine. However, from a practical point of view that creates an unnecessary inefficient spiking behaviour.

It seems to me, you might as well calculate i_new first and use that to update the neuron's voltage in Line 16 instead of the old state's i. There might also be some other options to have the current input directly influence the current voltage. I might have missed some theoretical background justifying this, but it looks like that may fit the spiking model, too. Does that seem reasonable to you?

[LennardBo]

As it might be of interest to others: My team actually implemented a wrapper around the LIF neuron to address this problem as it was the easiest solution. We basically set the history to the current input so it takes effect immediately - with phenomenal results. Both rate and latency coding work better and we were able to drastically reduce the sequence length for our classification problem.

I urge the authors to think about maybe changing the internal behaviour of the LIF neuron as described above.

[cpehle]

Thanks for bringing this up, I think you might be totally right that the delay is undesirable. I will look into this now and we surely could change the behaviour without too much trouble.

[cpehle]

One place where this is a potential issue is for recurrence, where this might mean there is no delay.

[cpehle]

I've now created an issue to investigate this here: #325