Helper functions for making symmetrical autoencoders with tied params (WIP)

[neosatrapahereje]

Hi!

I added a couple of helper methods for making symmetrical autoencoders using the InverseLayer. Everything is now in lasagne/layers/autoenc.py (including moving the InverseLayer there).

Have a nice weekend!

[f0k]

Looks like a good start!

There are several lines not covered in the tests: https://coveralls.io/builds/3368850/source?filename=lasagne%2Flayers%2Fautoenc.py
When running the tests locally, make sure you get 100% coverage for all the files you've changed.

I'll review and add some more specific comments!

[f0k]

I'd prefer to have special the last one in the list before the ones that need to be imported explicitly. special is the bin for everything falling through the previous categories. So please add autoenc before that!

[alexjc]

@neosatrapahereje Hi, nice work! Very interested in using this. Do you have any plans for this PR?

[benanne]

It looks like this PR was abandoned, but note that you can use arbitrary Theano expressions as layer parameters in Lasagne since last month: http://benanne.github.io/2015/11/10/arbitrary-expressions-as-params.html (see "Autoencoders with tied weights")

I don't know what you have in mind of course, but perhaps this suffices :)

[alexjc]

Thanks @benanne, I will look into it. I'll be adding Lasagne-based auto encoders to scikit-neuralnetwork and I think that will be a good starting point!

[neosatrapahereje]

@alexjc Thanks for your interest in these functions. @f0k helped me to get this code to the current state, and hopefully, will review this PR. Otherwise, you can use the methods directly from here:

https://gist.github.com/neosatrapahereje/69562642b92996fba408

[f0k]

We've squashed and rebased this PR, so it's mergeable again. We just need to decide:

1. Do we want to move InverseLayer to lasagne.layers.autoenc along with the helper functions, or should we add the helper functions somewhere else?
2. Is the code okay? Personally, I find the n_idx and b_idx a bit confusing, and the code will do funny things if the input network is not just a "tower". Can we protect against that?

[rmanor]

This PR seems very useful.
Are there any plants to commit this?
Thanks.

[f0k]

This PR seems very useful.
Are there any plants to commit this?

In hindsight I think this may be too specific / not general enough to be included. It only supports simple architectures of a single stack of layers. We could turn it into a Lasagne Recipe, though, for easier copy/pasting. For now, the easiest will be to copy/paste the functions from https://github.com/neosatrapahereje/Lasagne/blob/a61947c/lasagne/layers/autoenc.py into your code.