[WIP] Add training loops

[Joshua-Chin]

This adds a convenience class the helps train and evaluate networks, using a scikit-learn style interface.

[benanne]

This is great, but I'm not sure if we would want something like this in the main library at this point. I'm open to discussing it though! Note that there is also nolearn which provides the same functionality.

I still plan to come up with some training loop utilities that don't involve encapsulating the model into a class, which tends to fit my own workflow better -- but I guess that's mostly a matter of preference. Hopefully I can draft a proposal for that sometime soon (I've been postponing it for a while now...)

Hopefully I'll have some time to take a closer look at this this weekend. At a glance, it seems like it might actually be fairly limited in scope (e.g. only supporting (X, y)-style problems, i.e. supervised learning) and we might want something more broadly applicable. There is a fairly lengthy discussion at #12 which is probably worth (re-)reading as it touches some of these concerns.

[f0k]

Same here... we'd love to have some generic training loop code in Lasagne to make it easier to get started, but we need to get it right! This means the following restrictions of your proposal would need to be lifted:

• restriction to "single input, single output" network
• restriction to "X -> y" problems
• restriction to datasets that fit into host memory at once

The toughest part probably is that the training loop should cooperate with any kind of data iterator. We can provide one for the simple case of iterating over a numpy array, but other than that, it should support whatever people have cooked up already. This will still need some discussion.

Thank you for bringing this back to our attention, though, and feel free to add your two cents to #12! Meanwhile, nolearn provides what you proposed and then some more.

[Joshua-Chin]

Generic training loop code would be great for Lasagne, but common use cases should be easy. Many of the applications for neural networks can be phrased as "X -> y" problems.

I have also been working on a more generic batch iterator, that handles both numpy arrays and generators over individual examples. I plan to create a pull request for it sometime this coming week.

[benanne]

I agree that (X, y) should be the default, but if we add this to the main library it should definitely be more flexible than that.