Initialization of conjugate gradient in competitive_optimizer

[f-t-s]

Hi,

I saw that you have implemented competitive gradient descent in your dev-branch, thanks for giving it a try @mikkel !
I just wanted to mention that you seem to have a bug that we also found in our pytorch implementation recently, and will correct shortly.

In the definition of mgeneral_conjugate_gradient you initialize the residual with

r = [tf.identity(_b) for _b in b]

This is correct if the initial guess is zero, otherwise (if A is the matrix to be inverted), it should be
r = b - Ax_0

where x_0 is the initial guess, see also here

Initializing the residual wrongly can lead to different results and we found that there are cases where this bug breaks cgd, while the fixed version performs well.

[martyn]

Sorry for missing this. Conjugate gradient is awesome, thank you for your work! We've rewritten hypergan in pytorch and are working to re-add conjugate gradient.