-
-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Include searchdirection
in OptimizationState
#723
Comments
Well, I thought I could, but it turns out the
So it's missing a lot of information, and doesn't forward any internal |
Yes, we need to fill it in a bit more per optimizer, and not all optimizers have al linformation. |
It would be convenient if OptimizationState included a
searchdirection
and/or a step width. I can usually pull these quantities out of theoriginal
state, but it would be nice to have them in a well-defined location.For simple gradient descent, the search direction would be the negative gradient, but e.g. for LBFGS it would be the direction obtained from the internal quasi-Hessian. Knowing the search direction allows to really drill down into the linesearch, see, e.g., the plots in the README of
GRAPELinesearchAnalysis.jl
. In principle, I can infer the search direction as the difference inu
from the previous iteration, but then I don't know the step width (how far the optimizer went in the direction of the search direction). Knowing both quantities directly from the solver is useful for actually checking that the update is in the search direction, but if that's guaranteed, only one of them would be sufficient.The text was updated successfully, but these errors were encountered: