-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Autodiff across FE result #188
Comments
What do you have in mind? If you use the lower level code in FixedEffects.jl, there is some compatibility with AD. using ForwardDiff, FiniteDiff, FixedEffects, LinearAlgebra
N = 10
K = 2
d1=repeat(1:(N÷2), inner = 2)
d2=repeat(1:2, inner = N÷2)
p1 = FixedEffect(d1)
p2 = FixedEffect(d2)
x = rand(N,K)
y = rand(N)
function regfe(y,x,fes)
x = copy(x)
y = copy(y)
w = FixedEffects.uweights(eltype(x), size(x, 1))
feM = AbstractFixedEffectSolver{eltype(x)}(fes, w, Val{:cpu}, Threads.nthreads())
solve_residuals!(x, feM)
return((x'*x) \ (x'*y))
end
regfe(y, x, [p1, p2])
# derivative wrt x
J1 = ForwardDiff.jacobian(x->regfe(y, reshape(x,N,K), [p1, p2]) , vec(x))
J2 = FiniteDiff.finite_difference_jacobian(x->regfe(y, reshape(x,N,K), [p1, p2]) , vec(x))
norm(J1- J2)
# derivative wrt y
J1 = ForwardDiff.jacobian(y->regfe(y, reshape(x,N,K), [p1, p2]) , y)
J2 = FiniteDiff.finite_difference_jacobian(y->regfe(y, reshape(x,N,K), [p1, p2]) , y)
norm(J1- J2) It's dreadfully slow, but appears to give correct results. Reverse mode packages are probably harder. The in-place mutation of |
Someone noted to me that FixedEffectModels.jl is tricky to use AD on because there are so many explicit
Float64
type constraints -- does anyone have a good sense of how much effort it would take to remove/parameterize/reduce the explicit type constraints here?As an example, the
FixedEffectModel
struct has a lot ofFloat64
explicit types that could be parametric instead.FixedEffectModels.jl/src/FixedEffectModel.jl
Lines 8 to 45 in 2a2661e
Is there an appetite for this? I think it'd be lovely to be able to AD through high-dimensional fixed effect estimates.
The text was updated successfully, but these errors were encountered: