terminal support

[zsunberg]

This package does not seem to support isterminal.

[mykelk]

Should it? How would it be specified in a file? Zero transition probabilities? The behavior doesn't seem to be specified here:
https://pomdp.org/code/pomdp-file-spec.html

[zsunberg]

Good question. I can't seem to follow my own breadcrumbs back to the original problem, but I think the fundamental issue is what to do with cases like this:

transition(m::InconsistentTerminalPOMDP, s, a) = Deterministic(s+a)
reward(m::InconsistentTerminalPOMDP, s, a) = s^2
isterminal(m::InconsistentTerminalPOMDP, s) = s >= 3

If the solver or file format takes isterminal into account, the solution (try to stop at s=2) will be vastly different than if it doesn't (make s as large as possible). Should this be considered a well-formed POMDP model?

[mykelk]

I believe isterminal should win out over the transition probabilities. If the transition probabilities are all zero, then it will be treated as terminal even if isterminal is false. What do you think of that convention? If this is the case, then I think we don't have to change anything in POMDPFiles.jl.

[zsunberg]

That is indeed the unofficial . However, it puts a very large burden on solver-writers to remember to check. This is an issue that should be clarified in the POMDPs.jl documentation and we should put it in a linter someday (JuliaPOMDP/POMDPs.jl#232).

Currently, POMDPFiles does NOT handle this correctly. It never checks isterminal.

[mykelk]

Ah, got it. It should be checking that for the write command. That should be relatively straightforward to add to the loop here:
https://github.com/JuliaPOMDP/POMDPFiles.jl/blob/master/src/write.jl
Probably right after line 58.