10-15-2022, 07:16 PM
(10-15-2022, 01:23 PM)DennisSoemers Wrote: For our built-in MCTS, this issue is now fixed in this commit to our dev branch: https://github.com/Ludeme/Ludii/commit/3...89e947c030
The core tree reuse code that you've probably seen close to the start of the selectAction() method (approx around lines 477-540ish) was already correct. However, at the end of the method, I also already did a little bit of traversing (of just the action that we're planning to take, but not opponents' actions yet since they didn't move yet) to be able to clean up a small amount of memory (around lines 799-818). There, I was using the "returnMove" instead of a Move object extracted from a Trial (because we are traversing the move that we are planning to play, rather than one already played), and that one was in a different format (with consequents still in Ludeme-rule-format instead of having been converted into concrete Actions).
Thank you for your reply.
I finally understood the problem :)
I found very cleaver your solution for clear memory and also move the root for the next known step. (which is the one that the agent actually selected to do).
I'm not sure if I'm right or wrong but, it seems that the move format divergence can be avoided only for situations where the agent knows the subsequent action.
To give one example where I think it should not work: when the opponent made a move, with the same situation where the concrete move is different than the Ludeme-rule-format ([move ...], [setPlayer...]]), the agent still won't find the opponent step in the traversed nodes, because the only way to actually search the move is using the actionHistory list.
Let me know if it works on these situation. (I tested in Amazons and it looks like it is not working).
> I want to share a temporary solution (which is terrible and I'm not proud of it) but it seems to work in the situation mentioned before:
In the findChildForMove method, inside the children iteration, instead of comparing the 'move' parameter with child.parentMove (line 124 from DeterministicNode), create a copy of the node.context and apply the child.parentMove in the copied context,
something like: copyContext.game().apply(cpyContext, child.moveFromParent).
The 'concrete action' returned from copyContext.trial().lastMove() should have the same format as the one got from the actionHistory list.
The if condition should now be: concreteAction.equals(move)
It looks like it is working, but a different approach should be made for the clean memory situation. The 'returnMove' should be transformed into a concrete action (the same way previously shown) before passing it in the findChildForMove, in order to find the right child node.