+- Ludii Forum (https://ludii.games/forums)
+-- Forum: General (https://ludii.games/forums/forumdisplay.php?fid=1)
+--- Forum: Discussion (https://ludii.games/forums/forumdisplay.php?fid=4)
+--- Thread: Planned Ludii AIs (/showthread.php?tid=48)
Planned Ludii AIs - Wolfgang Konen - 04-08-2020
1) Is it planned to offer in Ludii agents from reinforcement learning area? E.g. TD, SARSA or Q-Learning? And if so, when?
2) Or is it planned to offer in the future MCTS-like agents which are enhanced in their policies by (deep) neural networks or other learned models?
3) For some games it is possible to have perfect or very good playing agents (e.g. Bouton's agent in Nim or a perfect playing AlphaBeta in ConnectFour or the very strong Edax in Othello/Reversi). Is it planned to include such game-specific agents as a possibility how strong your standard Ludii agents are against such strong agents in these special games?
RE: Planned Ludii AIs - DennisSoemers - 04-09-2020
1) I don't think we're really planning to do much with those single-agent RL algorithms ourselves. It happened to work famously well on Backgammon, but in most other adversarial settings they tend to not do too well. So we're focusing on other training approaches like AlphaZero-like training ourselves. We certainly hope that other people will find it relatively easy to set up their own training runs with their own algorithms though! The only complicated part should probably be finding a good state / action representations for general games (though that's also something we're looking into). We'll try to ramp up documentation on doing all kinds of stuff like this soonish, but in the meantime, if anyone already wants to try they can always ask for pointers here!
2) Yes, we're currently training agents for all games. Not with Deep Neural Networks though, because we're literally talking about hundreds of games and that'd get a bit problematic in terms of hardware... we're mostly working with simpler features ourselves. We're also looking at collaborations involving a bit more hardware, though probably not for ALL games.
3) If such agents are easy to re-implement, it would certainly be possible to do so using our API for agents (https://github.com/Ludeme/LudiiExampleAI). I don't know if we'll make it a priority to implement such agents ourselves directly in Ludii, since we're really more interested in general game playing AIs for a variety of research goals in our project. We could consider including them if others implement them though (if they'd like us to "officially" include them -- of course authors of these kinds of AIs could also keep the right of distributing them to themselves if they prefer
). If we're talking about really complex agents (say, Stockfish for Chess)... I do think it would be really nice if we could get the programs communicating without having to re-implement such an agent in our program from scratch, but really not sure at this moment about a timeline to get something like that working.
RE: Planned Ludii AIs - Michael - 02-23-2022
Let's say I there's a Ludii game for which my laptop is only able to generate fewer than 2 playouts per second.. Is there any hope of learning something about the game from AI agents? Or do I need a different (non-Ludii) implementation of such a game to get anywhere?
I don't know anything about these kinds of things, but I'm learning to code at the moment, so I might be able to learn different technology for this purpose, I just don't have any idea where to begin.. Any pointers?
RE: Planned Ludii AIs - DennisSoemers - 02-24-2022
(02-23-2022, 02:14 PM)Michael Wrote: Let's say I there's a Ludii game for which my laptop is only able to generate fewer than 2 playouts per second.. Is there any hope of learning something about the game from AI agents? Or do I need a different (non-Ludii) implementation of such a game to get anywhere?
I don't know anything about these kinds of things, but I'm learning to code at the moment, so I might be able to learn different technology for this purpose, I just don't have any idea where to begin.. Any pointers?
I imagine that an algorithm that heavily relies on those playouts (like a standard MCTS) would be fairly hopeless in such a scenario.
Algorithms that might be promising in such a case could be:
1) Minimax/AlphaBeta/other variants, which don't use (presumably deep, long and expensive) playouts for evaluations.
2) Variants of MCTS that don't use (long) playouts, but instead truncate them early (or maybe don't run them at all) and have different mechanisms for backing up values.
These have other downsides though. Like, Minimax/Alphabeta require (game-specific) heuristics. MCTS variants that truncate playouts early also need heuristics, or things like Deep Neural Networks, to backpropagate value estimates. Heuristics are generally game-specific and therefore annoying, whereas Deep Neural Networks require huge amounts of compute to get them trained initially.
RE: Planned Ludii AIs - Michael - 02-25-2022
Thank you! Is there a place I can read about the agents currently offered by Ludii? In the drop-down where you select an agent, I can't even read the full name of some of them.
RE: Planned Ludii AIs - DennisSoemers - 02-26-2022
(02-25-2022, 12:08 PM)Michael Wrote: Thank you! Is there a place I can read about the agents currently offered by Ludii? In the drop-down where you select an agent, I can't even read the full name of some of them.
Our User Guide (which you can find under "Documentation" on the downloads page here: https://ludii.games/download.php) has very brief descriptions of most/all agents on pages 17+18, and some longer descriptions of several (but not all) agents in Appendix B.