+- Ludii Forum (https://ludii.games/forums)
+-- Forum: General (https://ludii.games/forums/forumdisplay.php?fid=1)
+--- Forum: Discussion (https://ludii.games/forums/forumdisplay.php?fid=4)
+--- Thread: Choice of AI (/showthread.php?tid=95)
Choice of AI - ddyer0 - 07-29-2020
My experience with Boardspace is that "simple" MCTS almost always crushes "simple" alpha-beta,
and simple MCTS is absolutely trivial to use. I rarely bother to make an alpha-beta evaluator any more.
OTOH, when MCTS doesn't work well, trying to make it work better is a pretty frustrating exercise.
I think that for a generic system like ludii, spending time and effort on better AIs would be a time sink
and a waste of resources. You're never going to compete with engines specialized for the game; and
what you already have is more than adequate to expose the flavor of the games.
RE: Choice of AI - cambolbro - 07-29-2020
Hi,
The ultimate aim is to get the built-in Ludii AIs to play each game at "average human" level. The definition of "average human" level that I'm working with is: approximately 50% win rate against the top 50% of players in each game, once we get some play records in our game database. For some games the AIs will need to be trained up, for some games they may need to be dumbed down.
I agree that there is little point in aiming for expert or even superhuman AIs for the general case. As you say, they will never compete with specialised game engines, and they could even give skewed results for our automated game measurements that do not reflect the average players' experience of each game.
Regards,
Cameron
RE: Choice of AI - mrraow - 07-30-2020
Interesting. My own experience is that AlphaBeta beats MCTS for most movement and capture games, even with very simple heuristics (mobility, material). My default AlphaBeta implementation uses transposition tables, killer moves, best response, and the history heuristic.
RE: Choice of AI - DennisSoemers - 07-30-2020
mrraow's experience matches my own. We ran evaluation games for almost all games where both of these AIs are applicable, and store in the AI metadata files which algorithm yielded the best performance. This is how Ludii decides which algorithm to use when users select just the "Ludii AI". There's a decent number of games where that turned out to be AlphaBeta.
Note that so far we didn't really handcraft heuristics for any games (only exceptions being a few variants of Chess, where we only manually assigned some material values). We just have a few different heuristics implemented to cover a decent range of common cases, ran a bunch of evaluation rounds for all these games to figure out which heuristics to start working with, and then tried to train them a bit more (which sometimes yielded improvements, and sometimes didn't). So the time sink and resources aren't really much of a concern here... at least not if we're talking about human time. Of course it does take computation resources, but whenever we happen to have that available and don't need it for other purposes, we might as well use it for this purpose
