+- Ludii Forum (https://ludii.games/forums)
+-- Forum: General (https://ludii.games/forums/forumdisplay.php?fid=1)
+--- Forum: Discussion (https://ludii.games/forums/forumdisplay.php?fid=4)
+--- Thread: Ludii vs. Zillions (/showthread.php?tid=16)
Pages:
1
2
Ludii vs. Zillions - RogerCooper - 12-20-2019
I decided to match up Ludii 0.5.0 against the old Zillions program using international chess with 5 seconds per move. I had Ludii use the Alpha-Beta AI. The Alpha-Beta performance was poor, with a search depth of 3, compared to Zillions average search depth of 8. Zillions achieved checkmate in 18 moves.
Ludii has a long way to go be a competent GGP program. Does anyone know why Ludii has a such a low search depth?
RE: Ludii vs. Zillions - cambolbro - 12-20-2019
Zillions of Games is highly optimised for Chesslike games -- in particular Chess itself! -- whereas Ludii is designed to support a wider range of games. Compare Zillions and Ludii at Hex. Or Taikyoku Shogi.
Zillions uses a number of clever optimisations that make it work well for Chess and Chesslike games. For example, the Chess board is 8x8 = 64 cells, which Zillions exploits by using bitboards (i.e. the entire board represented by a single 64-bit long integer). These allow highly efficient encoding of game states and other useful data such as attack tables, positional tables, etc. The Ludii game states also uses bitwise encoding but in a more general format that includes additional information; there are about a dozen different state types to handle the wider range of games.
Ludii's primary purpose is as a game design and analysis tool. AI playing strength is of secondary concern, provided that the AI plays at a sufficient level to allow useful anlayses.
RE: Ludii vs. Zillions - RogerCooper - 12-20-2019
A 3 move depth is not enough for competent play of any game. And there are a lot of chess-like games, including more than half the games in your library.
RE: Ludii vs. Zillions - DennisSoemers - 12-20-2019
Remember that the current Ludii is only a pre-release, the first full release is yet to come. The current alpha-beta implementation is fairly simple, and we've definitely got some improvements planned that are likely to improve its performance -- the most obvious of which is a Transposition Table. We're already computing Zobrist hashes, just didn't get around to using them for a TT yet.
That said, the expectation should always be that we'll be outperformed by single-game-specific implementations or systems that are highly optimised for specific (small sets of) games. The ability to easily and clearly express the most complete range of games possible is more important for Ludii and the Digital Ludeme Project than having a small selection of specific games being highly optimised.
In the future we may consider having support for highly-specialised, optimised "single-game" ludemes that could encapsulate a complete, optimised implementation of a single game in a single ludeme. This is not something that we'll be actively using ourselves though for the "official" game descriptions, because it would break any possibility of comparing different games based on shared, high-level ludemes that they all use -- and it would also simply be infeasible to do this for the wide range of games we want to analyse. So this is not a high priority, and not planned for Ludii version 1.0.0.
RE: Ludii vs. Zillions - RogerCooper - 12-22-2019
(12-20-2019, 04:08 PM)DennisSoemers Wrote: Remember that the current Ludii is only a pre-release, the first full release is yet to come. The current alpha-beta implementation is fairly simple, and we've definitely got some improvements planned that are likely to improve its performance -- the most obvious of which is a Transposition Table. We're already computing Zobrist hashes, just didn't get around to using them for a TT yet.I am glad to hear that there are improvements coming. For right now, I am finding Ludii's playing strength too weak. I matched up Ludii against Zillions in the literal symbol of Ludii, Surukarta and Zillions, but not easily as it did at chess. (I like the Surukarta board as a symbol).
It is hard to match Ludii against Zillions directly by just entering moves because the game board is aligned differently, but I played against both programs and I found Zillions to be a greater challenge.
Taikyoku Shogi is not implemented in Ludii, so I can't compare, but Ludii locked up after 6 moves in regular Shogi which is not a good sign.
RE: Ludii vs. Zillions - DennisSoemers - 12-22-2019
(12-22-2019, 02:23 PM)RogerCooper Wrote: Taikyoku Shogi is not implemented in Ludii, so I can't compare, but Ludii locked up after 6 moves in regular Shogi which is not a good sign.
I guess you meant to say Taikyoku Shogi is not implemented in Zillions of Games (or any other general game system that I'm aware of for that matter
)?What do you mean when you say that Ludii "locked up" after 6 moves in regular Shogi? Do you just mean that one of Ludii's AIs got beaten? Or literally that the application locked up / froze / got stuck?
RE: Ludii vs. Zillions - RogerCooper - 12-23-2019
(12-22-2019, 09:02 PM)DennisSoemers Wrote:(12-22-2019, 02:23 PM)RogerCooper Wrote: Taikyoku Shogi is not implemented in Ludii, so I can't compare, but Ludii locked up after 6 moves in regular Shogi which is not a good sign.
I guess you meant to say Taikyoku Shogi is not implemented in Zillions of Games (or any other general game system that I'm aware of for that matter
What do you mean when you say that Ludii "locked up" after 6 moves in regular Shogi? Do you just mean that one of Ludii's AIs got beaten? Or literally that the application locked up / froze / got stuck?
After 6 moves, Ludii stopped moving at all. I will see if I can replicate the bug tomorrow. Shogi is a tricky game to implement.
RE: Ludii vs. Zillions - DennisSoemers - 12-23-2019
If you do manage to replicate it, especially in such a low number of moves that they'll all remain visible and fit in the Moves tab on the right-hand side, it would be ideal if you could make sure to have that Moves tab opened and take a screenshot of it. Just in case the specific sequence of moves helps to reproduce it.
RE: Ludii vs. Zillions - RogerCooper - 01-24-2020
(12-23-2019, 11:27 AM)DennisSoemers Wrote: If you do manage to replicate it, especially in such a low number of moves that they'll all remain visible and fit in the Moves tab on the right-hand side, it would be ideal if you could make sure to have that Moves tab opened and take a screenshot of it. Just in case the specific sequence of moves helps to reproduce it.My problem with Shogi was that I was not clicking again to promote. The game runs just fine.
However, weak play continues to be a problem with Ludii. Zillions easily beats Ludii 0.6.0 at Shogi & Hex, games Zillions does not play well. Furthermore Ludii offers little challenge to a human player. The fundamental issue is an insufficient lookahead, with Ludii only achieving 4 moves in Chess and similar games.
However the potential for a great GGP program is there. For example, unlike Zillions, Ludii actually runs plays Chess correctly, supporting the 50-move rule. An Ludii has the potential to handle randomness and simultaneous movement.
RE: Ludii vs. Zillions - DennisSoemers - 01-24-2020
For Hex: which AI did you select in Ludii? In Hex, MC-GRAVE is easily the strongest AI we currently have in Ludii, much better than Alpha-Beta (Biased MCTS and probably also UCT will also beat Alpha-Beta in this game).
We're working on a feature to have Ludii automatically switch to the strongest algorithm that we have for any particular game, but that's not available yet in the current pre-release. So for now it will sometimes be necessary to switch manually to figure out which AI is the best in any game.
For Shogi, the current pre-release of Ludii also doesn't have a proper evaluation function yet. We're working on learning these for all games, so when that's done, it would be interesting to compare them again.