Overview


Co-chairs

Chang-Shing Lee

Chang-Shing Lee

National University of Tainan, Taiwan
leecs@mail.nutn.edu.tw

Yusuke Nojima

Yusuke Nojima

Osaka Prefecture University, Japan
nojima@cs.osakafu-u.ac.jp

Naoyuki Kubota

Naoyuki Kubota

Tokyo Metropolitan University, Japan
kubota@tmu.ac.jp

Giovanni Acampora

Giovanni Acampora

University of Naples Federico II, Italy
giovanni.acampora@unina.it

Marek Reformat

Marek Reformat

University of Alberta, Canada
reformat@ualberta.ca

Yusuke Nojima

Ryosuke Saga

Osaka Prefecture University, Japan
saga@cs.osakafu-u.ac.jp

Scope and Topic

With the success of AlphaGo, there has been a lot of interest among students and professionals to apply machine learning to gaming and in particular to the game of Go. Several conferences have held competitions human players vs. computer programs or computer programs against each other. The goal of this competition includes:

Understand the basic concepts of an FML-based fuzzy inference system.

Use the FML intelligent decision tool to establish the knowledge base and rule base of the fuzzy inference system.

Use the data predicted by Facebook AI Research (FAIR) Open Source Darkforest AI Bot as the training data.

Use the data predicted by Facebook AI Research (FAIR) Open Source ELF OpenGo AI Bot as the desired output of the training data.

Optimize the FML knowledge base and rule base through the methodologies of evolutionary computation and machine learning in order to develop a regression model based on FML-based fuzzy inference system.

Competition Data

GameDataG1

AlphaGo Master series: 60 online games Master in Dec. 2016 and in Jan. Over one week, AlphaGo played 60 online fast time-control games. AlphaGo won this series of games 60 – 0.

Website: https://deepmind.com/research/alphago/match-archive/master/

Each GameData file includes the prediction by Darkforest AI Bot and EFL OpenGo AI Bot for one game.

MoveNo is the move number. MoveNo only lists "odd" numbers (i.e., 1, 3, 5, ...) because each row corresponds to a pair of one Black move and one White move. That is, the row with the MoveNo 1 corresponds to the Black first move (i.e., B1) and the White first move (i.e., W2). The row with the MoveNo 145 corresponds to the Black 145th move (i.e., B145) and the White 146th move (i.e., W146). If the final MoveNo is "odd", "White's information of the last row" will be vacant.

Each row includes eight values (i.e., DBSN, DWSN, DBWR, DWWR, DBTMR, and DWTMR were the outputs from Darkforest. EBWR and EWWR were the outputs from ELF OpenGo).

DBSN: The number of simulations for Black. DWSN: The number of simulations for White. DBWR: The win rate of Black. DWWR: The win rate of White. DBTMR: The top-move rate of Black. DWTMR: The top-move rate of White.

EBWR: The win rate of Black. EWWR: The win rate of White.

The number of rows represents the length of the game and is different among game data files

Game 1 to Game 45 are used as the training data. The inputs are the number of simulations (DBSN, DWSN), the win rates (DBWR, DWWR), and the top-move rates (DBTMR, DWTMR) predicted by Darkforest. The desired outputs are the win rates, (EBWR, EWWR) predicted by ELF OpenGo AI Bot. The participates construct the knowledge base and the rule base of the FML-based inference system. (Download Training Data from Game 1 to Game 45)

Game 46 to Game 60 are used as the test data to examine the generalization ability of the learned FML-based inference system. (Download Testing Data from Game 46 to Game 60)


Submission Instructions

The participants are invited to submit their results via the competition website (http://oase.nutn.edu.tw/fuzz2019-fmlcompetition). Participants are also encouraged to submit the results to the competition held in IEEE CEC 2019 (http://oase.nutn.edu.tw/cec2019-fmlcompetition). We will announce the winner at both conferences.


Submission Deadline

10th May 2019, 23:59 (GMT)


Metrics and Rules

The proposed regression model is evaluated by the mean squared error over all moves in the test game datasets (i.e., Game 46 to Game 60).

MSE
where M is the total number of moves (i.e., the total number of rows) in the test data from Game 46 to Game 60. xi and yi are the Black's win rate predicted by the regression model and the Black's win rate calculated by ELF OpenGo AI Bot (i.e., EBWR), respectively. Note: EWWR = 1.0 – EBWR.

Only the training data (i.e., Game 1 to Game 45) is available for optimizing/learning the regression model and tuning the parameters of optimization/learning algorithms.


Evaluation

The competition will be done before the conference. We will release the Java-based FML tool and call for applications to construct the knowledge base and rule base of FML. They should construct the FML system and write system description document with 2 or 3 pages. The competition will be held on the Internet. The winners can present the FML-based system at FUZZ-IEEE 2019. The participates must submit the following files using a ZIP file after login the competition website.

1) their original knowledge base and rule base described by FML and related files to explain them (30%),

2) their learned knowledge base and rule base described by FML, training data accuracy, testing data accuracy, and related files to explain them (35%), and

3) their slide to explain them (35%).


Additional Details

FUZZ-IEEE 2019 will provide a certificate of participation to all contestants and award a special certificate to the competition winners.

Cash prizes will be provided to the top three contestants, if the number of contestants exceeds 10 teams. The cash prizes will be 500USD, 300USD, and 200USD, respectively.

Participants are expected to apply for travel funds from the CIS-IEEE and attend the FUZZ-IEEE 2019 conference, where they will present their results.


Fuzzy Markup Language (FML ver. 0.1.1) Introduction and Implementation

For more details about FML, please download the FML user guide from

http://kws.nutn.edu.tw/fmldoc/

Available Software Tools

VisualFMLTool: It can be executed on platforms containing the Java Runtime Environment. The Java Software Development Kit, including JRE, compiler and many other tools can be found at here. The VisualFMLTool can download from here and then to extract it. Then it is only needed to click the file VisualFMLTool.bat included in the zip to execute the tool.

JFML: A spanish research group (Jose Manuel Soto Hidalgo, Giovanni Acampora, Jesus Alcala Fernandez, Jose Alonso Moral) has released a library for FML programming that is very simple to use and compliant with IEEE 1855. JFML can download from here. Additional information about the library is here.

Some References associated to JFML

J. M. Soto-Hidalgo, Jose M. Alonso, G. Acampora, and J. Alcala-Fdez, "JFML: A Java library to design fuzzy logic systems according to the IEEE Std 1855-2016," IEEE Access, vol. 6, pp. 54952-54964, 2018.

J. M. Soto-Hidalgo, A. Vitiello, J. M. Alonso, G. Acampora, J. Alcala-Fdez, "Design of fuzzy controllers for embedded systems with JFML," International Journal of Computational Intelligence Systems, vol. 12, no. 1, pp. 204-214, 2019.

Reference

C. S. Lee, M. H. Wang, S. C. Yang, P. H. Hung, S. W. Lin, N. Shuo, N. Kubota, C. H. Chou, P. C. Chou, and C. H. Kao, "FML-based dynamic assessment agent for human-machine cooperative system on game of Go," International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 25, no. 5, pp. 677-705, 2017. arXiv

G. Acampora, "Fuzzy Markup Language: A XML based language for enabling full interoperability in fuzzy systems design,” in G. Acampora, V. Loia, C. S. Lee, and M. H. Wang (editors)," On the Power of Fuzzy Markup Language, Springer-Verlag, Germany, Jan. 2013, pp. 17–33.

IEEE Standards Association, IEEE Standard for Fuzzy Markup Language, Std. 1855-2016, May 2016. [Online] Available: https://ieeexplore.ieee.org/document/7479441.

G. Acampora, B. N. Di Stefano, A. Vitiello, "IEEE 1855TM: The first IEEE standard sponsored by IEEE Computational Intelligence Society," IEEE Computational Intelligence Magazine, vol. 11, no. 4, pp. 4–6, 2016.

J. M. Soto-Hidalgo, J. M. Alonso, and J. Alcalá-Fdez, "Java Fuzzy Markup Language," Jan. 2019. [Oneline] Available: http://www.uco.es/JFML/.

Y. Tian and Y. Zhu, "Better computer Go player with neural network and long-term prediction," 2016 International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico, May 2–4, 2016. https://arxiv.org/pdf/1511.06410.pdf

Y. Tian and L. Zitnick, "Facebook Open Sources ELF OpengGo," May 2018, [Online] Available: https://research.fb.com/facebook-open-sources-elf-opengo/.

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel and D. Hassabis, "Mastering the game of Go with deep neural networks and tree search," Nature, no. 529, pp. 484–489, 2016.

D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. v. d. Driessche, T. Graepel, and D. Hassabis, "Mastering the game of Go without human knowledge," Nature, vol. 550, pp. 35–359, 2017.

Deepmind, "AlphaGo Master series: 60 online games,” Jan. 2019. [Online] Available: https://deepmind.com/research/alphago/match-archive/master/.

C. S. Lee, M. H. Wang, and S. T. Lan, "Adaptive personalized diet linguistic recommendation mechanism based on type-2 fuzzy sets and genetic fuzzy markup language," IEEE Transactions on Fuzzy Systems, vol. 23, no. 5, pp. 1777-1802, Oct. 2015.

C. S. Lee, M. H. Wang, H. Hagas, Z. W. Chen, S. T. Lan, S. E. Kuo, H. C. Kuo, and H. H. Cheng, "A novel genetic fuzzy markup language and its application to healthy diet assessment," International Journal of Uncertainty, Fuzziness, and Knowledge-Based Systems, vol. 20, no. 2, pp. 247-278, Oct. 2012.

C. S. Lee, M. H. Wang, L. C. Chen, Y. Nojima, T. X. Huang, J. Woo, N. Kubota, E. Sato-Shimokawara, T. Yamaguchi, "A GFML-based robot agent for human and machine cooperative learning on game of Go," 2019 IEEE Congress on Evolutionary Computation (IEEE CEC 2019), Wellington, New Zealand, Jun. 10-13, 2019. (Submitted) arXiv