{"id":1526,"date":"2023-09-26T21:04:12","date_gmt":"2023-09-26T19:04:12","guid":{"rendered":"http:\/\/blog.schmoigl-online.de\/?p=1526"},"modified":"2023-09-26T23:21:12","modified_gmt":"2023-09-26T21:21:12","slug":"tic-tac-toe-and-ai-who-is-the-winner","status":"publish","type":"post","link":"http:\/\/blog.schmoigl-online.de\/?p=1526","title":{"rendered":"Tic-Tac-Toe and AI: Who is the Winner? (Part 3)"},"content":{"rendered":"\n

After having determined if a board has a winner using TensorFlow in the previous blog post<\/a>, let us tackle a very similar question: Who is the winner?<\/p>\n\n\n\n

Again this is a binary decision: Either X (“0”) or O (“1”) may win a board. It is also possible that we run into a tie, and therefore no one is a winner. For the sake of simplicity, let’s then still say “0” as a result. If the board really has a winner, we have already found a high-accuracy neural network to decide that in first place.<\/p>\n\n\n\n

Using the same imports and setups as before, we again prepare our data. This time, we are interested in the information about the winner for our labels:<\/p>\n\n\n\n\n\n\n\n

winnerAllDataFrame = pd.DataFrame(list(zip([x.vector[0] for x in validtttRecordsList], \n                                     [x.vector[1] for x in validtttRecordsList],\n                                     [x.vector[2] for x in validtttRecordsList],\n                                     [x.vector[3] for x in validtttRecordsList],\n                                     [x.vector[4] for x in validtttRecordsList],\n                                     [x.vector[5] for x in validtttRecordsList],\n                                     [x.vector[6] for x in validtttRecordsList],\n                                     [x.vector[7] for x in validtttRecordsList],\n                                     [x.vector[8] for x in validtttRecordsList],\n                                     [x.winner for x in validtttRecordsList])), \n             columns =['pos1', 'pos2', 'pos3','pos4','pos5','pos6','pos7','pos8','pos9', 'winner'])\n\nprint(winnerAllDataFrame.tail())\nwinner_train_dataset = winnerAllDataFrame.sample(frac=0.8, random_state=42)\nwinner_test_dataset = winnerAllDataFrame.drop(winner_train_dataset.index)\n\nprint(winnerAllDataFrame.shape, winner_train_dataset.shape, winner_test_dataset.shape)\nprint(winner_train_dataset.describe().transpose())\n\n# split features from labels\nwinner_train_features = winner_train_dataset.copy()\nwinner_test_features = winner_test_dataset.copy()\n\nwinner_train_labels = winner_train_features.pop('winner')\nwinner_test_labels = winner_test_features.pop('winner')<\/code><\/pre>\n\n\n\n
Starting another normalizer<\/p>\n\n\n\n
winner_normalizer = preprocessing.Normalization()\nwinner_normalizer.adapt(np.array(winner_train_features))\nprint(winner_normalizer.mean.numpy())<\/code><\/pre>\n\n\n\n
We can define another model and train it:<\/p>\n\n\n\n
wwinner_model = keras.models.Sequential([\n    winner_normalizer,\n    layers.Dense(units=64, activation='relu'), #1\n    layers.Dense(units=64, activation='relu'), #2 \n    layers.Dense(units=128, activation='relu'), #3 \n    layers.Dense(units=1)\n])\nprint(winner_model.summary())\nwinner_model.compile(loss=keras.losses.BinaryCrossentropy(from_logits=True),\n              optimizer=keras.optimizers.Adam(learning_rate=0.05), \n              metrics = [\"accuracy\"])\nwinner_history = winner_model.fit(winner_train_features, winner_train_labels, \n          batch_size=512, \n          epochs=50, \n          shuffle=True,\n          callbacks=[\n              tf.keras.callbacks.EarlyStopping(monitor='accuracy', mode=\"max\", restore_best_weights=True, patience=5, verbose=1), \n              Accuracy1Stopping(),\n              tf.keras.callbacks.ReduceLROnPlateau(monitor='loss', factor=0.2, patience=2, min_lr=0.002),\n              tensorboard_callback\n          ],\n          verbose=1)<\/code><\/pre>\n\n\n\n
Note that we use the same configuration for our Sequential model as before. Running the training already terminates after three (!) epochs to reach an accuracy of 1.0:<\/p>\n\n\n\n
Model: \"sequential_6\"\n_________________________________________________________________\n Layer (type)                Output Shape              Param #   \n=================================================================\n normalization_2 (Normaliza  (None, 9)                 19        \n tion)                                                           \n                                                                 \n dense_19 (Dense)            (None, 64)                640       \n                                                                 \n dense_20 (Dense)            (None, 64)                4160      \n                                                                 \n dense_21 (Dense)            (None, 128)               8320      \n                                                                 \n dense_22 (Dense)            (None, 1)                 129       \n                                                                 \n=================================================================\nTotal params: 13268 (51.83 KB)\nTrainable params: 13249 (51.75 KB)\nNon-trainable params: 19 (80.00 Byte)\n_________________________________________________________________\nNone\nEpoch 1\/50\n567\/567 [==============================] - 5s 7ms\/step - loss: 0.1759 - accuracy: 0.9413 - lr: 0.0500\nEpoch 2\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0048 - accuracy: 0.9989 - lr: 0.0500\nEpoch 3\/50\n567\/567 [==============================] - 3s 6ms\/step - loss: 4.6310e-04 - accuracy: 1.0000 - lr: 0.0500<\/code><\/pre>\n\n\n\n
A brief evaluation also shows that the accuracy is effective for our test data:<\/p>\n\n\n\n
evaluationResult = winner_model.evaluate(winner_test_features, winner_test_labels, batch_size=256, verbose=1)\nprint(evaluationResult)<\/code><\/pre>\n\n\n\n
284\/284 [==============================] - 3s 10ms\/step - loss: 2.9924e-04 - accuracy: 1.0000\n[0.0002992423251271248, 1.0]<\/code><\/pre>\n\n\n\n
This means that our model is over-equipped. In fact, running a model with a single Dense layer of 40 units also still does the trick after 11 epochs:<\/p>\n\n\n\n
winner_model = keras.models.Sequential([\n    winner_normalizer,\n    layers.Dense(units=40, activation='relu'), #1\n    layers.Dense(units=1)\n])\n# ...<\/code><\/pre>\n\n\n\n
Epoch 1\/50\n567\/567 [==============================] - 7s 9ms\/step - loss: 0.2422 - accuracy: 0.9175 - lr: 0.0500\nEpoch 2\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.1134 - accuracy: 0.9610 - lr: 0.0500\nEpoch 3\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0890 - accuracy: 0.9680 - lr: 0.0500\nEpoch 4\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0718 - accuracy: 0.9730 - lr: 0.0500\nEpoch 5\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0605 - accuracy: 0.9766 - lr: 0.0500\nEpoch 6\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0412 - accuracy: 0.9847 - lr: 0.0500\nEpoch 7\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0287 - accuracy: 0.9901 - lr: 0.0500\nEpoch 8\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0181 - accuracy: 0.9947 - lr: 0.0500\nEpoch 9\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0063 - accuracy: 0.9997 - lr: 0.0500\nEpoch 10\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0113 - accuracy: 0.9970 - lr: 0.0500\nEpoch 11\/50\n567\/567 [==============================] - 3s 5ms\/step - loss: 0.0027 - accuracy: 1.0000 - lr: 0.0500<\/code><\/pre>\n\n\n\n
This winner model can be downloaded here.<\/p>\n\n\n
\"\"  tic-tac-toe-Winner-Model.zip<\/a><\/strong> (7.6 KiB, 842 hits)<\/p>\n\n\n\n
Note that reducing the number of units further makes it harder to eventually achieve an accuracy of 1.0. Already with 32 units only and several attempts made, I was not able to a model trained within 50 epochs.<\/p>\n\n\n\n
Obviously, answering the question who is the winner is much easier to answer always correctly for a neural network the task of answering whether there is a winner at all. <\/p>\n","protected":false},"excerpt":{"rendered":"
After having determined if a board has a winner using TensorFlow in the previous blog post, let us tackle a very similar question: Who is the winner? Again this is a binary decision: Either X (“0”) or O (“1”) may win a board. It is also possible that we run into a tie, and therefore …<\/p>\n
Continue reading ‘Tic-Tac-Toe and AI: Who is the Winner? (Part 3)’ »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[38],"tags":[48,50],"class_list":["post-1526","post","type-post","status-publish","format-standard","hentry","category-machine-learning","tag-tensorflow","tag-tic-tac-toe"],"_links":{"self":[{"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/posts\/1526","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1526"}],"version-history":[{"count":7,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/posts\/1526\/revisions"}],"predecessor-version":[{"id":1582,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=\/wp\/v2\/posts\/1526\/revisions\/1582"}],"wp:attachment":[{"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1526"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1526"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.schmoigl-online.de\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1526"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}