After having implemented neural networks for determining whether a Tic-Tac-Toe board has a winner or not, and which player the winner is, it is now time to have a look at which is the winning move.
Looking at this case, you will notice that there are only three possible cases:
- The winner is clear after the third move (of either “X” or “O”),
- the winner is determined after the fourth move (of either “X” or “O”), or
- the winner is determined after the fifth move (only “X” may achieve that state, as there is an odd number of fields available on the board).
Additionally, there is the special case that no winner exists, which we deserved the special identifier of “nine moves” (being the last single digit value). So, in total there are four possible states. There can only be one single state valid at a time (“one-hot case”).
However, this also means that we need to adjust our data preparation:
moveAllDataFrame = pd.DataFrame(list(zip([x.vector[0] for x in validtttRecordsList],
[x.vector[1] for x in validtttRecordsList],
[x.vector[2] for x in validtttRecordsList],
[x.vector[3] for x in validtttRecordsList],
[x.vector[4] for x in validtttRecordsList],
[x.vector[5] for x in validtttRecordsList],
[x.vector[6] for x in validtttRecordsList],
[x.vector[7] for x in validtttRecordsList],
[x.vector[8] for x in validtttRecordsList],
[x.move == 3 for x in validtttRecordsList],
[x.move == 4 for x in validtttRecordsList],
[x.move == 5 for x in validtttRecordsList],
[x.move == 9 for x in validtttRecordsList],
)),
columns =['pos1', 'pos2', 'pos3','pos4','pos5','pos6','pos7','pos8','pos9', 'move3', 'move4', 'move5', 'move9'])
print(moveAllDataFrame.tail())
move_train_dataset = moveAllDataFrame.sample(frac=0.8, random_state=42)
move_test_dataset = moveAllDataFrame.drop(move_train_dataset.index)
print(moveAllDataFrame.shape, move_train_dataset.shape, move_test_dataset.shape)
print(move_train_dataset.describe().transpose())
# split features from labels
move_train_features = move_train_dataset.copy()
move_test_features = move_test_dataset.copy()
moveColumns = ['move3', 'move4', 'move5', 'move9']
move_train_labels = move_train_features[moveColumns].copy()
move_train_features = move_train_features.drop(moveColumns, axis=1)
move_test_labels = move_test_features[moveColumns].copy()
move_test_features = move_test_features.drop(moveColumns, axis=1)and our model we typically used previously before. For example, the last layer must have four output values:
move_model = keras.models.Sequential([
move_normalizer,
layers.Dense(units=64, activation='relu'), #1
layers.Dense(units=64, activation='relu'), #2
layers.Dense(units=128, activation='relu'), #3
layers.Dense(units=4)
])and also the loss function needs to be able to handle multiple outputs:
move_model.compile(loss=keras.losses.CategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.Adam(learning_rate=0.01),
metrics = ["accuracy"])The normalization layer may still remain the old one.
The training shows that after 25 epochs, we get a model with a test accuracy of 1.0:
Model: "sequential_12"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
normalization_3 (Normaliza (None, 9) 19
tion)
dense_35 (Dense) (None, 64) 640
dense_36 (Dense) (None, 64) 4160
dense_37 (Dense) (None, 128) 8320
dense_38 (Dense) (None, 4) 516
=================================================================
Total params: 13655 (53.34 KB)
Trainable params: 13636 (53.27 KB)
Non-trainable params: 19 (80.00 Byte)
_________________________________________________________________
None
Epoch 1/50
567/567 [==============================] - 9s 12ms/step - loss: 0.7764 - accuracy: 0.6625 - lr: 0.0100
Epoch 2/50
567/567 [==============================] - 3s 6ms/step - loss: 0.4873 - accuracy: 0.7995 - lr: 0.0100
Epoch 3/50
567/567 [==============================] - 3s 6ms/step - loss: 0.3017 - accuracy: 0.8853 - lr: 0.0100
Epoch 4/50
567/567 [==============================] - 3s 6ms/step - loss: 0.1976 - accuracy: 0.9307 - lr: 0.0100
Epoch 5/50
567/567 [==============================] - 3s 5ms/step - loss: 0.1389 - accuracy: 0.9522 - lr: 0.0100
Epoch 6/50
567/567 [==============================] - 3s 6ms/step - loss: 0.1256 - accuracy: 0.9593 - lr: 0.0100
Epoch 7/50
567/567 [==============================] - 3s 6ms/step - loss: 0.1020 - accuracy: 0.9658 - lr: 0.0100
Epoch 8/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0795 - accuracy: 0.9731 - lr: 0.0100
Epoch 9/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0938 - accuracy: 0.9691 - lr: 0.0100
Epoch 10/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0746 - accuracy: 0.9755 - lr: 0.0100
Epoch 11/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0585 - accuracy: 0.9821 - lr: 0.0100
Epoch 12/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0744 - accuracy: 0.9761 - lr: 0.0100
Epoch 13/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0464 - accuracy: 0.9845 - lr: 0.0100
Epoch 14/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0677 - accuracy: 0.9792 - lr: 0.0100
Epoch 15/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0563 - accuracy: 0.9829 - lr: 0.0100
Epoch 16/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0089 - accuracy: 0.9978 - lr: 0.0020
Epoch 17/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0046 - accuracy: 0.9992 - lr: 0.0020
Epoch 18/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0035 - accuracy: 0.9995 - lr: 0.0020
Epoch 19/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0027 - accuracy: 0.9997 - lr: 0.0020
Epoch 20/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0020 - accuracy: 0.9998 - lr: 0.0020
Epoch 21/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0016 - accuracy: 0.9999 - lr: 0.0020
Epoch 22/50
567/567 [==============================] - 3s 6ms/step - loss: 0.0012 - accuracy: 0.9999 - lr: 0.0020
Epoch 23/50
567/567 [==============================] - 3s 6ms/step - loss: 9.4242e-04 - accuracy: 0.9999 - lr: 0.0020
Epoch 24/50
567/567 [==============================] - 3s 5ms/step - loss: 0.0089 - accuracy: 0.9975 - lr: 0.0020
Epoch 25/50
567/567 [==============================] - 3s 5ms/step - loss: 8.9431e-04 - accuracy: 1.0000 - lr: 0.0020Like before, we again may try to optimize to find a “minimal” neural network configuration such that we still keep test accuracy at 1.0. After a little trial and error, the best model which I could find with a little of fine-tuning (learning rate down to 0.0016) was with two hidden Dense layers: The first one with 80 units and the second one with 128 units. Making it any smaller did not allow me to find a model with test accuracy at 1.0. This “optimal” model can be found downloaded here.
tic-tac-toe-Move-Model.zip (121.4 KiB, 92 hits)