Skip to content

Getting Started with MicroKeras

Open In Colab

MicroKeras is a minimal implementation of the Sequential Class from the Keras deep-learning library, built from scratch using Python and NumPy. It provides a simple and intuitive API for building, training, and evaluating neural networks inspired by the Keras API.

Installation

To install MicroKeras, use pip:

pip install microkeras

Quick Start: MNIST Classification Example

Let's walk through a complete example using MicroKeras to classify handwritten digits from the MNIST dataset. This example will demonstrate how to load data, create a model, train it, make predictions, and save/load the model.

Importing Dependencies

First, let's import the necessary modules:

import numpy as np
from microkeras.models import Sequential
from microkeras.layers import Dense
from microkeras.optimizers import SGD
from microkeras.datasets import mnist

Loading and Preprocessing Data

Next, we'll load the MNIST dataset and preprocess it:

# Load and preprocess MNIST data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Reshape and normalize the input data
X_train = X_train.reshape(X_train.shape[0], -1)
X_test = X_test.reshape(X_test.shape[0], -1)

# One-hot encode the labels
y_train = np.eye(10)[y_train]
y_test = np.eye(10)[y_test]

print("Data shapes:")
print(f"X_train: {X_train.shape}, y_train: {y_train.shape}")
print(f"X_test: {X_test.shape}, y_test: {y_test.shape}")

Output:

Data shapes:
X_train: (56000, 784), y_train: (56000, 10)
X_test: (14000, 784), y_test: (14000, 10)

Creating the Model

Now, let's create our Sequential model:

model = Sequential([
    Dense(200, activation='sigmoid', input_shape=(784,)),
    Dense(200, activation='sigmoid'),
    Dense(10, activation='softmax')
])

This model consists of two hidden layers with sigmoid activation and an output layer with softmax activation, suitable for multi-class classification.

Compiling the Model

We'll compile the model using Stochastic Gradient Descent (SGD) as the optimizer and Categorical Cross-Entropy as the loss function:

optimizer = SGD(learning_rate=0.1)
model.compile(optimizer=optimizer,
              loss='categorical_crossentropy',
              metrics=['accuracy'])

Training the Model

Let's train the model for 10 epochs with a batch size of 32:

history = model.fit(X_train,
                    y_train,
                    batch_size=16,
                    epochs=10)

Output:

Epoch 1/10
Batch 3500/3500 - Loss: 4.9410, Acc: 0.9238: : 3712it [00:37, 99.54it/s] 
Epoch 2/10
Batch 3500/3500 - Loss: 1.9095, Acc: 0.9746: : 3712it [00:20, 180.73it/s]
Epoch 3/10
Batch 3500/3500 - Loss: 1.5755, Acc: 0.9844: : 3712it [00:21, 168.76it/s]
Epoch 4/10
Batch 3500/3500 - Loss: 1.2145, Acc: 0.9844: : 3712it [00:22, 162.42it/s]
Epoch 5/10
Batch 3500/3500 - Loss: 0.9108, Acc: 0.9902: : 3712it [00:23, 157.83it/s]
Epoch 6/10
Batch 3500/3500 - Loss: 0.7050, Acc: 0.9941: : 3712it [00:21, 175.94it/s]
Epoch 7/10
Batch 3500/3500 - Loss: 0.4757, Acc: 0.9980: : 3712it [00:20, 183.64it/s]
Epoch 8/10
Batch 3500/3500 - Loss: 0.4843, Acc: 1.0000: : 3712it [00:22, 164.84it/s]
Epoch 9/10
Batch 3500/3500 - Loss: 0.4095, Acc: 0.9980: : 3712it [00:22, 164.12it/s]
Epoch 10/10
Batch 3500/3500 - Loss: 0.3842, Acc: 0.9961: : 3712it [00:21, 176.15it/s]

Plotting Training History

We can plot the training history using Matplotlib:

# Plot training history
import matplotlib.pyplot as plt

plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history['accuracy'], label='Training Accuracy')
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(history['loss'], label='Training Loss')
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend()

plt.tight_layout()
plt.show()

Output:

Model Accuracy and Loss

Evaluating the Model

After training, we can evaluate the model on the test set:

test_loss, test_accuracy = model.evaluate(X_test, y_test)
print(f"Test accuracy: {test_accuracy:.4f}")

Output:

Test accuracy: 0.9671

Making Predictions

Let's make predictions for the first 5 test samples:

predictions = model.predict(X_test[:5])
print("Predictions for the first 5 test samples:")
print(np.argmax(predictions, axis=1))
print("Actual labels:")
print(np.argmax(y_test[:5], axis=1))

# Visualize the predictions
plt.figure(figsize=(12, 4))
for i in range(5):
    plt.subplot(1, 5, i+1)
    plt.imshow(X_test[i].reshape(28, 28), cmap='gray')
    plt.title(f"Pred: {np.argmax(predictions[i])}\nTrue: {np.argmax(y_test[i])}")
    plt.axis('off')
plt.tight_layout()
plt.show()

Output:

Predictions for the first 5 test samples:
[8 4 8 7 7]
Actual labels:
[8 4 8 7 7]

Predictions

Saving and Loading the Model

MicroKeras allows you to save and load models:

# Save the model
model.save('mnist_model.json')

# Load the model
loaded_model = Sequential.load('mnist_model.json')

# Compile the loaded model
loaded_model.compile(optimizer=optimizer,
                     loss='categorical_crossentropy',
                     metrics=['accuracy'])

# Evaluate the loaded model
loaded_test_loss, loaded_test_accuracy = loaded_model.evaluate(X_test, y_test)
print(f"Loaded model test accuracy: {loaded_test_accuracy:.4f}")

Output:

Loaded model test accuracy: 0.9671

Viewing Training History

Finally, let's print out the training history:

print("\nTraining History:")
print("Epoch\tAccuracy\tLoss")
for epoch, (accuracy, loss) in enumerate(zip(history['accuracy'], history['loss']), 1):
    print(f"{epoch}\t{accuracy:.4f}\t{loss:.4f}")

Output:

Training History:
Epoch   Accuracy    Loss
1   0.9235  14096.9166
2   0.9467  9728.8764
3   0.9592  7629.1049
4   0.9683  6160.9156
5   0.9730  5208.7316
6   0.9761  4604.9854
7   0.9803  3969.1862
8   0.9828  3443.3674
9   0.9831  3310.2258
10  0.9851  3007.7300

Features

MicroKeras offers the following features:

  • Sequential model API
  • Dense (fully connected) layers
  • Various activation functions (ReLU, Sigmoid, Softmax, Linear)
  • Loss functions (Mean Squared Error, Categorical Cross-Entropy)
  • Optimizers (Stochastic Gradient Descent)
  • Dataset loaders (MNIST, California Housing)
  • Model saving and loading

Additional Examples

California Housing Regression

For an example of using MicroKeras for regression tasks, see our California Housing Regression Example.