Neural Network Framework from Scratch

A NumPy-based implementation of a neural network framework developed from scratch for classification tasks.

This project was created as part of a university Machine Learning course to explore the internal mechanics of neural networks without relying on deep learning frameworks.

Features

• Fully connected (Dense) layers
• Custom RBFDense layer based on Gaussian RBF kernels
• Forward and backward propagation with analytical gradients
• Multiple activation functions:
  • Identity
  • ReLU
  • Tanh
• Numerically stable Softmax Cross-Entropy loss
• Adam optimizer
• Residual (skip) connections
• Sequential model API
• Mini-batch gradient descent training
• Automatic parameter counting
• Model evaluation on train and test sets

Experiments

The framework supports three model families:

• Dense neural networks
• RBF neural networks
• Hybrid Dense + RBF architectures

For each architecture the following experiments were conducted:

• varying network depth;
• residual vs. non-residual architectures;
• comparison of classification accuracy;
• comparison of parameter count;
• comparison of the number of learnable layers;
• visualization of learning curves.

Dataset

The experiments were performed on the EEG Eye State dataset.

Preprocessing included:

• train/test split;
• missing value imputation using training-set medians;
• feature standardization;
• one-hot encoding of target labels.

Results

The framework compares different neural network architectures and visualizes:

• training and test learning curves;
• accuracy versus number of parameters;
• accuracy versus number of learnable layers;
• comparison between Dense, RBF, and mixed architectures.

Technologies

• Python
• NumPy
• Pandas
• Matplotlib
• Scikit-learn

Learning Outcomes

Through this project I implemented from scratch:

• backpropagation;
• analytical gradient computation;
• adaptive optimization (Adam);
• residual connections;
• RBF-based learnable layers;
• neural network training and evaluation pipelines.

The project demonstrates the implementation of core deep learning components without using high-level frameworks such as PyTorch or TensorFlow.