Interns: Afnan Ahmed, Shlok Bohra
Advisor: Professor Richard Martin
Develop an embedded machine learning accelerator by transforming quantized neural networks into FPGA hardware. The project combined two major components: (1) a Python-based framework that automatically generates Verilog for trained CNNs such as WFNet, and (2) the implementation and optimization of a complete FPGA accelerator for a quantized LeNet-5 model capable of classifying MNIST handwritten digits. The work emphasized efficient hardware generation, FPGA-aware memory architecture, parallel computation, and low-latency inference on a Xilinx Zynq platform.
https://docs.google.com/presentation/d/1jTHMpZ7_LhuIwtOTm-0YNasZIDWobEyB1WlW5ffb_H8/edit?usp=sharing
During the first week, the project focused on understanding how trained neural networks can be transformed into efficient FPGA hardware. The overall system architecture was planned, including automatic hardware generation for CNNs and deployment of a quantized LeNet-5 accelerator on a Xilinx Zynq FPGA.
https://docs.google.com/presentation/d/1h-LHQSiRt0h7-OWZR39Ylzq2kRmjuWzrF6HNiYmP0jQ/edit?usp=sharing
The second week focused on building a Python framework capable of reading trained neural network models and automatically generating Verilog. The framework extracted model parameters such as weights, biases, scales, zero-points, multipliers, and shifts, providing the foundation for hardware generation.
Automatic hardware generation expanded to produce the entire CNN inference pipeline. Since model parameters remain constant during inference, weights were embedded directly into generated Verilog rather than fetched from memory, reducing runtime overhead and simplifying the hardware.
https://docs.google.com/presentation/d/10HKofhVna3wJoH-8sskYgye2ey0QkFgMiRhKgb6LpfE/edit?usp=sharing
Each neural network layer required a different hardware organization because of varying tensor dimensions and memory access patterns. Memory layouts, banking strategies, and indexing logic were developed to efficiently supply data to parallel compute units while working within FPGA BRAM limitations.
This week shifted toward implementing a complete LeNet-5 neural network accelerator for MNIST classification. The quantized network was translated into Verilog and verified through simulation, achieving approximately 97% classification accuracy before deployment onto FPGA hardware.
https://docs.google.com/presentation/d/19u5y6EXu2mALtZ_Poim3BarfogOoR3Y86OvObgi03xI/edit?usp=sharing
The accelerator was integrated into the Zynq system as a custom AXI4-Lite IP core. Software running on the ARM processor transferred image data to the accelerator and retrieved predictions, completing the hardware/software communication pipeline.
https://docs.google.com/presentation/d/1feI7rk1HzttKMpGM4hcXINWdL50kiZiDHh-tChvLLTw/edit?usp=sharing
Although simulations performed correctly, deploying the accelerator onto FPGA hardware introduced several implementation challenges. Extensive debugging was performed to verify memory initialization, register communication, signed arithmetic, and data transfers until hardware predictions matched simulation results.
https://docs.google.com/presentation/d/1hXUdz_UEhFIrR6dhxg__uDr7pRXNmugE5TLJMq1cFNo/edit?usp=sharing
One of the largest engineering challenges involved balancing FPGA resources. The original implementation relied heavily on LUTs, leading to routing congestion and timing violations. The architecture was redesigned to utilize dedicated DSP blocks for arithmetic operations while reorganizing BRAM usage to improve throughput and hardware efficiency.
https://docs.google.com/presentation/d/1yValUWiJm4cBnG-dnuRQPa4wGBemiucHBi4tEkOXipU/edit?usp=sharing
The complete embedded system was assembled using SD card storage, custom hardware, and bare-metal software. Thousands of MNIST images were processed automatically while collecting prediction accuracy and inference timing, validating the accelerator under realistic operating conditions.
https://docs.google.com/presentation/d/1ATzjFQvZL-0WKYsF-vdoOWB2i5wPh-a5yr8J-S-fHc8/edit?usp=sharing
During the final week, the complete embedded machine learning system was documented and demonstrated. The project successfully combined automatic RTL generation for quantized CNNs with a deployed FPGA accelerator capable of low-latency inference. The final implementation reduced execution time from approximately 661 billion clock cycles to 4.1 billion clock cycles (roughly 161× fewer clock cycles) while demonstrating how compiler-generated hardware, memory-aware design, and FPGA-specific optimizations can significantly improve embedded machine learning performance.