Latest Part 6: Building The First Neural Network 2025

Posted on Dec 4

• Originally published at palashkantikundu.in

My program was running successfully and was using GPU to do the heavy lifting and bring the complexity down from $O(n^3)$ to a highly parallelized, faster execution. Having successfully integrated the CUDA code into the logistic regression module, I was happy. I know my GPU kernel programs are not the best in optimization standards, but I really didn't care too much at that point. I had sufficient performance boost to run 20000 training loops under 10 seconds, which was taking almost 4 hours a few days ago.

I'd rather deep dive into learning the next step of machine learning than focus on making my code super efficient.

But there is no harm in doing some kind of baseline check while my machine does the hard work and I take rest. It was so satisfying to see GPU getting used by my own program, I could not resist the urge, I bumped up the number of epochs to 10 million. My GPU frequently went 100% usage. Wow!!! A cherishable moment.

10 million iterations took almost 45 minutes in my machine, still less than the time it took for CPU to run only 5000 iterations. Accuracy also hit 93.09% this time.

After experimenting with other data and fine-tuning a few hyperparameters, I took my next step in the journey - building a neural network.

At that point my inventory included quite a few modules:

The main objective of machine learning is to minimize the loss, calculated by the difference of actual test values and predicted values. To do this we heavily rely on mathematics.

For a refresher, let's revisit how Linear Regression works. The objective of Linear Regression is to find a fitting straight line through the data. The equation for a line is represented as follows:

In linear regression, we have to find the $m$ and the $c$ from $y$ and $x$ where $x$ is input data and $y$ is the output for that $x$.

Source: Dev.to