The creation of OPE Prop is a very funny, but interesting story. While watching an online lecture on multivariate derivatives in a Udemy course on Calculus from Mike X Cohen, one of my favorite instructors and courses, I started thinking about how this could be used. And the first thing that came to my head was combining the partial derivatives into a vector, computing the vector's size, and finding where it is equal to zero - the extrema's of a n-dimensional function! Finding the extrema's of a function is the main goal of gradient descent or any other backpropagation algorithm in AI, so I started testing.

The first idea that came after the initiative was just trying to find a formula, that simply finds where the size of the derivative vector is equal to zero. For some unexplained reason, I ended that idea and tried to come up with something else.

After forgetting about the previous plan, I started coming up with something new. My main concern was that gradient descent updates its weights and biases with each new input, which could probably lead to biased results, targeting the last data points in the training set (as gradient descent goes through every data point and re-calculates new weights). After programming and designing the algorithm for a few days, the final concept started arising. The first tests were done on a output layer with one weight and one bias, and turned out to be successful. Then, I used the same process to derive the formula for multiple weights, and it also turned out to be successful with initial tests leading to great results! Now, it's time to delve deeper into the algorithm!

OPE Prop formulas on this website are licensed under the CC BY-SA 4.0 License. More details here