![]() ![]() Here is the python code: import pandas as pd Note that as we passed the bias into the first column, the first theta value denotes the bias weight.īelow you can find my implementation of gradient descent for linear regression problem.Īt first, you calculate gradient like X.T * (X * w - y) / N and update your current theta with this gradient simultaneously. Which is actually quite close to the equation that was calculated by excel (y = x + 30). If you run my example, the theta returned will look like this: Iteration 99997 | Cost: 47883.706462 For that matter you should always track your cost every iteration, maybe even plot it. The next thing you need to take care about is to track the convergence and adjust the learning rate. In this case, this is the average of the sum over the gradients, thus the division by m. As you do a complete batch pass over your data X, you need to reduce the m-losses of every example to a single weight update. You need to take care about the intuition of the regression using gradient descent. Theta = gradientDescent(x, y, theta, alpha, m, numIterations)Īt first I create a small random dataset which should look like this:Īs you can see I also added the generated regression line and formula that was calculated by excel. ![]() # gen 100 points with a bias of 25 and 10 variance as a bit of noise Y = (i + bias) + random.uniform(0, 1) * variance Print("Iteration %d | Cost: %f" % (i, cost)) # But to be consistent with the gradient, I include it) # avg cost per example (the 2 in 2*m doesn't really matter here. # m denotes the number of examples here, not the number of featuresĭef gradientDescent(x, y, theta, alpha, m, numIterations): Let's have a look at my variation of your code: import numpy as np Here m denotes the number of examples in your training set, not the number of features. In your case, I guess you have confused m with n. Update the parameters theta = theta - alpha * gradient.Calculate the loss = h - y and maybe the squared cost (loss^2)/2m. ![]() In the end this regression boils down to four operations: I think your code is a bit too complicated and it needs more structure, because otherwise you'll be lost in all equations and operations. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |