Activity address: CSDN 21-day Learning Challenge
1 Introduction
There are many simple regression problems in life, such as a significant linear relationship between two variables. We can use regression to quantify analysis problems and predict variables.
In this example, we explore the relationship between speed and braking displacement.
import matplotlib.pyplot as plt import numpy as np import pandas as pd cars=pd.read_csv("../data/cars.csv",usecols=["speed","dist"]) cars[:5]#Speed is the speed and dist is the braking distance.
0 | 4 | 2 |
1 | 4 | 10 |
2 | 7 | 4 |
3 | 7 | 22 |
4 | 8 | 16 |
View the data
x=cars["speed"] y=cars["dist"] plt.title("fig1.Scatter plot of vehicle speed and braking distance") plt.scatter(x,y,marker="*",c='red')
From the image, we can see that there is a linear relationship between the two variables. Next, we quantify it by building a regression model.
2 Regression Model
Our goal is to find the parameters for the following functions
Where x is the speed and y is the braking distance.
We need to set up a function to measure the difference between the predicted and the true values so that we know how well the final parameters we get work.
The loss function is:
n is the number of samples,For the true value of a brake,
Is the predicted value.
Naturally, we need L to reach the minimum so that we can get the most accurate regression model. There are two ways to do this.
3 Two Solutions
3.1 Direct Method
The direct method can be used to find the optimal solution directly, but it needs to satisfy that the loss function is convex and the solution is analytical.
[Concavity and Convexity of Functions]
Set a function to f(x) with two points on the image, the point above chord AB is
.
Then the equation of chord AB is:
amongThen
The parametric equation for chord AB is:
Because the function value of point P on the chord where f(x) is predecessor A and B is greater than that on f(x), there are:
Parameter calculation method refers to blog: least square method
Experimental section:
import sympy #Set the equation to y=ax+b #Regression coefficient: Scale factor a and offset value b a,b=sympy.symbols("a b") L=0.5*np.sum((y-a*x-b)**2) #Derivation f1=sympy.diff(L,b) f2=sympy.diff(L,a) print(sympy.diff(L,a)) print(sympy.diff(L,b)) ans=sympy.solve([f1,f2],[b,a]) # Understanding {b: -17.5790948905109, a: 3.93240875912409} alpha=ans[a] beta=ans[b] pre=alpha*x+beta plt.title("Fig2 Fitting results") plt.scatter(x,y,c="red") plt.plot(x,pre,c="green") plt.show()
Draw the resulting line:
Then the function relationship is:
3.2 Iteration Method
There are limitations to optimizing loss functions directly, and it is difficult to find an optimal solution if the loss function is nonconvex. Therefore, an iterative method is proposed, similar to the back propagation algorithm of the previously learned neural network, to minimize the loss value by updating the parameters continuously and slightly. Therefore, the difference between Iteration and Direct is that the parameters can be updated by direct calculation, while the latter needs to be updated by small batch gradient descent.
The loss function is:
Is batch size
Goal Solving:
Parameter update:For learning rate
Experimental section:
import random #Variable Update Function def update_var(pre_alpha,pre_beta,y,x,lr): n=len(x) diff_alpha=np.sum(-(y-pre_beta*x-pre_alpha))/n diff_beta =np.sum(-x*(y-pre_beta*x-pre_alpha))/n new_alpha=pre_alpha-lr*diff_alpha new_beta=pre_beta-lr*diff_beta return (new_alpha,new_beta) #Implement Iterative Process def iterative_func(y,x,raw_alpha,raw_beta,lr,num,sample_num): alpha_list = [] beta_list = [] alpha = raw_alpha beta = raw_beta num_list = list(range(1, len(y)+1)) for i in range(num): alpha_list.append(alpha) beta_list.append(beta) random.shuffle(num_list) index = num_list[:sample_num] alpha, beta = update_var(alpha, beta,y[index], x[index], lr) print("[{}]Now alpha:{},beta:{}".format(i,alpha,beta)) return (alpha_list, beta_list) #Random initialization of alpha and beta raw_alpha=np.random.random()*10 raw_beta =np.random.random()*10 raw_alpha #Setting superparameters: learning rate lr=0.005 iterations num=2000,sample_num=16 lr = 0.005 num = 10000 sample_num = 16 alpha_list, beta_list = iterative_func(y, x, raw_alpha, raw_beta, lr, num,sample_num) print("after {} times iteration:alpha: {}, beta:{}".format(num,alpha_list[-1], beta_list[-1]))
after 10000 times iteration:alpha: -17.791370073935074, beta:4.000650060840287
#Persistence of data from parameter iteration import csv var_data=zip(alpha_list,beta_list) with open("../data/20220801_vardata.csv",'w',newline='')as f: csv_writer=csv.writer(f) csv_writer.writerows(['alpha','beta']) csv_writer.writerows(var_data) plt.subplot(121) plt.plot(alpha_list) plt.title("alpha change process") plt.subplot(122) plt.plot(beta_list) plt.title("beta change process") plt.show()