|
梯度检验用于检验后向传播正常工作
一维数组模型梯度检验
前向传播，计算损失函数：
J = theta * x
后向传播：
dtheta = x
梯度检验：
首先计算gradapprox：
thetaplus = theta + epsilon
thetaminus = theta - epsilon
J_plus = forward_propagation(x, thetaplus)
J_minus = forward_propagation(x, thetaminus)
gradapprox = (J_plus - J_minus)/(2*epsilon)
然后计算grad：
grad = backward_propagation(x, theta)
计算梯度差：
numerator = np.linalg.norm(grad - gradapprox)
denominator = np.linalg.norm(grad) + np.linalg.norm(gradapprox)
difference = numerator/denominator
n维数组模型梯度检验
计算步骤
代码：
thetaplus = np.copy(parameters_values)
thetaplus[i][0] = thetaplus[i][0] + epsilon
J_plus[i], _ = forward_propagation_n(X, Y, vector_to_dictionary(thetaplus))
thetaminus = np.copy(parameters_values)
thetaminus[i][0] = thetaminus[i][0] - epsilon
J_minus[i], _ = forward_propagation_n(X, Y, vector_to_dictionary(thetaminus))
gradapprox[i] = (J_plus[i] - J_minus[i])/(2*epsilon)
| 
转播
