深圳集团网站建设企业管理客户的免费软件

深圳集团网站建设企业,管理客户的免费软件,wordpress大气全屏主题,织梦dede模板自带的网站地图优化指南文章目录前言二分类问题多元线性函数σ\sigmaσ 函数输出函数似然函数极大似然估计梯度下降法函数准备求偏导损失函数梯度更新python 实战LogisticRegression训练及结果运行结果总结当你迷茫的时候#xff0c;请回头看看 目录大纲#xff0c;也许有你意想不到的收获 前言 前…文章目录前言二分类问题多元线性函数σ \sigmaσ函数输出函数似然函数极大似然估计梯度下降法函数准备求偏导损失函数梯度更新python 实战LogisticRegression训练及结果运行结果总结当你迷茫的时候请回头看看 目录大纲也许有你意想不到的收获前言前面讲到几篇都是线性回归问题的求解求解方式有最小二乘法正规方程梯度下降法。现在讲讲另一种回归问题就是逻辑回归也就是分类问题分类问题也包括二分类与多分类问题。二分类输出是(1)或否(0)例如逻辑门与门、或门、非门、异或门等等。多分类输出的是多个值对应着每个分类的概率是小于或等于 1 的值。最经典的例子莫过于手写数字的识别给出一张手写的数字图片最后输出是各个数字1~9的概率比如我手写了一个 7像 7 又像 1那么多分类输出可能为7 有 0.9 概率1 有 0.1 的概率其他数字概率接近于 0。二分类问题先简单点讨论二分类问题多元线性函数u w 1 x 1 w 2 x 2 . . . b uw_1x_1w_2x_2...buw1​x1​w2​x2​...bσ \sigmaσ函数σ ( x ) 1 1 e − x e x 1 e x \sigma(x)\frac{1}{1e^{-x}}\frac{e^x}{1e^x}\\[10pt]σ(x)1e−x1​1exex​可以看出σ ( x ) ∈ ( 0 , 1 ) \sigma(x) \in (0,1)σ(x)∈(0,1)如果令P σ ( x ) P\sigma(x)Pσ(x)那么有x ln ⁡ P 1 − P x\ln{\frac{P}{1-P}}xln1−PP​这个也比较好推导∵ σ ( x ) 1 1 e − x P ∴ 1 e − x 1 P ∴ e − x 1 P − 1 1 − P P ∴ x − ln ⁡ 1 − P P ln ⁡ P 1 − P \because\sigma(x)\frac{1}{1e^{-x}}P\\[10pt] \therefore 1e^{-x}\frac{1}{P}\\[10pt] \therefore e^{-x}\frac{1}{P}-1\frac{1-P}{P}\\[10pt] \therefore x-\ln{\frac{1-P}{P}}\ln{\frac{P}{1-P}}∵σ(x)1e−x1​P∴1e−xP1​∴e−xP1​−1P1−P​∴x−lnP1−P​ln1−PP​σ ( x ) \sigma(x)σ(x)导数:σ ′ ( x ) − 1 ( 1 e − x ) 2 ⋅ ( 1 e − x ) ′ e − x ( 1 e − x ) 2 1 1 e − x ⋅ e − x 1 e − x 1 1 e − x ⋅ ( 1 − 1 1 e − x ) ∴ σ ′ ( x ) σ ( x ) ⋅ ( 1 − σ ( x ) ) \sigma(x)-\frac{1}{(1e^{-x})^2} \cdot (1e^{-x})\frac{e^{-x}}{(1e^{-x})^2}\\[10pt] \frac{1}{1e^{-x}} \cdot \frac{e^{-x}}{1e^{-x}} \frac{1}{1e^{-x}} \cdot (1- \frac{1}{1e^{-x}})\\[10pt] \therefore \sigma(x) \sigma(x)\cdot (1-\sigma(x))\\[10pt]σ′(x)−(1e−x)21​⋅(1e−x)′(1e−x)2e−x​1e−x1​⋅1e−xe−x​1e−x1​⋅(1−1e−x1​)∴σ′(x)σ(x)⋅(1−σ(x))输出函数将线性函数与σ \sigmaσ函数结合在一起得到一个函数σ ( u ) \sigma(u)σ(u)这里把 b 也看作是w n w_nwn​相应地x n x_nxn​也就为 1σ ( u ) 1 1 e − u 1 1 e − ( w 1 x 1 w 2 x 2 . . . w n x n ) \sigma(u)\frac{1}{1e^{-u}}\frac{1}{1e^{-(w_1x_1w_2x_2...w_nx_n)}}σ(u)1e−u1​1e−(w1​x1​w2​x2​...wn​xn​)1​这里我令输出函数z σ ( u ) z\sigma(u)zσ(u)即z σ ( u ) 1 1 e − u 1 1 e − ( w 1 x 1 w 2 x 2 . . . w n x n ) z\sigma(u)\frac{1}{1e^{-u}}\frac{1}{1e^{-(w_1x_1w_2x_2...w_nx_n)}}\\[10pt]zσ(u)1e−u1​1e−(w1​x1​w2​x2​...wn​xn​)1​根据σ \sigmaσ函数导数性质有z ′ σ ′ ( u ) σ ( u ) ⋅ ( 1 − σ ( u ) ) z ( 1 − z ) z\sigma(u)\sigma(u)\cdot (1-\sigma(u))z(1-z)\\[10pt]z′σ′(u)σ(u)⋅(1−σ(u))z(1−z)上面我们提到过σ ( u ) ∈ ( 0 , 1 ) \sigma(u) \in (0,1)σ(u)∈(0,1)令P σ ( u ) P\sigma(u)Pσ(u)那么有u ln ⁡ P 1 − P u\ln{\frac{P}{1-P}}uln1−PP​P 可以看作是某个类别的概率对于二分类来说概率要么是 0要么是 1那么从概率的角度上看∵ P σ ( u ) ∴ u ln ⁡ P 1 − P ln ⁡ P ( y 1 ∣ x ) P ( y 0 ∣ x ) \because P\sigma(u)\\[10pt] \therefore u\ln{\frac{P}{1-P}}\ln{\frac{P(y1|x)}{P(y0|x)}}∵Pσ(u)∴uln1−PP​lnP(y0∣x)P(y1∣x)​y 为真实标签u 的含义为事件为1的概率P ( y 1 ∣ x ) P(y1|x)P(y1∣x)与事件为0的概率P ( y 0 ∣ x ) P(y0|x)P(y0∣x)比值再取对数。似然函数似然(likelihood)一听这个名字就知道是以前的老前辈翻译的带有文言色彩似然相似的样子假设总共有 m 个样本第 i 个样本二分类的似然函数如下其中z i z_izi​为预测输出y i y_iyi​为实际输出L ( w ) ∏ i 1 m z i y i ( 1 − z i ) ( 1 − y i ) % 似然函数 L(w)\prod\limits_{i1}^{m} z_i^{y_i}(1-z_i)^{(1-y_i)}\\[10pt]L(w)i1∏m​ziyi​​(1−zi​)(1−yi​)要注意的是这里y i y_iyi​与1 − y i 1-y_i1−yi​其中有一个为 0z i z_izi​要么为 0要么为 1二分类令w [ w 1 w 2 … w n ] , x i [ x i 1 x i 2 … x i n ] z i σ ( w T x i ) % w 参数 w\begin{bmatrix} w_1\\w_2\\\dots\\ w_n \end{bmatrix}, % x 输入 x_i\begin{bmatrix} x_{i1}\\x_{i2}\\\dots\\ x_{in} \end{bmatrix}\\[10pt] z_i\sigma(w^Tx_i)\\[10pt]w​w1​w2​…wn​​​,xi​​xi1​xi2​…xin​​​zi​σ(wTxi​)为了方便可以对似然函数取对数就由乘法转为了加法称之为对数似然函数ℓ ( w ) ln ⁡ L ( w ) ln ⁡ ( ∏ i 1 m z i y i ( 1 − z i ) ( 1 − y i ) ) ∑ i 1 m [ y i ln ⁡ z i ( 1 − y i ) ln ⁡ ( 1 − z i ) ] \ell(w)\ln{L(w)}\ln{(\prod\limits_{i1}^{m} z_i^{y_i}(1-z_i)^{(1-y_i)})}\\[10pt] \sum\limits_{i1}^{m}[{y_i\ln{z_i}}{(1-y_i)\ln{(1-z_i)}}]ℓ(w)lnL(w)ln(i1∏m​ziyi​​(1−zi​)(1−yi​))i1∑m​[yi​lnzi​(1−yi​)ln(1−zi​)]因为输出z i ∈ ( 0 , 1 ) z_i \in(0,1)zi​∈(0,1)所以ln ⁡ z i \ln{z_i}lnzi​或者ln ⁡ ( 1 − z i ) \ln(1-z_i)ln(1−zi​)都是负数因此负对数似然为正数:L N L L ( w ) − ∑ i 1 m [ y i ln ⁡ z i ( 1 − y i ) ln ⁡ ( 1 − z i ) ] \mathcal{L}_{NLL}(w) - \sum\limits_{i1}^{m}[{y_i\ln{z_i}}{(1-y_i)\ln{(1-z_i)}}]LNLL​(w)−i1∑m​[yi​lnzi​(1−yi​)ln(1−zi​)]极大似然估计还记得我们前面的最小二乘法吗目标是最小残差平方和(RSS)通过 RSS 来判断函数拟合的优劣那么对于回归回题目标则是求极大似然估计似然函数或对数似然函数越大等效于负对数似然越小,函数拟合的就越好。求极大似然估计等效于要求负对数似然极小。这就和最小化残差平方和有点相似了都是求最小就可以采用梯度下降法进行求解。梯度下降法函数准备令 i 为样本号eg.z i z_{i}zi​为第 i 个样本预测概率y i y_{i}yi​为真实概率有u i w 1 x i 1 w 2 x i 2 . . . b [ x i 1 x i 2 … 1 ] [ w 1 w 2 … b ] z i σ ( u i ) 1 1 e − u i u_{i}w_{1}x_{i1}w_{2}x_{i2}...b\\[10pt] \begin{bmatrix} x_{i1} x_{i2}\dots 1 \end{bmatrix} \begin{bmatrix} w_{1}\\ w_{2}\\\dots\\ b \end{bmatrix} \\[10pt] z_{i}\sigma(u_{i})\frac{1}{1e^{-u_{i}}}\\[10pt]ui​w1​xi1​w2​xi2​...b[xi1​​xi2​​…​1​]​w1​w2​…b​​zi​σ(ui​)1e−ui​1​可以把 b 作为w n w_{n}wn​那么x n 1 x_{n}1xn​1它们的偏导为∂ u i ∂ w k x i k ∂ z i ∂ u i z i ( 1 − z i ) \frac{\partial u_{i}}{\partial w_{k}} x_{ik}\\[10pt] \frac{\partial z_{i}}{\partial u_{i}} z_{i}(1-z_{i})\\[10pt]∂wk​∂ui​​xik​∂ui​∂zi​​zi​(1−zi​)负对数似然函数为L N L L ( w ) − ∑ i 1 m [ y i ln ⁡ z i ( 1 − y i ) ln ⁡ ( 1 − z i ) ] \mathcal{L}_{NLL}(w) - \sum\limits_{i1}^{m}[{y_i\ln{z_i}}{(1-y_i)\ln{(1-z_i)}}]LNLL​(w)−i1∑m​[yi​lnzi​(1−yi​)ln(1−zi​)]令E i − [ y i ln ⁡ z i ( 1 − y i ) ln ⁡ ( 1 − z i ) ] ∴ L N L L ( w ) ∑ i 1 m E i E_{i}-[{y_i\ln{z_i}}{(1-y_i)\ln{(1-z_i)}}]\\[10pt] %% \therefore \mathcal{L}_{NLL}(w) \sum\limits_{i1}^{m}E_{i}\\[10pt]Ei​−[yi​lnzi​(1−yi​)ln(1−zi​)]∴LNLL​(w)i1∑m​Ei​求偏导∂ E i ∂ z i − ( y i z i − 1 − y i 1 − z i ) z i − y i z i ( 1 − z i ) %% \frac{\partial E_{i}}{\partial z_{i}} -(\frac{y_{i}}{z_{i}}-\frac{1-y_{i}}{1-z_{i}})\\[10pt] \frac{z_{i}-y_{i}}{z_{i}(1-z_{i})} \\[10pt]∂zi​∂Ei​​−(zi​yi​​−1−zi​1−yi​​)zi​(1−zi​)zi​−yi​​所以有∂ E i ∂ w k ∂ E i ∂ z i ⋅ ∂ z i ∂ u i ⋅ ∂ u i ∂ w k z i − y i z i ( 1 − z i ) ⋅ z i ( 1 − z i ) ⋅ x i k ( z i − y i ) ⋅ x i k ∴ ∂ L N L L ∂ w k ∑ i 1 m ∂ E i ∂ w k ∑ i 1 m ( z i − y i ) ⋅ x i k \frac{\partial E_{i}}{\partial w_{k}} \frac{\partial E_{i}}{\partial z_{i}} \cdot \frac{\partial z_{i}}{\partial u_{i}}\cdot \frac{\partial u_{i}}{\partial w_{k}}\\[10pt] \frac{z_{i}-y_{i}}{z_{i}(1-z_{i})} \cdot z_{i}(1-z_{i}) \cdot x_{ik}\\[10pt] (z_{i}-y_{i}) \cdot x_{ik}\\[10pt] %% \therefore\frac{\partial \mathcal{L}_{NLL}}{\partial w_{k}} \sum\limits_{i1}^{m}\frac{\partial E_{i}}{\partial w_{k}} \sum\limits_{i1}^{m}(z_{i}-y_{i}) \cdot x_{ik}\\[10pt]∂wk​∂Ei​​∂zi​∂Ei​​⋅∂ui​∂zi​​⋅∂wk​∂ui​​zi​(1−zi​)zi​−yi​​⋅zi​(1−zi​)⋅xik​(zi​−yi​)⋅xik​∴∂wk​∂LNLL​​i1∑m​∂wk​∂Ei​​i1∑m​(zi​−yi​)⋅xik​损失函数项目上多用均值作为损失函数即E 1 m L N L L ( w ) 1 m ∑ i 1 m E i − 1 m ∑ i 1 m [ y i ln ⁡ z i ( 1 − y i ) ln ⁡ ( 1 − z i ) ] E\frac{1}{m}\mathcal{L}_{NLL}(w)\frac{1}{m}\sum\limits_{i1}^{m}E_{i}\\[10pt] -\frac{1}{m}\sum\limits_{i1}^{m}[{y_i\ln{z_i}}{(1-y_i)\ln{(1-z_i)}}]\\[10pt]Em1​LNLL​(w)m1​i1∑m​Ei​−m1​i1∑m​[yi​lnzi​(1−yi​)ln(1−zi​)]所以有∂ E ∂ w k 1 m ∂ L N L L ∂ w k 1 m ∑ i 1 m ( z i − y i ) ⋅ x i k \frac{\partial E}{\partial w_{k}} \frac{1}{m}\frac{\partial \mathcal{L}_{NLL}}{\partial w_{k}} \frac{1}{m}\sum\limits_{i1}^{m}(z_{i}-y_{i}) \cdot x_{ik}\\[10pt]∂wk​∂E​m1​∂wk​∂LNLL​​m1​i1∑m​(zi​−yi​)⋅xik​梯度更新上面我们求出了偏导是为了求梯度准备的。再来翻译一下梯度梯子的斜度也就是斜率对于多维度的梯度就是每个维度方向上的斜率。那么 E 的梯度为∂ E ∂ w ⃗ [ ∂ E ∂ w 1 ∂ E ∂ w 2 … ∂ E ∂ w n ] [ 1 m ∑ i 1 m ( z i − y i ) ⋅ x i 1 1 m ∑ i 1 m ( z i − y i ) ⋅ x i 2 … 1 m ∑ i 1 m ( z i − y i ) ⋅ x i n ] \frac{\partial E}{\partial \vec{w}}\begin{bmatrix} \frac{\partial E}{\partial w_{1}}\\[10pt] \frac{\partial E}{\partial w_{2}}\\[10pt] \dots\\[10pt] \frac{\partial E}{\partial w_{n}}\\[10pt] \end{bmatrix} \begin{bmatrix} \frac{1}{m}\sum\limits_{i1}^{m}(z_{i}-y_{i}) \cdot x_{i1}\\[10pt] \frac{1}{m}\sum\limits_{i1}^{m}(z_{i}-y_{i}) \cdot x_{i2}\\[10pt] \dots\\[10pt] \frac{1}{m}\sum\limits_{i1}^{m}(z_{i}-y_{i}) \cdot x_{in}\\[10pt] \end{bmatrix}∂w∂E​​∂w1​∂E​∂w2​∂E​…∂wn​∂E​​​​m1​i1∑m​(zi​−yi​)⋅xi1​m1​i1∑m​(zi​−yi​)⋅xi2​…m1​i1∑m​(zi​−yi​)⋅xin​​​前面我们推导出沿着梯度方向下降最快Δ w k η Δ r ∂ E ∂ w k w k ( t 1 ) w k ( t ) − η Δ r ∂ E ∂ w k \Delta{w_k} \eta\Delta{r}\frac{\partial E}{\partial w_{k}} \\[10pt] w_{k}^{(t1)}w_{k}^{(t)} - \eta\Delta{r}\frac{\partial E}{\partial w_{k}} \\[10pt]Δwk​ηΔr∂wk​∂E​wk(t1)​wk(t)​−ηΔr∂wk​∂E​这里Δ r \Delta{r}Δr为 w 在梯度l ⃗ \vec{l}l方向增长一个微小单位分解到各维度方向增长就是Δ r cos ⁡ θ k \Delta{r}\cos{\theta_k}Δrcosθk​θ k \theta_kθk​为梯度l ⃗ \vec{l}l与w k w_kwk​维度方向的夹角。η 1 ∑ k 1 n ( ∂ E ∂ w k ) 2 cos ⁡ θ k η ∂ E ∂ w k ∂ E ∂ w k ∑ k 1 n ( ∂ E ∂ w k ) 2 \eta\frac{1}{\sqrt{\sum\limits_{k1}^{n}{(\frac{\partial E}{\partial {w_k}})^2}}}\\[10pt] %% \cos{\theta_k}\eta\frac{\partial E}{\partial {w_k}}\frac{\frac{\partial E}{\partial {w_k}}}{\sqrt{\sum\limits_{k1}^{n}{(\frac{\partial E}{\partial {w_k}})^2}}}\\[10pt]ηk1∑n​(∂wk​∂E​)2​1​cosθk​η∂wk​∂E​k1∑n​(∂wk​∂E​)2​∂wk​∂E​​python 实战以异或门为例真值表为i iix i 1 x_{i1}xi1​x i 2 x_{i2}xi2​y i y_iyi​i1000i2011i3101i4110可以把真值表中的每一行都看作是一个样本x i 1 x_{i1}xi1​与x i 2 x_{i2}xi2​为异或门的 第 i 个样本输入y i y_iyi​为第 i 个样本的输出。LogisticRegression定义好逻辑回归训练类 LogisticRegression包括前向计算反向传播importnumpyasnpimportmatplotlib.pyplotaspltfrommatplotlib.colorsimportListedColormapclassLogisticRegression:手动实现的逻辑回归def__init__(self,learning_rate0.1,n_iterations10000):self.lrlearning_rate self.n_itern_iterations self.weightsNoneself.biasNoneself.losses[]defsigmoid(self,z):Sigmoid激活函数return1/(1np.exp(-z))deffit(self,X,y):训练逻辑回归模型n_samples,n_featuresX.shape# 初始化参数self.weightsnp.random.randn(n_features)self.bias0# 梯度下降foriterationinrange(self.n_iter):# 前向传播unp.dot(X,self.weights)self.bias zself.sigmoid(u)# 计算损失,避免log中出现0的情况,加上一个非常小的值loss-np.mean(y*np.log(z1e-15)(1-y)*np.log(1-z1e-15))self.losses.append(loss)# 计算梯度dw(1/n_samples)*np.dot(X.T,(z-y))db(1/n_samples)*np.sum(z-y)# 更新参数self.weights-self.lr*dw self.bias-self.lr*db# 打印进度ifiteration%10000:accuracyself.score(X,y)print(fIteration{iteration}, Loss:{loss:.4f}, Accuracy:{accuracy:.4f})defpredict_proba(self,X):预测概率linear_outputnp.dot(X,self.weights)self.biasreturnself.sigmoid(linear_output)defpredict(self,X,threshold0.5):预测类别probabilitiesself.predict_proba(X)return(probabilitiesthreshold).astype(int)defscore(self,X,y):计算准确率predictionsself.predict(X)returnnp.mean(predictionsy)训练及结果准备样本数据进行训练并绘制损失函数分别尝试线性特征与使用多项式特征来训练求解defcreate_xor_data():创建异或门数据Xnp.array([[0,0],[0,1],[1,0],[1,1]])ynp.array([0,1,1,0])returnX,ydefadd_polynomial_features(X):添加多项式特征解决线性不可分问题# 添加平方项和交叉项X_polynp.hstack([X,X[:,0:1]**2,# x1²X[:,1:2]**2,# x2²(X[:,0:1]*X[:,1:2]),# x1*x2])returnX_polydefvisualize_results(X,y,model,title,feature_typelinear):可视化决策边界和结果fig,axesplt.subplots(1,2,figsize(12,5))# 创建网格点x_min,x_maxX[:,0].min()-0.5,X[:,0].max()0.5y_min,y_maxX[:,1].min()-0.5,X[:,1].max()0.5xx,yynp.meshgrid(np.arange(x_min,x_max,0.01),np.arange(y_min,y_max,0.01))iffeature_typepoly:# 为多项式特征准备网格数据gridnp.c_[xx.ravel(),yy.ravel()]grid_polyadd_polynomial_features(grid)Zmodel.predict(grid_poly)else:gridnp.c_[xx.ravel(),yy.ravel()]Zmodel.predict(grid)ZZ.reshape(xx.shape)# 绘制决策边界axes[0].contourf(xx,yy,Z,alpha0.4,cmapListedColormap([#FFAAAA,#AAAAFF]))axes[0].scatter(X[:,0],X[:,1],cy,s100,cmapListedColormap([red,blue]),edgecolorsk)axes[0].set_xlabel(x1)axes[0].set_ylabel(x2)axes[0].set_title(fDecision Boundary -{title})axes[0].grid(True,alpha0.3)# 绘制损失曲线axes[1].plot(model.losses)axes[1].set_xlabel(Iteration)axes[1].set_ylabel(Loss)axes[1].set_title(Loss Curve)axes[1].grid(True,alpha0.3)plt.tight_layout()plt.show()defmain():# 1. 准备数据X,ycreate_xor_data()print(原始数据:)print(X:,X)print(y:,y)print()# 2. 尝试线性特征无法解决异或问题print(*50)print(1. 使用线性特征应该失败:)print(*50)model_linearLogisticRegression(learning_rate0.1,n_iterations10000)model_linear.fit(X,y)print(f线性模型准确率:{model_linear.score(X,y):.4f})print(f权重:{model_linear.weights}, 偏置:{model_linear.bias:.4f})print(f预测结果:{model_linear.predict(X)})visualize_results(X,y,model_linear,Linear Features,linear)# 3. 使用多项式特征可以解决异或问题print(\n*50)print(2. 使用多项式特征应该成功:)print(*50)X_polyadd_polynomial_features(X)print(多项式特征形状:,X_poly.shape)print(特征示例:,X_poly[0])model_polyLogisticRegression(learning_rate0.1,n_iterations20000)model_poly.fit(X_poly,y)print(f多项式模型准确率:{model_poly.score(X_poly,y):.4f})print(f权重:{model_poly.weights}, 偏置:{model_poly.bias:.4f})print(f预测结果:{model_poly.predict(X_poly)})# 显示详细预测print(\n详细预测:)foriinrange(len(X)):probmodel_poly.predict_proba(X_poly[i:i1])[0]predmodel_poly.predict(X_poly[i:i1])[0]print(fX{X[i]}, y{y[i]}, 预测概率{prob:.6f}, 预测类别{pred})visualize_results(X,y,model_poly,Polynomial Features,poly)运行结果运行main()得到如下结果使用线性特征原始数据: X: [[0 0] [0 1] [1 0] [1 1]] y: [0 1 1 0] 1. 使用线性特征应该失败: Iteration 0, Loss: 0.6989, Accuracy: 0.5000 Iteration 1000, Loss: 0.6931, Accuracy: 0.5000 Iteration 2000, Loss: 0.6931, Accuracy: 0.7500 Iteration 3000, Loss: 0.6931, Accuracy: 0.7500 Iteration 4000, Loss: 0.6931, Accuracy: 0.7500 Iteration 5000, Loss: 0.6931, Accuracy: 0.7500 Iteration 6000, Loss: 0.6931, Accuracy: 0.7500 Iteration 7000, Loss: 0.6931, Accuracy: 0.7500 Iteration 8000, Loss: 0.6931, Accuracy: 0.7500 Iteration 9000, Loss: 0.6931, Accuracy: 0.5000 线性模型准确率: 0.5000 权重: [-2.48590055e-16 -2.48352572e-16], 偏置: 0.0000 预测结果: [1 1 1 1]多项式特征 1. 使用多项式特征应该成功: 多项式特征形状: (4, 5) 特征示例: [0 0 0 0 0] Iteration 0, Loss: 0.9076, Accuracy: 0.5000 Iteration 1000, Loss: 0.2192, Accuracy: 1.0000 Iteration 2000, Loss: 0.1162, Accuracy: 1.0000 Iteration 3000, Loss: 0.0774, Accuracy: 1.0000 Iteration 4000, Loss: 0.0576, Accuracy: 1.0000 Iteration 5000, Loss: 0.0457, Accuracy: 1.0000 Iteration 6000, Loss: 0.0378, Accuracy: 1.0000 Iteration 7000, Loss: 0.0322, Accuracy: 1.0000 Iteration 8000, Loss: 0.0280, Accuracy: 1.0000 Iteration 9000, Loss: 0.0248, Accuracy: 1.0000 Iteration 10000, Loss: 0.0223, Accuracy: 1.0000 Iteration 11000, Loss: 0.0202, Accuracy: 1.0000 Iteration 12000, Loss: 0.0184, Accuracy: 1.0000 Iteration 13000, Loss: 0.0170, Accuracy: 1.0000 Iteration 14000, Loss: 0.0157, Accuracy: 1.0000 Iteration 15000, Loss: 0.0146, Accuracy: 1.0000 Iteration 16000, Loss: 0.0137, Accuracy: 1.0000 Iteration 17000, Loss: 0.0129, Accuracy: 1.0000 Iteration 18000, Loss: 0.0121, Accuracy: 1.0000 ... X[0 0], y0, 预测概率0.014450, 预测类别0 X[0 1], y1, 预测概率0.989687, 预测类别1 X[1 0], y1, 预测概率0.989687, 预测类别1 X[1 1], y0, 预测概率0.008247, 预测类别0总结逻辑回归是线性分类器决策边界是超平面w 1 x 1 w 2 x 2 . . . w n x n 0 w_1x_1w_2x_2...w_nx_n0w1​x1​w2​x2​...wn​xn​0对于原始异或问题(0,0)→ 类别0(0,1)→ 类别1(1,0)→ 类别1(1,1)→ 类别0在二维空间中不存在一条直线能完美分离这 4 个点。这是线性不可分问题的经典例子。解决方案特征映射通过添加多项式特征将数据映射到高维空间[ x 1 x 2 ] → [ x 1 x 2 x 1 2 x 2 2 x 1 x 2 ] \begin{bmatrix} x_1 x_2 \end{bmatrix} \rightarrow \begin{bmatrix} x_1 x_2 {x_1}^2 {x_2}^2 {x_1x_2} \end{bmatrix}[x1​​x2​​]→[x1​​x2​​x1​2​x2​2​x1​x2​​]那么决策边界就变成了多项式w 1 x 1 w 2 x 2 w 3 x 1 2 w 4 x 2 2 w 5 x 1 x 2 0 w_1x_1w_2x_2w_3{x_1}^2w_4{x_2}^2w_5{x_1x_2}0w1​x1​w2​x2​w3​x1​2w4​x2​2w5​x1​x2​0在高维空间中数据变得线性可分这就是神经网络中隐藏层的思想基础。