1.全連接層是什么意思

假設(shè)有一份如下的用于監(jiān)督學(xué)習(xí)分類任務(wù)的數(shù)據(jù)集，共包含10個特征，其中有一個強(qiáng)特征（意味著該特征與標(biāo)簽值具備很強(qiáng)的關(guān)聯(lián)性，包含了充分的標(biāo)簽信息）在一般的分類任務(wù)中，“golden feature”對于分類器的性能有著重要的意義，此類特征的發(fā)現(xiàn)通常可以帶來模型性能的突破性提升。

2.全連接層圖片

在該任務(wù)中，既然該強(qiáng)特征包含了較多的可用于解釋標(biāo)簽的信息，那么我們希望模一室一廳出租型在自行訓(xùn)練過程中對該特征賦予更多的“注意力”，使得該特征在模型中的貢獻(xiàn)度得以正常的表現(xiàn)數(shù)據(jù)準(zhǔn)備與處理定義一個生成訓(xùn)練樣本的函數(shù)，將強(qiáng)特征的索引設(shè)置為5。

3.全連接層的原理

# 生成樣本的函數(shù)defmake_classifiction_dataset(n_samples,n_features,golden_feature_idx=0):assert0<=golden_feature_idx

4.全連接層的缺點(diǎn)

<=n_featureslabels=np.random.randint(0,2,n_samples)features=np.random.rand(n_samples,n_features)一室一廳出租features

5.全連接層工作原理

[:,golden_feature_idx-1]=labels+np.random.rand()/500returnfeatures,np.expand_dims(labels,1)# 總樣本數(shù)n_samples

6.全連接層如何進(jìn)行分類

=100000# 特征數(shù)n_features=20# 設(shè)置強(qiáng)特征索引為5features,labels=make_classifiction_dataset(n_samples=n_samples,n_features

7.全連接層和全連接網(wǎng)絡(luò)

=n_features,golden_feature_idx=5)# 對數(shù)據(jù)集進(jìn)行切分一室一廳出租，生成訓(xùn)練集與驗證集，訓(xùn)練集占比10%X_trin,X_test,y_train,y_test=train_test_split

8.全連接層后面接什么

(features,labels,test_size=0.1)2. 全連接層attention網(wǎng)絡(luò)# input layerinput_layer=Input(shape=(n_features,))# soft attention

9.全連接層參數(shù)怎么設(shè)置

attention_probs=Dense(n_features,activation=softmax,name=attention_vec)(input_layer)# multipyatte一室一廳出租ntion_mul

10.全連接層怎么實(shí)現(xiàn)

=Multiply(name=attention_mul)([input_layer,attention_probs])# fcdense=Dense(100)(attention_mul)# fcdense

=Dense(20)(dense)# output layeroutput_layer=Dense(1,activation=sigmoid)(dense)model=Model([input_layer

],outputs=[output_layer])model.compile(optimizer=Adam(0.003),loss="binary_一室一廳出租crossentropy",metrics=["acc"

])model.fit(X_trin,y_train,batch_size=256,epochs=5,verbose=1)2.1 訓(xùn)練過程Epoch1/590000/90000[==============================

]-17s188us/step-loss:0.0651-acc:0.9721Epoch2/590000/90000[==============================]-2s23us/step

-loss:4.8880e-05-acc:1.0000Epoch3/590000/90000[=====一室一廳出租=========================]-2s23us/step-loss:2.0385e-05

-acc:1.0000Epoch4/590000/90000[==============================]-2s23us/step-loss:1.1043e-05-acc:1.0000

Epoch5/590000/90000[==============================]-2s23us/step-loss:6.8297e-06-acc:1.00002.2 網(wǎng)絡(luò)結(jié)構(gòu)_______________________________________________一室一廳出租___________________________________________________

Layer(type)OutputShapeParam# Connected to ==================================================================================================

input_1(InputLayer)(None,20)0________________________________________________________________________________一室一廳出租__________________

attention_vec(Dense)(None,20)420input_1[0][0]__________________________________________________________________________________________________

attention_mul(Multiply)(None,20)0input_1[0][0]attention_vec[0][0]__________________________________________________________________________一室一廳出租________________________

dense_1(Dense)(None,100)2100attention_mul[0][0]__________________________________________________________________________________________________

dense_2(Dense)(None,20)2020dense_1[0][0]___________________________________________________________________________________________一室一廳出租_______

dense_3(Dense)(None,1)21dense_2[0][0]==================================================================================================

Totalparams:4,561Trainableparams:4,561Non-trainableparams:0

2.3 attention層可視化attention_model=Model(model.input,model.get_layer("attention_vec").output)atten_X_一室一廳出租test

=attention_model.predict(X_test)plt.bar([str(i)foriinrange(1,21)],np.mean(atten_X_test,axis=0))

從圖中可以看出在生成數(shù)據(jù)時設(shè)置的強(qiáng)特征（索引為5）在模型訓(xùn)練過程中得到了最多的關(guān)注，如果使用該softmax的輸出作為歸一化的特征重要性評估值的話，該強(qiáng)特征的特征重要度約為0.5，相當(dāng)于其他所有特征的總和。

3. 參考https://github.com/philipperemy/keras-attention-mechanism?github.com/philipperemy/keras-atten一室一廳出租tion-mechanism