Tensorflow Version: 2.x
Python: 3.7.4
Disconnected graph: I am trying to replicate the below model architecture, but right part seems to be disconnected when I tried to plot the model in Keras. I already passed the hidden matrices HQ(For question) and HA(For answer) as inputs to attention layer (We can see the inputs to Coattention layer in summary below - The input shapes are (512,600) and (512, 600) and Coattention output shapes are also same for matrices CQ and CA) . Please help me understand this disconnection. Does this needs to be corrected or can this be ignored?
Final Model:
inputs = [input_text1, input_text2]
outputs = score_oq_oa
model = Model(inputs=inputs, outputs=outputs)
model.summary()
Expected Model Architecture:
Model generated graph: Why is it disconnected on right side? Please help me understand. I did not use concatenate layer after bidirectional layers of question and answer but I just passed output matrices of both bidirectional layers as inputs to attention layer as stated above.
Question updated with code for Coattention layer as below:
Here HQ and HA are hidden state matrices/outputs of two separate bidirectional layers as we see in model architecture.
class coattention(tf.keras.layers.Layer):
def __init__(self):
super(coattention, self).__init__()
def call(self, HQ, HA):
L = tf.linalg.matmul(HA, HQ, transpose_a = True, transpose_b = False)
AQ = tf.nn.softmax(L, axis = 1)
AA = tf.nn.softmax(tf.transpose(L), axis = 1)
CQ = tf.linalg.matmul(HA, AQ, transpose_a = False, transpose_b = False)
CA = tf.linalg.matmul(HQ, AA, transpose_a = False, transpose_b = False)
return CQ, CA
coattention_layer = coattention()
CQ, CA = coattention_layer(HQ, HA)
print ("Shape of Context vector of Question (CQ): ", CQ.shape)
print ("Shape of Context vector of Answer (CA): ", CA.shape)
Shape of Context vector of Question (CQ): (512, 600)
Shape of Context vector of Answer (CA): (512, 600)
Because you did not provide the code, I believe you forgot to call the coattention layer with Bidirectional_7 layer as input.
Here is the example code
Ha = Input(shape=(1,2,3), name='Ha')
Hq = Input(shape=(1,2,3), name='Hq')
your_coattention_layer = Dense(12, name='your_coattention_layer')
# this part that I think you forgot
Ca = your_coattention_layer(Ha)
cQ = your_coattention_layer(Hq)
out1 = Dense(123, name='your_Ca_layer')(Ca)
out2 = Dense(123, name='your_Cq_later')(cQ)
M = Model(inputs=[Ha,Hq], outputs=[out1,out2])
M.summary()
from keras.utils import plot_model
plot_model(M, to_file='Example.png')
This is the model picture.
