"image_seq_length" vs "input_length" in ops/inputs.py

[brewer71]

In this section of code

START

image_seq_length = tf.shape(image_ids)[0]

input_length = tf.subtract(image_seq_length, 0) # change 1 to 0

cap_indicator = tf.cast(tf.not_equal(captions,tf.zeros_like(captions)),tf.int32)

indicator = tf.ones(tf.expand_dims(input_length, 0), dtype=tf.int32)

loss_indicator = tf.ones(tf.expand_dims(image_seq_length, 0), dtype=tf.int32)

END

what is the difference between "image_seq_length" and "input_length" ?

both the variable "image_seq_length" and "input_length" will have same value , is that correct ?

if both are same then why are we using two seperate varaibles and creating two seperate mask ?

what is the use of input_mask vs loss_mask ?

tf.subtract(image_seq_length, 0) # change 1 to 0

[zuoxiang95]

I don't understand this line of code either.

polyvore/polyvore/ops/inputs.py

Line 206 in e0ca93b

input_length = tf.sub(image_seq_length, 0) # change 1 to 0

[xthan]

Sorry for the late response.

polyvore/polyvore/ops/inputs.py

Line 206 in e0ca93b

input_length = tf.sub(image_seq_length, 0) # change 1 to 0

It is a legacy issue. During traditional image captioning task, the input_length is - 1. In our task, we just use input_length = image_seq_length. You can just need ignore this line of code.

As for input_mask and loss_mask:

polyvore/polyvore/polyvore_model_bi.py

Line 548 in e0ca93b

input_mask = tf.to_float(

input_mask is one dimensional longer than loss_mask.

[zuoxiang95]

@xthan thanks for your kind reply. There is another question that puzzles me.

polyvore/polyvore/set_generation.py

Line 93 in e0ca93b

if next_image == test_feat.shape[0] - 1 or curr_score[0][-1] > 0.00001:

When i use my own dataset to generate outfit, the model generate more than 10 images. Generating conditions seem to be a bit problematic. why use the last image's probability > 0.00001 or next image is last image to stop generation. Looking forward to your kind reply!

[xthan]

The last test_feat is the representation of <\end> (EOS), meaning the sequence should stop. Also, we think the confidence should be larger than a threshold (0.00001 in our case) to output confident sequence, otherwise, we terminate the generation.

[zuoxiang95]

we think the confidence should be larger than a threshold (0.00001 in our case) to output confident sequence, otherwise, we terminate the generation.

If we think the confidence should be larger than a threshold, we should set curr_score[0][-1] < 0.00001 to stop the generation?
And I think we should sort curr_score first, and get the maximum probability to compare with threshold.

[xthan]

Oh, sorry I said it wrong. curr_score[0][-1] is the confidence of predicting EOS. That is to say, if it predicts EOS larger than a threshold, it should stop.

[zuoxiang95]

Okay, i understand it. Thanks for your reply.

[zuoxiang95]

@xthan hello, xthan.

polyvore/polyvore/set_generation.py

Line 159 in e0ca93b

curr_score = np.exp(np.dot(lstm_output, np.transpose(test_feat)))

From this line, i don't think the curr_score[0][-1] is the probability of predicting EOS.
I guess it's the score of input image's rnn feature and the last image's rnn feature of test set.
lstm_output's shape is [batch_size, 512], test_feat's shape is [num_image, 521], so curr_score's shape is [batch_size, num_image].
I‘’m just getting started with this. If there has anything wrong, please don't mind. Thanks again.