Segmentation Fault in TFRecurrentLanguageModel.cc

[mattiadg]

I went around issue #11 by commenting the line searching for the bias tensor, however I'm now getting another error that I'm not sure it's related.
The problem is that this loop can go on until a parent is a nullptr and then it crashes.

rasr/src/Lm/TFRecurrentLanguageModel.cc

Line 61 in 473d202

while (parent->state.empty()) {

In my case it performs a full execution of the while body and then crashes the second time it checks the condition.

I get there the first time TFRecurrentLanguageModel::forward is called. It enters from this line

rasr/src/Lm/TFRecurrentLanguageModel.cc

Line 609 in 473d202

request_graph.add_cache(const_cast<ScoresWithContext*>(sc));

[mattiadg]

If I change the while condition with
while (parent != nullptr && parent->state.empty())
then it crashes in

rasr/src/Lm/TFRecurrentLanguageModel.cc

Line 735 in 473d202

require(initial_cache != nullptr);

Now, I'll try to understand if the first error is linked to the absence of a bias, though I currently don't see how it can be.

[mattiadg]

I think that the problem is related to #11 because in BlasNceSoftmaxAdapter::get_score we have this line that assumes the existence of the bias tensor

rasr/src/Lm/BlasNceSoftmaxAdapter.cc

Line 44 in 473d202

result += tensors_[1].data<float>()[output_idx];

[mattiadg]

Update: now I have the bias tensor in the output layer wrapped by BlasNceSoftmaxAdapter but I still get the precondition error. Something odds happens in the history when using this layer

[mattiadg]

The error doesn't occur when the lstm in Returnn is compiled with the option "initial_state": "keep_over_epoch_no_init", but it is probably because the network was trained with the same option and it was missing from my compiled graph.