prefix_tuning error: RuntimeError: The size of tensor a (170) must match the size of tensor b (186) at non-singleton dimension 2

[ngls2021]

the problem of prefix tuning, I don't know how to fix this problem when using prefix tuning.
File "/root/.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py", line 826, in forward
full_attention_mask = self.get_masks(input_ids, past_key_values, padding_mask=attention_mask)
File "/root/.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py", line 689, in get_masks
full_attention_mask = full_attention_mask * padding_mask.unsqueeze(1)
RuntimeError: The size of tensor a (170) must match the size of tensor b (186) at non-singleton dimension 2

the dim of input_id is [4, 166], not contain the virtual token

the second dim of full_attention_mask is 166, which is not equal to the second dim of padding_mask