RMSNorm weight initialization

[bugm]

I noticed that chatglm3 uses torch.empty to initialize the weight of trainable parameter in RMSNorm instead of torch.ones. That is different from the implementation in the original paper and llama.
Is there any special reason to use a tensor with uninitialized data to initialize the weight?

[Btlmd]

If you are referring the following code

class RMSNorm(torch.nn.Module):
    def __init__(self, normalized_shape, eps=1e-5, device=None, dtype=None, **kwargs):
        super().__init__()
        self.weight = torch.nn.Parameter(torch.empty(normalized_shape, device=device, dtype=dtype))
        self.eps = eps

The modeling_chatglm.py model is not intended for training from scratch, but only for inference and finetuning. The weight is expected to be loaded from a pre-trained model, so the torch.empty() is not expected to take any effect.