 Li Dong
|
d0d9b384f2
|
LayerNorm initialization
The LayerNorm gamma and beta should be initialized by .fill_(1.0) and .zero_().
reference links:
https://github.com/tensorflow/tensorflow/blob/989e78c412a7e0f5361d4d7dfdfb230c8136e749/tensorflow/contrib/layers/python/layers/layers.py#L2298
https://github.com/tensorflow/tensorflow/blob/989e78c412a7e0f5361d4d7dfdfb230c8136e749/tensorflow/contrib/layers/python/layers/layers.py#L2308
|
2019-01-07 15:51:33 +08:00 |
|