magicslabnu
/

Clip_OutEffHop_bert_base

Model card Files Files and versions

robinzixuan commited on Jun 15, 2024

Commit

986b348

·

verified ·

1 Parent(s): 51eda75

Update modeling_bert.py

Files changed (1) hide show

modeling_bert.py +1 -1

modeling_bert.py CHANGED Viewed

@@ -286,7 +286,7 @@ class BertSelfAttention(nn.Module):
         self,
         config,
         position_embedding_type=None,
-        softmax_fn=partial(clipped_softmax, gamma=-0.025, eta=1.0),
         max_seq_length = 128,
         alpha=None,
         ssm_eps=None,

         self,
         config,
         position_embedding_type=None,
+        softmax_fn=partial(clipped_softmax1, gamma=-0.025, eta=1.0),
         max_seq_length = 128,
         alpha=None,
         ssm_eps=None,