Update example files so that tr_loss is not affected by args.gradient_accumulation_step

This commit is contained in:
Mathieu Prouveur committed 2019-04-24 14:07:00 +02:00
1 parent c36cca075a
commit ed8fad7390
2 files changed
+2 -2

No files matched your search

+1 -1
View File
@@ -845,7 +845,7 @@ def main():
else:
loss.backward()
tr_loss += loss.item()
tr_loss += loss.item() * args.gradient_accumulation_steps
nb_tr_examples += input_ids.size(0)
nb_tr_steps += 1
if (step + 1) % args.gradient_accumulation_steps == 0: