Skip to content

v0.2.1
Compare
Choose a tag to compare
@laekov laekov released this 23 Aug 08:28
· 155 commits to master since this release
v0.2.1
d2392de
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Load balancing

  • Fix gradient for balance loss.

Misc

  • Typos.
  • Update benchmark interface.
  • Remove some redundant code for performance improvement.
  • Enable USE_NCCL by default.
  • Compatibility for PyTorch <1.8.0 and >=1.8.0.

Megatron adaption

  • Patch for numerical correctness of gradient clipping.
  • Support to pipeline parallelism.
Assets 2