{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":541386254,"defaultBranch":"main","name":"TencentPretrain","ownerLogin":"Tencent","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2022-09-26T03:01:31.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/18461506?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1704684685.0","currentOid":""},"activityList":{"items":[{"before":"27547a48e610f5ca5ee9aca6acb5fafe5f7843e5","after":"ed7984355235d35b19c9770e35c3c763b36434df","ref":"refs/heads/main","pushedAt":"2024-05-01T10:38:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"Update config.py","shortMessageHtmlLink":"Update config.py"}},{"before":"dc155e40c631a27e41db494404e6c5d3f8758bad","after":"27547a48e610f5ca5ee9aca6acb5fafe5f7843e5","ref":"refs/heads/main","pushedAt":"2024-04-26T08:24:04.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"Fixed some bugs regarding activation checkpoints and updated the BPE vocabulary loader (#125)\n\n* Add token counter, update BPE vocab init\r\n\r\n* Add special token security check\r\n\r\n* update no_decay list\r\n\r\n* [Fix] Fixed the impact of passing parameters on activation checkpointing.\r\n\r\n* update\r\n\r\n* update\r\n\r\n* update\r\n\r\n---------\r\n\r\nCo-authored-by: kaeli <kaeli@tencent.com>","shortMessageHtmlLink":"Fixed some bugs regarding activation checkpoints and updated the BPE …"}},{"before":"c13cf17823039d5cae3eff443677126e84bf5190","after":"dc155e40c631a27e41db494404e6c5d3f8758bad","ref":"refs/heads/main","pushedAt":"2024-04-25T06:52:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"optimize the way of appending. (#126)\n\n* optimize the way of appending.\r\n\r\n* optimize the way of appending.\r\n\r\n---------\r\n\r\nCo-authored-by: wheatxzhang <wheatxzhang@tencent.com>","shortMessageHtmlLink":"optimize the way of appending. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2255491666\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/126\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/126/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/126\">#126</a>)"}},{"before":"b0a9591cf19ffc996006cfe919552702aced63e7","after":"c13cf17823039d5cae3eff443677126e84bf5190","ref":"refs/heads/main","pushedAt":"2024-03-09T12:08:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"change readme","shortMessageHtmlLink":"change readme"}},{"before":"86531a8a3814ec38537dd3cac2e6a33f156894d6","after":"b0a9591cf19ffc996006cfe919552702aced63e7","ref":"refs/heads/main","pushedAt":"2024-03-06T06:45:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"hhou435","name":"Cheng hou","path":"/hhou435","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/59219579?s=80&v=4"},"commit":{"message":"Update lm_target.py\n\nFixed bug when not using model parallel training","shortMessageHtmlLink":"Update lm_target.py"}},{"before":"6feab42642ab05fa8d18f45aab42067e270b02f4","after":"86531a8a3814ec38537dd3cac2e6a33f156894d6","ref":"refs/heads/main","pushedAt":"2024-02-14T18:50:17.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"update readme","shortMessageHtmlLink":"update readme"}},{"before":"f929ae0f92f8a853a8c264620ef1680872d07a6a","after":"22201efa890d6ff7bd9b68994901cdeb40c5b038","ref":"refs/heads/clip","pushedAt":"2024-02-06T08:09:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"294315f6422894f3970ed8696e7fa499cace4bfb","after":"f929ae0f92f8a853a8c264620ef1680872d07a6a","ref":"refs/heads/clip","pushedAt":"2024-02-06T04:28:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"f5ce4e5e0c30125e6f626dde9872752c0a129afc","after":"6feab42642ab05fa8d18f45aab42067e270b02f4","ref":"refs/heads/main","pushedAt":"2024-01-24T11:34:47.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"fix bug (#123)\n\n* Update word_embedding.py\r\n\r\n* Update transformer_encoder.py","shortMessageHtmlLink":"fix bug (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2098068920\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/123\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/123/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/123\">#123</a>)"}},{"before":"1dc9dd1af343707e4ec6e1c866221a233f58572e","after":"f5ce4e5e0c30125e6f626dde9872752c0a129afc","ref":"refs/heads/main","pushedAt":"2024-01-24T11:20:28.000Z","pushType":"pr_merge","commitsCount":5,"pusher":{"login":"hhou435","name":"Cheng hou","path":"/hhou435","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/59219579?s=80&v=4"},"commit":{"message":"Merge pull request #114 from hhou435/pipeline\n\nadd support for pipeline parallelism","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2051776728\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/114\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/114/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/114\">#114</a> from hhou435/pipeline"}},{"before":"817a1b27abc5868ebfdfb6fcc5a10a1000699c11","after":"1dc9dd1af343707e4ec6e1c866221a233f58572e","ref":"refs/heads/main","pushedAt":"2024-01-24T10:53:39.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"fix lm bug (#122)\n\nCo-authored-by: hermanyu <hermanyu@tencent.com>","shortMessageHtmlLink":"fix lm bug (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2097874625\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/122\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/122/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/122\">#122</a>)"}},{"before":"c703d0314f27e6d0f284317479892555530816f4","after":"817a1b27abc5868ebfdfb6fcc5a10a1000699c11","ref":"refs/heads/main","pushedAt":"2024-01-23T02:47:00.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"fix unit test (#121)\n\nCo-authored-by: hermanyu <hermanyu@tencent.com>","shortMessageHtmlLink":"fix unit test (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2073495360\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/121\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/121/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/121\">#121</a>)"}},{"before":null,"after":"294315f6422894f3970ed8696e7fa499cace4bfb","ref":"refs/heads/clip","pushedAt":"2024-01-08T03:31:25.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"81904fb85c403412be77c656773ebdfad1b6f673","after":"21dca391ab5aa07e09f56debbed27ede8dd74dbe","ref":"refs/heads/flash_attention","pushedAt":"2024-01-08T03:20:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"1f28a18e71e731965f69b1ecea2846afe81d4787","after":"c703d0314f27e6d0f284317479892555530816f4","ref":"refs/heads/main","pushedAt":"2024-01-06T10:26:07.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"add mt classifier by using deepspeed (#117)\n\n* add mt classifier by using deepspeed\r\n\r\n* add parallel of infer\r\n\r\n* fix bug for mt cls and cls\r\n\r\n* fix bug\r\n\r\n* restore model loader\r\n\r\n* fix bug\r\n\r\n---------\r\n\r\nCo-authored-by: hermanyu <hermanyu@tencent.com>","shortMessageHtmlLink":"add mt classifier by using deepspeed (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2064996566\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/117\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/117/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/117\">#117</a>)"}},{"before":"120ff855141454cfc76bab87597189bad5ffb4fa","after":"81904fb85c403412be77c656773ebdfad1b6f673","ref":"refs/heads/flash_attention","pushedAt":"2023-12-25T07:11:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"d1af2536e140545ef60762ad164dff214becb807","after":"120ff855141454cfc76bab87597189bad5ffb4fa","ref":"refs/heads/flash_attention","pushedAt":"2023-12-25T07:10:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"51350e64971f8f64d8c625625c64a8bfb41efef6","after":"d1af2536e140545ef60762ad164dff214becb807","ref":"refs/heads/flash_attention","pushedAt":"2023-12-25T02:56:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"45c39bc91925374bac9667817bc0d97d9fb1a29f","after":"51350e64971f8f64d8c625625c64a8bfb41efef6","ref":"refs/heads/flash_attention","pushedAt":"2023-12-24T13:38:48.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"fa766f9016a3589edad5aa12c5471fe7287ebded","after":"45c39bc91925374bac9667817bc0d97d9fb1a29f","ref":"refs/heads/flash_attention","pushedAt":"2023-12-15T09:56:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"08c75fbee18bfa1da071322f4d2f376a5f924c64","after":"fa766f9016a3589edad5aa12c5471fe7287ebded","ref":"refs/heads/flash_attention","pushedAt":"2023-12-04T02:49:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"f53f8abb0cf35048f93a4a407b034821dbbd4ba9","after":"08c75fbee18bfa1da071322f4d2f376a5f924c64","ref":"refs/heads/flash_attention","pushedAt":"2023-12-03T14:07:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"test","shortMessageHtmlLink":"test"}},{"before":"b27211ed664ca3669e7860e0e55e796e215549c9","after":"1f28a18e71e731965f69b1ecea2846afe81d4787","ref":"refs/heads/main","pushedAt":"2023-11-28T07:16:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"Refactor transformer encoder (#111)","shortMessageHtmlLink":"Refactor transformer encoder (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2013731762\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/111\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/111/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/111\">#111</a>)"}},{"before":"f7f18c87486479a3c9ecd004e131f43fca47a1cc","after":"b27211ed664ca3669e7860e0e55e796e215549c9","ref":"refs/heads/main","pushedAt":"2023-11-21T08:42:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"hhou435","name":"Cheng hou","path":"/hhou435","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/59219579?s=80&v=4"},"commit":{"message":"fix bug","shortMessageHtmlLink":"fix bug"}},{"before":"669d46cb1006b8656b34db3de86aab176dbc2d4e","after":"f7f18c87486479a3c9ecd004e131f43fca47a1cc","ref":"refs/heads/main","pushedAt":"2023-11-16T12:09:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"add support for model parallelism (#110)\n\n* Add support for Model Parallelism\r\n\r\n* Add support for Model Parallelism\r\n\r\n* Add support for Model Parallelism\r\n\r\n* Add support for Model Parallelism\r\n\r\n* Add support for Model Parallelism\r\n\r\n* Update\r\n\r\n* update\r\n\r\n* update\r\n\r\n* Update trainer.py\r\n\r\n* Update trainer.py\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Add files via upload\r\n\r\n* Update convert_llama_from_megatron_checkpoint_to_pytorch_checkpoint.py\r\n\r\n* Update convert_llama_from_pytorch_checkpoint_to_megatron_checkpoint.py\r\n\r\n* Update trainer.py\r\n\r\n* Update convert_llama_from_megatron_checkpoint_to_pytorch_checkpoint.py\r\n\r\n* Update convert_llama_from_pytorch_checkpoint_to_megatron_checkpoint.py\r\n\r\n* update dataloader name\r\n\r\n* update comment\r\n\r\n---------\r\n\r\nCo-authored-by: Cheng <435405393@qq.com>\r\nCo-authored-by: “karots123” <“962”813115@qq.com>\r\nCo-authored-by: kaeli <kaeli@tencent.com>","shortMessageHtmlLink":"add support for model parallelism (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1978376142\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/110\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/110/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/110\">#110</a>)"}},{"before":"459769193818224667d5d20f3ae8e0f18163fb6e","after":"f53f8abb0cf35048f93a4a407b034821dbbd4ba9","ref":"refs/heads/flash_attention","pushedAt":"2023-11-05T02:44:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"Update pretrain.py","shortMessageHtmlLink":"Update pretrain.py"}},{"before":"3ab116106fa85c66a45673a7242a0ea8c76b3e3f","after":"459769193818224667d5d20f3ae8e0f18163fb6e","ref":"refs/heads/flash_attention","pushedAt":"2023-11-05T02:42:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"Update dataloader.py","shortMessageHtmlLink":"Update dataloader.py"}},{"before":"d72dcbed4b98427e1b41b6adf07d096d4993f185","after":"669d46cb1006b8656b34db3de86aab176dbc2d4e","ref":"refs/heads/main","pushedAt":"2023-10-27T08:04:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"fix pegasusu convert (#109)\n\n* fix pegasus convert\r\n\r\n* add line\r\n\r\n---------\r\n\r\nCo-authored-by: janinezhao <janinezhao@tencent.com>","shortMessageHtmlLink":"fix pegasusu convert (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1964660868\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/109\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/109/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/109\">#109</a>)"}},{"before":"1baa4073ba430fe1b379efbb581006e0cad4b26f","after":"d72dcbed4b98427e1b41b6adf07d096d4993f185","ref":"refs/heads/main","pushedAt":"2023-10-25T11:31:21.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zhezhaoa","name":null,"path":"/zhezhaoa","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10495098?s=80&v=4"},"commit":{"message":"fix s2t prepare dataset (#107)\n\n* fix configs\r\n\r\n* fix specaugment\r\n\r\n* fix\r\n\r\n* fix\r\n\r\n* fix\r\n\r\n* fix name\r\n\r\n* fix\r\n\r\n* fix s2t\r\n\r\n* fix ft s2t\r\n\r\n* rm model_source args\r\n\r\n* fix prepare\r\n\r\n* fix prepare\r\n\r\n---------\r\n\r\nCo-authored-by: janinezhao <janinezhao@tencent.com>","shortMessageHtmlLink":"fix s2t prepare dataset (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1961093259\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Tencent/TencentPretrain/issues/107\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Tencent/TencentPretrain/pull/107/hovercard\" href=\"https://github.com/Tencent/TencentPretrain/pull/107\">#107</a>)"}},{"before":"ecf369cf02f9d74561eca0cd6707995e8798c32b","after":"3ab116106fa85c66a45673a7242a0ea8c76b3e3f","ref":"refs/heads/flash_attention","pushedAt":"2023-10-25T10:04:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ydli-ai","name":"Li Yudong (李煜东)","path":"/ydli-ai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/20391895?s=80&v=4"},"commit":{"message":"Update multi_headed_attn.py","shortMessageHtmlLink":"Update multi_headed_attn.py"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEPs2ZlwA","startCursor":null,"endCursor":null}},"title":"Activity · Tencent/TencentPretrain"}