Hoping to be what Paul Graham called hacker.
Hey, I'm zhuzilin, an engineer driven by curiosity.
My main focus is on MLSys.
- You can ask me about deep learning frameworks. I am contributor to many tools like pytorch, tensorflow and horovod.
- I am a LLM believer and am really lucky to get hands dirty on training them @WeChat, from pretraining from scratch to sft and rlhf, along with writing training frameworks for those.
- Recently, I wrote ring-flash-attention and am working on improving OpenRLHF/OpenRLHF.
I'm also interested in JavaScript engine. I've read the es5 spec to write es and helped fixed bugs in the early stage of oven-sh/bun.
Avatar is Shoyo Hinata, from Haikyu!!.
我是 zhuzilin，一个由兴趣驱动的工程师~
我的主要精力放在 MLSys 领域。
- 我对深度学习训练框架比较了解，是 pytorch, tensorflow, horovod 等工具的 contributor。
- LLM 信徒，在微信大模型团队打工中。有幸深入接触过 LLM 训练的各个环节，不管是从零预训练，还是 sft 与 rlhf，以及写用来做这些事的训练框架。
- 最近写了 ring-flash-attention，并且在尝试优化 OpenRLHF/OpenRLHF 中。
我对 JavaScript 引擎也比较感兴趣。读过 spec，写过解释器（es），还给早期的 oven-sh/bun 提过一些 bugfix。
头像是日向翔阳，《排球少年》。