Tag: PyTorch

Scale LLMs with PyTorch 2.0 FSDP on Amazon EKS – Part 2

This is a guest post co-written with Meta’s PyTorch team and is a continuation of Part 1 of this series,

A cost-effective approach Photo by Adi Goldstein on Unsplash Amazon Web Service (“AWS”) Elastic Compute Cloud (“EC2”) presents a powerful

Let’s implement a regression example where the aim is to train a network to predict the value of a node

Using torch.index_select, torch.gather and torch.take In some situations, you’ll need to do some advanced indexing / selection with Pytorch, e.g.

Large language model (LLM) training has surged in popularity over the last year with the release of several popular models