Lately, we have been using Lightning Fabric, which brings together:
• The flexibility of PyTorch.
• And distributed training features that PyTorch Lightning provides.
You only need to make 4 minor changes to your existing PyTorch code to easily scale it to the largest billion-parameter models/LLMs. (View Highlight)
While creating the Fabric object above, you can specify:
• the accelerator and the number of devices
• the parallelism strategy to use
• the floating point precision, etc.
Next, configure the model, the optimizer, and the dataloader (View Highlight)
