PyTorch 2.0 introduces accelerated Transformer API to democratize ML
The Transformer API supports more use cases now, such as models using Cross-Attention.
Linux Foundation Transformer Decoders Python Frame Evaluation Hooks Accelerated Pytorch Metal Performance Shaders
Source: sdtimes.com