Flash Attention News Today

The State Space Model taking on Transformers

This article codes the self-attention mechanisms used in transformer architectures and large language models (LLMs) such as GPT-4 and Llama from scratch in PyTorch.

AMD launched its Instinct MI300X AI accelerator and the Instinct MI300A, the world’s first data center APU, during its Advancing AI event in San Jose, California.

VinAI has launched PhoGPT, an open-source, large language model designed specifically for the Vietnamese market.

AMD has announced the launch of its flagship AI GPU accelerator, the MI300X, which offers up to 60% better performance than NVIDIA's H100.

AMD Enters the AI Race with new MI300 and Partners Microsoft support will enable market share gains, AMD fully launched the long-awaited MI300, the company’s first GPU...

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks. - GitHub - mlabonne/llm-course: Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

<p>We proudly introduce LeoLM (<strong>L</strong>inguistically <strong>E</strong>nhanced <strong>O</strong>pen <strong>L</strong>anguage <strong>M</strong>od...

Listen now | Breaking down the viral Transformers Math 101 article and high performance distributed training for Transformers-based architectures (or "How I Learned to Stop Handwaving and Make the GPU go brrrrrr")

<p>In this blogpost, we introduce our latest series of chatbot models, LongChat-7B and LongChat-13B, featuring a new level of extended context length up to 1...