Pathways: Asynchronous Distributed Dataflow for ML

Attention Is All You Need

The seminal paper introducing the Transformer model, which has become central to many state-of-the-art NLP models.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Introduction of BERT, a new method for pre-training language representations that achieve state-of-the-art results on a variety of NLP tasks.

Generative Pretrained Transformer 3 (GPT-3)

The third-generation model in the GPT-n series by OpenAI, showcasing the power of scaling up language models.

GPT-3: Language Models are Few-Shot Learners

Details the development and capabilities of GPT-3, illustrating its few-shot learning ability across diverse tasks.

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Introduces EfficientNet, a systematic method for scaling CNN architectures, achieving state-of-the-art accuracy with significantly reduced parameters.

DALLE: Creating Images from Text

Presents DALL·E, a model that generates diverse and detailed images from textual descriptions, demonstrating the intersection of language understanding and visual creativity.

T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Introduces the T5 model, showcasing its versatility across multiple NLP tasks through a unified framework for text-to-text processing.

DeepMind's AlphaFold: A Solution to a 50-Year-Old Grand Challenge in Biology

Details the AlphaFold system by DeepMind, which made significant breakthroughs in protein folding, impacting biological sciences.

AI for Procedural Content Generation in Games

Reviews the application of AI in generating game content, emphasizing the role of machine learning in creative processes.

Quantum Machine Learning for 6G Communication Networks

Explores the potential of quantum machine learning to revolutionize 6G communication networks, highlighting future research directions.