Transformer

Definition

A neural network architecture that uses self-attention mechanisms to process sequential data, revolutionizing NLP.

In-Depth Explanation

Introduced in the 2017 paper "Attention Is All You Need," transformers replaced recurrent networks for sequence tasks. They process all positions simultaneously using attention mechanisms, enabling massive parallelization and capturing long-range dependencies. All modern LLMs are based on transformer architecture.

Real-World Example

GPT (Generative Pre-trained Transformer) and BERT are both built on transformer architecture.

6 views0 found helpful

Related Terms

Attention Mechanism

A neural network component that allows models to focus on relevant parts of the input when producing outputs.

Generative Pre-trained Transformer(GPT)

A family of large language models developed by OpenAI that generate human-like text based on input prompts.

Large Language Model(LLM)

A type of AI model trained on massive amounts of text data to understand, generate, and manipulate human language.