Exploring mixed-precision techniques to speed up large language model training and inference by up to 3x without losing accuracy.
Explores how mixed-precision training techniques can speed up large language model training and inference by up to 3x, reducing memory use.
A curated list of open-source Large Language Models (LLMs) available for commercial use, including community-contributed updates and details.
A technical tutorial on fine-tuning a 20B+ parameter LLM using PyTorch FSDP and Hugging Face on Amazon SageMaker's multi-GPU infrastructure.
Explains parameter-efficient finetuning methods for large language models, covering techniques like prefix tuning and LLaMA-Adapters.
A guide to parameter-efficient finetuning methods for large language models, covering techniques like prefix tuning and LLaMA-Adapters.
Introduces IGEL, an instruction-tuned German large language model based on BLOOM, for NLP tasks like translation and QA.
A guide to finetuning large language models like BLOOM on a single GPU using gradient accumulation to overcome memory limits.
Guide to finetuning large language models on a single GPU using gradient accumulation to overcome memory limitations.
A technical guide on fine-tuning the large FLAN-T5 XXL model efficiently using LoRA and Hugging Face libraries on a single GPU.
Guide to fine-tuning the large FLAN-T5 XXL model using Amazon SageMaker managed training and DeepSpeed for optimization.
Argues against the 'lossy compression' analogy for LLMs like ChatGPT, proposing instead that they are simulators creating temporary simulacra.
A curated reading list of key academic papers for understanding the development and architecture of large language models and transformers.
A curated reading list of key academic papers for understanding the development and architecture of large language models and transformers.
Analyzes the limitations of AI chatbots like ChatGPT in providing accurate technical answers and discusses the need for curated data and human experts.
Discusses the limitations of AI chatbots like ChatGPT in providing accurate technical answers and proposes curated resources and expert knowledge as future solutions.
Learn to optimize GPT-J inference using DeepSpeed-Inference and Hugging Face Transformers for faster GPU performance.
An engineer shares insights and tutorials on applying Cohere's large language models for real-world tasks like prompt engineering and semantic search.
Explores how Large Language Models perform implicit Bayesian inference through in-context learning, connecting exchangeable sequence models to prompt-based learning.
Explains how retrieval-augmented language models like RETRO achieve GPT-3 performance with far fewer parameters by querying external knowledge.
