Peft on AI Knowledge Base

Adapter injection

Mon, 01 Jan 0001 00:00:00 +0000

Adapters

Mon, 01 Jan 0001 00:00:00 +0000

Contribute to PEFT

Mon, 01 Jan 0001 00:00:00 +0000

Custom models

Mon, 01 Jan 0001 00:00:00 +0000

DeepSpeed

Mon, 01 Jan 0001 00:00:00 +0000

Fully Sharded Data Parallel

Mon, 01 Jan 0001 00:00:00 +0000

IA3

Mon, 01 Jan 0001 00:00:00 +0000

IA3

Mon, 01 Jan 0001 00:00:00 +0000

Installation

Mon, 01 Jan 0001 00:00:00 +0000

LoRA

Mon, 01 Jan 0001 00:00:00 +0000

LoRA is the workhorse PEFT method, and this guide is where you move past the basics into the options that actually affect results: initialization, rank, target modules, and merging. Pay close attention to weight initialization, since PEFT defaults to an identity transform and the init_lora_weights=False option exists only for debugging and must never be used in real training. A common mistake is targeting the wrong modules or setting rank without regard to the task, which wastes the method’s efficiency. Read the PEFT overview first if LoRA as a concept is new; this is the practical follow-up.

LoRA methods

Mon, 01 Jan 0001 00:00:00 +0000

Mixed adapter types

Mon, 01 Jan 0001 00:00:00 +0000

Model merging

Mon, 01 Jan 0001 00:00:00 +0000

Orthogonal Finetuning (OFT and BOFT)

Mon, 01 Jan 0001 00:00:00 +0000

PEFT

Mon, 01 Jan 0001 00:00:00 +0000

This is the entry point to PEFT and the right place to grasp why parameter-efficient fine-tuning matters: methods like LoRA train a small set of extra parameters instead of all of them, cutting compute and storage enough to fine-tune large models on consumer hardware. Focus on the menu of method families and on PEFT’s integration with Transformers, Diffusers, and Accelerate, which is what makes it practical. A common misconception is that PEFT sacrifices quality, when in practice it reaches performance comparable to full fine-tuning. Read the quicktour next, then the LoRA developer guide for the method you will actually reach for.

PEFT checkpoint format

Mon, 01 Jan 0001 00:00:00 +0000

PEFT configurations and models

Mon, 01 Jan 0001 00:00:00 +0000

PEFT integrations

Mon, 01 Jan 0001 00:00:00 +0000

Prompt-based methods

Mon, 01 Jan 0001 00:00:00 +0000

Quantization

Mon, 01 Jan 0001 00:00:00 +0000

Quicktour

Mon, 01 Jan 0001 00:00:00 +0000

Soft prompts

Mon, 01 Jan 0001 00:00:00 +0000

torch.compile

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000