ONROL — AI Execution School

    — Techniques

    LoRA (Low-Rank Adaptation)

    An efficient way to fine-tune large models by training only a small set of new parameters.

    What is LoRA (Low-Rank Adaptation)?

    LoRA (Low-Rank Adaptation) is a fine-tuning technique that adds small trainable layers to a frozen base model. Result: 100x cheaper than full fine-tuning, with comparable quality for most tasks. The standard technique for fine-tuning open-source models like Llama. ONROL covers LoRA in the Architect track.

    — Related

    Terms connected to LoRA (Low-Rank Adaptation)

    Techniques

    Fine-Tuning

    Adjusting a pre-trained AI model on your specific data to change its behaviour.

    Open →

    Techniques

    RAG (Retrieval-Augmented Generation)

    Giving an AI access to your private documents so it can answer questions about them.

    Open →