add lora adapter

[Shubin-vadim]

What does this PR do?

This PR adds comprehensive LoRA (Low-Rank Adaptation) integration to the Open-Unlearning project, enabling efficient parameter-efficient fine-tuning and unlearning capabilities.

Key Features Added

1. LoRA Model Integration

• New LoRAModelForCausalLM class in src/model/lora.py for loading models with LoRA adapters
• Enhanced model loading in src/model/__init__.py with automatic LoRA detection via use_lora: true configuration
• Model registry integration allowing seamless switching between standard and LoRA models

2. Configuration Support

• Model configurations for LoRA-enabled models:
  • Qwen2.5-3B-Instruct-lora
  • Llama-2-7b-hf-lora
  • Llama-2-7b-chat-hf-lora
• Experiment configurations for fine-tuning and unlearning with LoRA:
  • TOFU dataset experiments
  • MUSE dataset experiments
  • WMDP dataset experiments

3. Community Method Implementation

• Complete LoRA method in community/methods/LoRA/ with:
  • Comprehensive README documentation
  • Execution script (run.sh)
  • Usage examples and best practices

4. Documentation Updates

• Enhanced components documentation explaining LoRA integration
• Updated links documentation with relevant LoRA resources
• Dependency updates adding peft==0.17.1 requirement

Technical Implementation

LoRA Configuration

lora_config:
  target_modules: ["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "down_proj", "up_proj", "lm_head"]
  lora_alpha: 128
  lora_dropout: 0.05
  r: 128
  bias: "none"
  task_type: "CAUSAL_LM"

Key Benefits

• Memory Efficiency: Only trains ~1% of model parameters
• Faster Training: Reduced computational requirements with higher learning rates
• Modularity: Easy switching between different LoRA configurations
• Automatic Device Placement: Uses device_map: "auto" for optimal performance
• No Authentication Required: Works without HuggingFace tokens

Usage Examples

# Fine-tuning with LoRA
python src/train.py --config-name=train @experiment=finetune/tofu/lora

# Unlearning with LoRA  
python src/train.py --config-name=unlearn @experiment=unlearn/tofu/lora

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Have you gone through the contributions guide?
• Are your changes documented? Read documentation guidelines here.

Testing

• All code formatting checks pass (make quality)
• LoRA integration tested with multiple model configurations
• Documentation updated and comprehensive
• Configuration files validated for all supported datasets

References

• LoRA: Low-Rank Adaptation of Large Language Models
• Parameter-Efficient Fine-Tuning (PEFT) Library