Komal Kumar

Thanks for your reply, I have added PR here https://github.com/huggingface/peft/pull/3342. Please feel free to provide your suggestions.

Thanks for analysis. Very helpful.

What about DEFT?
It adapts a pre-trained weight matrix by decomposing its update into two components with two trainable matrices: (1) a projection onto the complement of a low-rank subspace spanned by a low-rank matrix, and (2) a low-rank update. The single trainable low-rank matrix defines the subspace, while the other trainable low-rank matrix enables flexible parameter adaptation within that subspace.

GitHub https://github.com/MAXNORM8650/DEFT

Paper: https://arxiv.org/abs/2509.22793