10.29 LORA & ControlNet

Post Reply
glegrady
Posts: 266
Joined: Wed Sep 22, 2010 12:26 pm

10.29 LORA & ControlNet

Post by glegrady » Sat Sep 19, 2026 5:12 pm

10.29 LORA & ControlNet

LoRA (Low-Rank Adaptation) changes what the model knows. It’s a small set of extra weights trained on a handful of images (often 10–50) to teach the model a new subject, style, or concept, such as a particular car, a person’s face, or a painter’s brushwork. It works by adding small low-rank matrices to the model’s attention layers, so the file is tiny (often 10–200 MB) compared with a full checkpoint. At generation time you load it and set a strength. It shapes appearance and identity, but it has no say over where things go in the frame.

ControlNet changes how the image is laid out. It’s a parallel network that takes a structural guide image at generation time and forces the output to follow it. The guide can be an edge map (Canny), a depth map, a human pose skeleton (OpenPose), line art, segmentation, or a scribble. You don’t train it yourself. You pick a pretrained ControlNet for the kind of guide you have. It fixes composition, pose, and geometry, but it doesn’t teach the model any new subject or style.

Screenshot 2026-10-13 at 3.12.11 PM.png
George Legrady
legrady@mat.ucsb.edu

Post Reply