10.29 LORA & ControlNet
LoRA (Low-Rank Adaptation) changes what the model knows. It’s a small set of extra weights trained on a handful of images (often 10–50) to teach the model a new subject, style, or concept, such as a particular car, a person’s face, or a painter’s brushwork. It works by adding small low-rank matrices to the model’s attention layers, so the file is tiny (often 10–200 MB) compared with a full checkpoint. At generation time you load it and set a strength. It shapes appearance and identity, but it has no say over where things go in the frame.
ControlNet changes how the image is laid out. It’s a parallel network that takes a structural guide image at generation time and forces the output to follow it. The guide can be an edge map (Canny), a depth map, a human pose skeleton (OpenPose), line art, segmentation, or a scribble. You don’t train it yourself. You pick a pretrained ControlNet for the kind of guide you have. It fixes composition, pose, and geometry, but it doesn’t teach the model any new subject or style.
10.29 LORA & ControlNet
10.29 LORA & ControlNet
Post by glegrady » Sat Sep 19, 2026 5:12 pm
George Legrady
legrady@mat.ucsb.edu
legrady@mat.ucsb.edu
Jump to
- Courses
- ↳ MAT 594CM
- ↳ Spring 2010
- ↳ Winter 2011
- ↳ MAT 200A
- ↳ Fall 2010
- ↳ Fall 2024
- ↳ Fall 2025
- ↳ Fall 2026
- ↳ Art 130
- ↳ Fall 2010
- ↳ Winter 2012
- ↳ Fall 2012
- ↳ MAT 201B
- ↳ Fall 2010
- ↳ MAT 259
- ↳ Winter 2011
- ↳ Winter 2012
- ↳ Winter 2013
- ↳ Winter 2014
- ↳ Winter 2015
- ↳ Winter 2016
- ↳ Winter 2017
- ↳ Winter 2018
- ↳ Winter 2019
- ↳ Winter 2020
- ↳ Winter 2021
- ↳ Winter 2022
- ↳ Winter 2023
- ↳ Winter 2024
- ↳ ART 102
- ↳ Winter 2011
- ↳ Spring 2012
- ↳ Winter 2013
- ↳ NMDS 551
- ↳ MAT 594CR
- ↳ Spring 2012
- ↳ MAT 254
- ↳ Fall 2013
- ↳ Fall 2014
- ↳ MAT 265
- ↳ Spring 2016
- ↳ Fall 2022
- ↳ Winter 2023
- ↳ Fall 2025
- ↳ Spring 2026
- ↳ ART 185GL
- ↳ Spring 2017
- ↳ Spring 2018
- ↳ MAT 594G
- ↳ Fall 2020
- ↳ MAT 255
- ↳ Spring 2021
- ↳ Spring 2022
- ↳ Fall 2022
- ↳ Fall 2023
- ↳ Fall 2024
- ↳ Fall 2025
- ↳ Winter 2026
- ↳ Fall 2026