ControlNet in practice: canny, depth and openpose, and which to choose

ControlNet (Zhang, Rao and Agrawala, 2023) adds a second input to the model: a map that the image must follow. The prompt still decides content and style. Source and models: https://github.com/lllyasviel/ControlNet Which map - Canny: hard edges traced from a reference. Keeps outlines and fine linework. Best for line art, architecture, and remaking a painting or relief. - Depth: a near/far map. Keeps spatial layout and volume while letting surfaces and outlines change. Best for "same room, different style". - Openpose: a stick-figure skeleton (body, optionally hands and face). Keeps the pose and nothing else. Best for putting a new character in a known pose. - Lineart, softedge, scribble: gentler cousins of canny when the reference is a sketch. Settings that matter - Control weight: 1.0 is strict. 0.6-0.8 lets the model fix awkward details. - Start/end: ending control at about 0.6-0.8 of the steps lets the final steps add natural detail instead of copying the map's artefacts. - Preprocessor resolution: match your render size, or thin lines vanish. - Match the ControlNet to the base model. SD 1.5 ControlNets do not work on SDXL. How the Studio uses it Hathor Studio's layout reference is ControlNet: the outlines of a source picture are kept while everything else is re-rendered. For ancient art the source is smoothed first so cracks and brush strokes do not come through as edges. See https://wiki.soapbox.community/wiki/Remakes Reproduce it Post the reference, the control map the preprocessor produced, and the result. The map is the part people skip, and it is usually where the problem is.