10:01 Text to Image | Prompt Exploration

[glegrady]

10:01 Text to Image | Prompt Exploration

This assignment explores studies in creating images with text prompts using this attached json or the image. To use the json, copy the text into a text file, and change the ending to .json.

Code: Select all

{
  "1": {
    "inputs": {
      "ckpt_name": "flux1-schnell-fp8.safetensors"
    },
    "class_type": "CheckpointLoaderSimple",
    "_meta": {
      "title": "Load Checkpoint"
    }
  },
  "2": {
    "inputs": {
      "text": "an irregular, asymmetric, natural shaped, rock that is transparent and glows from within overall",
      "clip": [
        "1",
        1
      ]
    },
    "class_type": "CLIPTextEncode",
    "_meta": {
      "title": "CLIP Text Encode (Prompt)"
    }
  },
  "3": {
    "inputs": {
      "text": "watermark, text",
      "clip": [
        "1",
        1
      ]
    },
    "class_type": "CLIPTextEncode",
    "_meta": {
      "title": "CLIP Text Encode (Prompt)"
    }
  },
  "4": {
    "inputs": {
      "width": 1920,
      "height": 1080,
      "batch_size": 1
    },
    "class_type": "EmptyLatentImage",
    "_meta": {
      "title": "Empty Latent Image"
    }
  },
  "5": {
    "inputs": {
      "seed": 0,
      "steps": 20,
      "cfg": 7,
      "sampler_name": "euler",
      "scheduler": "normal",
      "denoise": 1,
      "model": [
        "1",
        0
      ],
      "positive": [
        "2",
        0
      ],
      "negative": [
        "3",
        0
      ],
      "latent_image": [
        "4",
        0
      ]
    },
    "class_type": "KSampler",
    "_meta": {
      "title": "KSampler"
    }
  },
  "6": {
    "inputs": {
      "samples": [
        "5",
        0
      ],
      "vae": [
        "1",
        2
      ]
    },
    "class_type": "VAEDecode",
    "_meta": {
      "title": "VAE Decode"
    }
  },
  "7": {
    "inputs": {
      "filename_prefix": "txtPrompt",
      "images": [
        "6",
        0
      ]
    },
    "class_type": "SaveImage",
    "_meta": {
      "title": "Save Image"
    }
  }
}

GOALS: The course focuses on technique but within a conceptual framework: Given that the phrasing of the text may be a significant influence on the outcome, it is critical to explore different ways to describe something. There are therefore two goals:

1. Explore various ways to describe something, explore the impact of what unexpected results a text may produce. Ideal texts critically examines the process, what can be achieved, and what new insights we can gain.

2. Try variations in the settings in the various nodes: checkpoints, Ksampler, scale, etc. Once an image has been created, you ocan save the workflow by saving the image, which will include th json data, or by clicking on "Workflow" in the top menu, and select "export".

3. Present 5-10 images resulting from at least 20 tries. Provide a report of your process, and results.

DEADLINES: 1st pass: 9/24, 2nd pass: 10:01

GRADING: Innovation in the following: a) text prompt, b) visual outcome, c) unusual setting configurations, d) record your changes in settings.

[glegrady]

Follow-UP to today's class

How to post an assignment in the Student Forum:

Click on "Post Reply" and you get this window. You then type text, and can add an image by pulling it into this space from outside the frame

[ruoxi_du]

I started with the provided workflow and used the original image as my starting point. I first changed the prompt while keeping the settings mostly the same. When I used “fox-eared musician,” I got a woman with an actual fox head. I then added “human face” and “human facial features,” and the human face came back while the fox ears stayed.

Then I tried making the prompt much shorter. I only kept the main things I wanted in the image: the woman, fox ears, guitar, campfire, and Milky Way. These were still there in the result, but a lot of the smaller details were gone. The clothing and appearance also became more general. With less description, the model had more freedom to decide these details.

I also tried describing the character as “part woman and part fox.” I thought it might combine them into one character, but instead it made a woman and a separate fox. I didn’t expect the prompt to be interpreted this way.

I also tested a few settings. With only one sampling step, I could already see the basic composition, but everything was blurry and there was almost no detail. More steps made it much clearer, but going from 8 to 13 steps didn’t change much. I also changed the sampler, but I couldn’t see a big difference.

For the last one, I tried a more abstract prompt about the woman and fox dissolving into sound, fire, and the night sky. This changed the image a lot. The woman and the background started to blend into abstract shapes and textures, but the fox was still pretty clear. Compared with changing the settings, changing the way I wrote the prompt gave me much bigger and more unexpected changes.