Prompts Guide by CDcruz

EDIT: For more up to date and comprehensive information, visit my newly made webpage: https://stablediffusion.cdcruz.com/ There are no ads, just information.

Hello, I've been playing around with Stable Diffusion and Waifu Diffusion almost non-stop for the last few days. It is super fun and facinating to see what this ai can create and what its limits currently are.

With all of my generating, I've learnt a lot about how the prompts work and what works best for desired outcomes. I've also found tips from other people online. I'd like to make this post a summary of everything I've learnt so far to help others create cool art.

For reference, I am using the WebGui that was downloaded following this guide https://rentry.org/voldy

This GUI includes many useful features including batch processing, face restoration, upscaling, inpating, outpainting and more.

This github link provides more detailed information on special features of the webgui https://github.com/AUTOMATIC1111/stable-diffusion-webui-feature-showcase

Sampling Methods

There are quite a few different models to choose from. However, from my testing there isn’t a huge difference in what they produce, except the a versions which look similar to each other.

For me, the Euler A sampling method is the best for both anime and realistic renders. The only down side is that the output changes much more than the other methods depending on sampling steps.

The Euler (not A) method does stay consistant no matter how many steps you render for, but in my limited testing, I feel it creates a image that isn't as refined as Euler A. I'm not sure why this is, or I haven't rendered enough Euler images, but that's how I feel from testing so far. And cranking the steps up on Euler does not improve the image much after about 200 steps. I have done a quick test of the same seed from 10 - 5000 samples and the results were consistant, but not improved by the higher sample steps.

If anyone has better results with a method other than Euler A please let me know. For now, my advice is to stick with Euler A and all of my current advice is mostly based on that method.

Prompt Order

I found this useful video https://www.youtube.com/watch?v=c5dHIz0RyMU about how different things can affect the ai output. One notable affect is the position of prompts. Prompts that are at the front of the list can sway the ai more than ones further down the list.

However there are ways to manipulate the strength of prompts even more.

Using (curly brackets) around words can emphasis it more, more brackets the stronger it is. Opposingly, [square brackets] will make the prompt less impactful.

Another way to do this is by doing prompt:1. I am unsure if the number refers to position or strength, but from testing I believe it means the position of the prompt regardless of actual position in the list. So if I did gold, shoes:1 shoes would be more influential than gold even though it is second in the list.

Styles

There are many ways to create different artistic styles using prompts. Most common is things like Artstation, Dali, photo, painting, etc. As this is a Waifu sub, I assume you are wanting an anime style, It will require more testing, but currently the best working prompt for an anime style I have found is "by Toei Animation", I'm sure other animation companies will work too, but this one has worked best for my needs.

Here's a more detailed list of art/photo styles that I've found to work well:

These were all tested with Waifu Diffusion, Euler A, with each prompt at the beginning of the prompt list, so results will vary a lot if you use Stable Diffusion and different settings.

• Monochome or black and white - massive influence - will make everything black and white

• Sepia - mild influence - will give a sepia colour palette.

• Color/ colour - weak influence - Not really required for more images as they are usually in colour.

• Colourful/ colorful - mild influence - Will add more variety of colour in the image.

• Traditional Art - mild influence - a general art prompt with varying styles and detail.

• Painting - strong influence - will add paintery brush strokes to image.

• Ink art - mild influence - Basically makes black and white drawings.

• Chalk art - mild influence - Sometimes adds a chalk effect to image, sometimes adds random blackboard scribbles to background.

• Oil painting - weak influence - seems to be less powerful than simply using "painting".

• Watercolour/watercolor - strong influence - adds a similar effect as "painting" but does distinctly look more watercolour-like.

• Drawing - mild influence - Will add a more pencil-like style to the image. But seems more digital pencil than real pencil.

• Photo/photograph - strong influence - will make the image much more realistic, but in Waifu Diffusion it will maintain anime characteristics and skin will be smooth.

• Canon 50D (or other camera types) - mild/ strong influence - will make the image more realistic.

• Cinematic lighting - mild/strong influence - will add more dramatic lighting and usually a point light in the scene. Also darker scenes and backlit scenes.

• God Rays - mild/strong influence - will add a strong sun light that will usually shine on one side of a persons head/ hair. Can also add rays of light but less likely.

• Cell Shading - strong influence - a detailed anime style, most similar to visual novel characters.

• Anime - strong influence - Anime

• Waifu - Despite the name of the ai, doesn't seem to change much style wise.

• Key Visual - strong influence - A detailed anime style while keeping the anime characteristics or big eyes, etc.

• Hentai - mild/strong influence - This prompt varies between realistic and anime style depending on other prompts and will also obviously make the image more lewd.

• Artstation - strong influence - A variety of artstyles, usually more professional looking.

• By Toei Animation - strong influence - A better anime prompt for more detailed anime.

• By Studio Ghibli - strong influence - A better anime prompt for simple anime characters and classic Ghibli backgrounds.

• In the style of Mappa - mild influence - Another anime style.

• By Kyoto Animation - mild influence - Anime

• In the style of Cloverworks - mild influence - Anime

• Model - mild influence - Helps a person look more "perfect", they will also usually look directly at the camera.

• Instagram - mild influence - Characters have a much wider range of poses and camera angles

• vtuber - strong influence - will generate anime characters mainly in the style of vtubers.

• Snapchat - weak influence - Similar to "Instagram" but also produces more selfie-like images.

• Professional - weak influence - looks more professional shot and has more background variety butthe background is usually out of focus.

• Ponytail/pigtails/braids/(Various hairstyles) - strong influence - will change hairstyle a lot.

Negative Prompts

Negative prompts are extremely useful. Whatever negative prompt you enter, will be excluded, or reduced in the generating of an image. For example, if I had the prompt "Vehicle" it could show cars, bikes, buses, etc. If I were to add "Buses" to the Negative prompts, the ai will be much less likely to generate a bus image and would instead only generate bikes or cars.

This can be extremely useful for fine-tuning a image, removing disfigured body parts, or reducing the strength of prompts. The prompt order system works for negative prompts as well, but instead, the stronger the prompt, the less likely it will appear and vice versa.

My current default negative prompts list is below:

deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra_limb, ugly, poorly drawn hands, two heads,child, kid, gross, mutilated, disgusting, horrible, scary, evil, old, conjoined

A more controlled image

As stated previously, Euler A can be very unpredictable in its output. On the otherhand, Euler (not A) is incredibly consistant. If you are looking to really finetune an image, Euler (not A) is what you need. I recommend putting in all of your basic settings that you want until you start to see images that represent what you're looking for. Once you've found a good image, lock down the seed and start changing prompts & even try changing the CFG scale to see how it changes the image. Unlike Euler A, it will keep fairly consistent even with different prompts added/ removed. On top of this, you can use the "Extra" section next to "Seed" that adds see variation. This will mix in a different seed to your main seed to create subtle differences while keeping your main seeds image characteristics intact.

Following these steps, you'll be able to create the exact image you're looking for, and if it's still got some problems, move over the inpainting to try and fix those. Or even photoshop could work for finishing touches.

Fixing Issue Areas

There are some body parts that the ai is completely horrible at drawing. Most notably hands, eyes, ears, arms and giantic boobs.

A few ways to combat this is to firstly use negative prompts to remove disfigured body parts. But you can also prefix "perfect" infront of prompts to help the ai.

For eyes, "deep eyes" seems to help it create less errors, although I'm not entirely sure why.

Hands seem to be extremely hard to get right and I am yet to figure out how to render them properly. If anyone has tips for hands, It would be really helpful.

Gigantic boobs seem to appear even when you aren't specifically stating them in your prompt. Even writing "small boobs" or similar may sometimes still create massive honkers. One way to combat this, is to add boobs to your negative prompts, but this can have unintended consequences so I will usually compensate and add similar prompts to the normal prompts that sounds like boobs but isn't, like "curvy".

Sampling Steps

Your natural instinct when generating images might be to max out the sampling steps for the highest quality image possible. However theres a few reasons this might not be the best thing to do.

1. Takes a long time to render.

2. The end result art may be very different compared to if you used a lower sampling step, and the lower step result may be a better result.

3. Especially for anime style art, real life anime art is not very detailed, so making the ai run for a low amount of steps will help immitate that simplistic artstyle better than over-doing it.

4. If you're trying to render longer to remove glitches an errors, (from my testing) I don't believe it will fix much or may even change the image drastically as point 2 mentions. You are better off trying inpainting to fix mistakes.

I have tried extremely high sampling steps (5000) and while the results can be impressive, you can acheive roughly the same results in the 100-500 steps range, and for anime using the Euler A model, you can even get decent results from 20-100 steps.

I hope these tips are helpful, I will continue to update this post as I learn new information. Also please leave comments with your tips and tricks for greating cool art.