Tutorial: Train SDXL LoRAs on DiffusionHub Koby, January 19, 2024March 15, 2024 Learn how to train SDXL LoRA models using DiffusionHub.io and bring your creative visions to life. Table of contents: Getting familiar with Khoya SS GUI Preparing your training images Navigating file browser and folder preparations. Dataset preparation SDXL training parameters Captioning your images. Upload and Use your trained LORAs in DiffusionHub Automatic 1111 1. Getting familiar with Khoya SS GUI Upon launching DiffusionHub, Choose Kohya instance in the dashboard. Familiarize yourself with the graphical user interface (GUI) to seamlessly navigate through the training process. Note: For long training sessions, it is recommended to disable AFK protection. By default, AFK protection is enabled and it will terminate the session if no activity is found for 30 mins. (1) click on LORA tab to if you aim to train SDXL LoRA. (2) Select the model you want to train with, we recommend SDXL base model(stable-diffusion-xl-base-1.0) which can be selected with a drop-down list. For any other model, you can select “custom” and then paste in the location of your model. (3) Select the file type you want to train your LORAs on (Safetensors or CKPT). We recommend you use safetensors. (4) Clicking on 4 will take you to the DiffusionHub file explorer, where you will set up your folders. (5) Last is the logger, which will show you what’s going on in the background while you work in GUI. 2. Prepare your training images More is better? Not necessarily In our testing, we found that high-quality images give better results. Training images can be as low as 15, the quality of images has a huge impact on your results. SDXL supports training with high-resolution images (1024 X1024) Variations in aspect ratios among images are acceptable. Enable the bucketing option during training to categorize images with different aspect ratios effectively. Ensure that all selected images are either in PNG or JPEG format. If you are training a person, you better use more close-up photos/upper body photos and around 30% of your images should be full-body, if you want to train the full body. Avoid below a. Blurry b. Similar backgrounds c. Duplicate images Cropping is not required but can give good results if the images have the same aspect ratio, you can use a free bulk image cropper (https://www.birme.net) to resize your images: 3. Navigating file browser and folder preparations. Once you click on File Explorer, you will see two folders “output” and “data” Click on the output folder (6) Once you are inside the output folder, click on “add folder” to add three folders example, Output, Train_images and Regularization_images. Note : Names can be as you wish. (7) Output folder: This folder is where the kohya SS will prepare its folder structure. (8) Train_images : This will contain the images you want to train LORA on. Upload your training images into this folder. (9)Regularization images (Optional): Will contain the regularization images, upload your regularization images into this folder. it is recommended to have your regularization images in the same aspect ratios as your training images. DiffusionHub offers a free 30-minute trial, which is a fantastic way to explore the platform’s capabilities without any commitment. This trial period allows you to familiarize yourself with the user interface, experiment with different models, and understand the potential of Stable Diffusion technology in image generation 4. Dataset preparations for training Click on the “Dataset preparation” Tab and you will see below options (10) Instance prompt: This is the unique identifier of your subject you want to train to make sure it is unique, we use “ohwx”, this will be used in your prompts to activate your LORA. (11) Class: Will define the subject you want train, Ex: Man, Woman, Car, etc. (12) Training images: This is where you will put the path of the folder where you have uploaded your training images in this case it is /output/Train_images you can copy the folder path with the below icon showing on all directory. Note: make sure you remove “/dh-user” from the copied path. (13) Repeats: Will define how many times your training images will be used for training the LORA ( Aim for around 1600+ steps, so if I am using 15 training images make this 100) for 30 training images 55. (14) Regularization images (Optional): Put the path of the regularization folder where you have put in the regularization images. You can skip this if you don’t want to use regularization images for training, ( these images are used to prevent overfitting and improve generalization performance). (15) Repeats for Regularization images: Keep it 1 (16) The Destination training directory will be the path where Kohya SS will put its folder structure. In this case, it is /output/Output. (17) Click on prepare training data: This will prepare the folder structure as per the inputs we have given. Click on the logger and you should see that the folder structure is created. You should see 4 folders inside the output/Output folder: reg, img, log and model. reg: contains your regularization images img: contains your training images log: training logs are collected model: this will contain your SDXL Lora model after training. (18) Click on copy info to folder tab: This will copy the Kohya SS folder structure to the folders tab automatically. Name Your LORA: in the Model Output name, put your customized LORA name. 5. SDXL training parameters If you are following along we recommend you train your LORAs as per the below training parameters and customize them as per your needs. (19) LR Scheduler: COSINE and CONSTANT gave us the best results. (20) Learing rate: 0.0004 (21) Optimizer: Adafactor (22) Optimizer extra Arguments: “ scale_parameter=False relative_step=False warmup_init=False “ (23) Max resolution: 1024,1024 (24) Text encoder learning rate: 0.0004 (25) Unet learning rate: 0.0004 (26) Network Rank (Dimension): 32 or 16 (with a higher number your LORA file size will increase) Click on advanced. (28) Enable Gradient checkpointing 6. Captioning your images (Optional) This will help SDXL to understand and train your images better. (29) Click on “Utilities” tab and Select BLIP captioning, select the images folder in this case “/output/Output/img/55_Ohwx_woman“. (30) Prefix to add to BLIP caption: use your instance prompt in this case “ohwx” (31) Click caption images to generate captions for your images. You should see txt files corresponding to each image which will contain the captions for each image, to see the caption you will have to download these txt files. If you need to modify the captions as per your requirements then you will have to download them and edit and re-upload. (32) Go back to LORA tab and click start training to start training your LORA Keep checking your logs to see the progress of your training. Note: Sometimes the logs will be frozen, don’t panic the training will be still in progress, you can click start training again multiple times to see the logs. Once the training is done you will see the above logs on the logger and your model can be downloaded to your local machine from the folder /output/Output/model/<yourLORAname> 6. Upload and Use your trained LORAs in DiffusionHub Automatic 1111 You can upload the LORAs you created to Automatic 1111 in two ways. Faster/Recommended : Use “Batchlinks downloader” extension (https://github.com/etherealxx/batchlinks-webui.git). If you see batchlinks downloader in GUI then proceed to the next step, if not then you can download it as explained below. Go to Extensions > Install from URL > put the git URL for Batch link downloader https://github.com/etherealxx/batchlinks-webui.git > click install and once the installation is complete, reload the UI. Upload your LORA on any of your platforms, Gdrive ,Hugging face etc.. And use the Batchlinks downloader extension to download any model or LORA using your link. For the current training demonstration put (as shown below) #lora and the LORA link, which will send the LORA file directly to Automatic 1111 LORA directory. Another way (Slower compared to the last method ) is to upload your LORAs to the Automatic 1111 file explorer manually from your local machine to the LORA folder in Automatic 1111 file Explorer. The path to the LORA folder is /data/models/Lora , click upload files and upload your LORA. After refreshing you should also see LORA in your Auto 1111 GUI NOW YOU ARE ALL SET TO TEST YOUR LORA! Here are some of the results we were able to get from our training With a user-friendly interface and a free 30-minute trial, DiffusionHub is the perfect place to start your journey into the future of visual content creation. Embrace the potential of AI image generation technology with DiffusionHub today! Share on FacebookPost on XFollow usSave Stable Diffusion LoRASDXL
Generative AI Guide to Stable Diffusion Models in 2024 January 21, 2024March 4, 2024 Introduction: Stable Diffusion Models, also known as checkpoint models, are pre-trained weights designed for generating specific styles of images. The type of images a model can generate is determined by the images used during its training phase. For instance, a model trained without any cat images will not be able… Read More
Generative AI How does Stable Diffusion AI model work? October 21, 2023March 5, 2024 The Stable Diffusion AI model is a cutting-edge text-to-image generation tool that has captivated the imagination of artists, developers, and technologists alike. At its core, Stable Diffusion is a deep learning model that translates textual descriptions into detailed images, effectively turning words into visual art. This article will delve into… Read More
Generative AI Exploring the Boundless Creativity with DreamShaper XL January 14, 2024March 15, 2024 DreamShaper XL is a revolutionary image creation tool that uses advanced AI technology to produce incredible images with various styles and themes. It allows users to recreate existing images or generate completely unique visuals, offering a seamless and intuitive experience. But how does DreamShaper XL differ from DreamShaper on SD… Read More