Mastering ComfyUI: Setup, Models, and Workflow Management

ComfyUI is a cost-free application designed for generating AI images and videos locally on your device. It caters to users seeking greater control than standard prompt-and-generate tools, offering visibility and adjustability over every stage of the generation process. While this flexibility introduces a learning curve, this guide will assist you in getting started, covering everything from installation to executing and customizing your first workflow.

Prerequisites for Getting Started

Running ComfyUI requires a computer equipped with sufficient GPU resources to handle the intended models and workflows. More complex workflows and larger models typically demand higher VRAM.

You will also need the specific model files utilized by your workflow. Depending on the model type, this may involve checkpoints, diffusion models, VAEs, text encoders, LoRAs, or other components. These files are generally stored within the ComfyUI/models directory.

If your local GPU is insufficient, you can run ComfyUI on a remote GPU-powered desktop. This approach offloads the generation workload to the remote hardware while allowing you to access the interface from your regular computer.

Installing ComfyUI

For users on Windows and macOS, ComfyUI recommends its desktop application for the most seamless experience. Alternative installation methods include manual setup and the use of the ComfyUI command-line tool. The best method depends on your specific operating system and environment.

Once ComfyUI is installed, launch the application to access the interface. You will find the workflow canvas along with the tools necessary to create and manage your projects.

The Importance of ComfyUI Workflows

A ComfyUI workflow dictates the generation process for an image or video. It manages the models, settings, and processing steps required to produce the final output.

This level of detail provides significantly more control than a simple prompt box. You have the ability to swap models, integrate a LoRA, utilize input images, tweak generation parameters, upscale results, or insert additional processing stages.

Workflows are also designed for saving and reuse. Rather than reconstructing the same setup repeatedly, you can retain a workflow that yields satisfactory results and modify specific settings as needed. Additionally, you can download and adapt workflows created by others to fit your own configuration.

Understanding ComfyUI Workflows

A workflow consists of interconnected nodes. Each node manages a specific aspect of the generation process, with connections determining the flow of information between them.

A standard text-to-image workflow typically includes nodes for loading the model, inputting the prompt, generating initial image data, running the generation, decoding the result, and saving the final file.

• Model loader: Loads the specific model required for generation.
• Text encoder: Transforms your prompt into data interpretable by the model.
• Sampler: Executes the generation process based on the selected parameters.
• VAE: Facilitates conversion between latent data and pixel data.
• Save Image: Writes the generated image to your storage device.

There is no need to construct every workflow from scratch. ComfyUI offers ready-made templates, and numerous community workflows are available for download and immediate use.

Loading Existing Workflows

Starting with an existing workflow is often the most efficient approach. ComfyUI includes sample workflows for various models and tasks, and community platforms provide an even wider selection.

Many workflow images embed the workflow data within their metadata. You can simply drag the image into the ComfyUI interface or use the Workflows → Open option to load it. The workflow will then appear on the canvas with all nodes and settings pre-configured.

After loading the workflow, verify the expected models. If files are missing, ComfyUI can identify missing models for supported templates. For other workflows, you may need to locate and install the required models independently.

Sourcing Models for ComfyUI

Models can be found on repositories like Hugging Face and Civitai, as well as on the official project pages for specific models. The key is to ensure the model is compatible with your intended workflow.

Do not assume that any model file is universally compatible with any workflow. Different model architectures may necessitate specific loaders and supporting files.

Before downloading a model, verify the following:

• The model's architecture and version
• The required ComfyUI workflow compatibility
• The model file format
• Recommended VRAM and hardware specifications
• Any required VAEs, text encoders, LoRAs, or other supporting files
• The model's licensing and usage restrictions

ComfyUI supports various model file types, with the destination folder depending on the model type. For instance, checkpoints belong in models/checkpoints, LoRAs in models/loras, and VAEs in models/vae. Some newer models utilize folders such as models/diffusion_models and models/text_encoders.

Installing a Model

Once you have downloaded a model, place it in the directory expected by the workflow. You can then select it within the corresponding model loader.

For example, a checkpoint might be stored in:

ComfyUI/models/checkpoints/

A LoRA might instead be stored in:

ComfyUI/models/loras/

If the new model does not appear in the selection list, refresh the interface or restart ComfyUI.

Installing Custom Nodes

Many advanced ComfyUI workflows rely on custom nodes that are not part of the standard installation. If these dependencies are missing, the workflow may indicate missing nodes.

ComfyUI includes a Manager for installing custom nodes. Alternatively, nodes can be installed manually by placing their repositories in the custom_nodes directory and installing their required dependencies.

Only install custom nodes from trusted sources. Since custom nodes may contain executable code and have specific dependencies, security considerations must be taken into account.

Running and Modifying Your Workflow

Once the necessary models and custom nodes are installed, review the critical settings in the workflow. Begin by checking the model, prompt, image dimensions, and sampling parameters.

When all elements are in place, use the Queue button to execute the workflow. ComfyUI will process each step and generate the output defined by the configuration.

You can then adjust individual components of the workflow without rebuilding it entirely. Add a LoRA, connect an input image, switch the sampler, insert an upscaler, or fine-tune other settings to alter the result.

Saving Your Workflows

Save workflows that you plan to reuse. While a workflow contains the node graph and its settings, it does not necessarily include the model files themselves. It is crucial to track which models and custom nodes are required for the workflow.

This is particularly important when migrating a workflow to another computer or cloud desktop. You may need to install the identical models and custom nodes before the workflow will function correctly.

Trying ComfyUI on DaDesktop

There is no need to purchase a new GPU solely to run ComfyUI. If your current computer lacks sufficient GPU resources, you can run ComfyUI on a cloud desktop instead, utilizing it as needed.

DaDesktop offers cloud desktops with dedicated GPU resources, ideal for workloads such as AI image and video generation. You can install ComfyUI, download your preferred models, and develop custom workflows without adding a dedicated GPU to your local machine.

Learn more about AI image and video generation on DaDesktop. You can also view the available GPUs and select a configuration suited to the models and workflows you intend to run.