Stable Diffusion img2img API: Creating Images from Reference Images

Leveraging Stable Diffusion img2img API for Image Generation

Vaporwave 50s Woman by GenerativeLabs

In my previous blog post (RunPod Custom Serverless Deployment of Stable Diffusion), I shared my journey and lessons learned with RunPod's custom serverless deployment. Here's the video tutorial describing how to get started with StableDiffusion on RunPod Serverless.

Building upon that video & blog post, I want to delve deeper into the usage of the Stable Diffusion img2img API.

Specifically, I will discuss how to modify the handler python file to leverage the API for generating images from reference images.

This modification allows developers to choose between the img2img and txt2img handlers, providing greater flexibility and control over the content generation process.

Default Python Handler

The default handler provided by RunPod will look something like this:

def run_inference(request):
    response = automatic_session.post(url='http://127.0.0.1:3000/sdapi/v1/txt2img', json=request, timeout=600)
    return response.json()

As you can wee, the API is hard coded to call the Standard Diffusion A1111 `txt2img` API.

Modifying the Handler Python

To enable the use of the Stable Diffusion img2img API, we need to make adjustments to the handler python file. I created a dictionary mapping from the API to the API Path. You can easily update it to expose other Stable Diffusion APIs.

🔗 Here's a link to my complete file source on GitHub for full context.

📄 Here's the relevant snippet of modified code:

def run_inference(params):
    config = {
        "baseurl": "http://127.0.0.1:3000",
        "api": {
            "txt2img":  ("POST", "/sdapi/v1/txt2img"),
            "img2img":  ("POST", "/sdapi/v1/img2img"),
            "getModels": ("GET", "/sdapi/v1/sd-models"),
            "getOptions": ("GET", "/sdapi/v1/options"),
            "setOptions": ("POST", "/sdapi/v1/options"),
        },
        "timeout": 600
    }

    api_name = params["api_name"]
    path = None

    if api_name in config["api"]:
        api_config = config["api"][api_name]
    else:
        raise Exception("Method '%s' not yet implemented")

    api_verb = api_config[0]
    api_path = api_config[1]

    response = {}

    if api_verb == "GET":
        response = automatic_session.get(
                url='%s%s' % (config["baseurl"], api_path),
                timeout=config["timeout"])

    if api_verb == "POST":
        response = automatic_session.post(
                url='%s%s' % (config["baseurl"], api_path),
                json=params, 
                timeout=config["timeout"])

    return response.json()

Once you modify the handler, then build your image and deploy to RunPod, you can specify "img2img" as the value for `api_name` param when making API requests body.

You also need to specify your input image as a value to the `init_images` param -- the param is expecting an array of base64 encoded images.

Here's an example body:

Make a POST request to the RunPod serverless deployment with the following parameters:

• api_name: Set this value to "img2img" to use the img2img handler, or txt2img for text to image.

• init_images: An array of base64 encoded images

Additional parameters: Depending on your use case, you may need to provide extra parameters like seed, steps, or others supported by the Stable Diffusion API.

Watch the video for a complete walk through, with examples, etc.

In Conclusion

With the modified handler python file and the Stable Diffusion img2img API, you can now take advantage of reference images to create customized and context-aware image generation apps.

You can experiment further and update the config object to easily expose other Stable Diffusion APIs.

Happy diffusing.