top of page
Writer's pictureBilly @ Generative Labs

RunPod Custom Serverless Deployment: Lessons Learned

Updated: Jul 10, 2023

My Journey and Lessons Learned with RunPod Custom Serverless Deployment



TLDR: 📚 Key takeaways:

  • 🙌 The RunPod Team is super supportive

  • 📥 Avoid wget for model downloads

  • 🛠️ Avoid auto-build on Docker Hub

  • 💡 Discord for incredible support


I recently embarked on a journey to create my own custom serverless deployment with my own model using RunPod.


Getting Started

I followed RunPod's documentation and wanted to share my experiences. Here's a video tutorial I put together on YouTube documenting my experience and the lessons learned along the way.



The Good

Overall, the process of creating a custom serverless deployment was smooth and enjoyable. The blog post provided clear instructions and guided me through the necessary steps, from cloning the repository to building and pushing the Docker image. I appreciated the simplicity of the tutorial.


The Great

In fact, one of the standout aspects of my journey with RunPod was the incredible responsiveness and support from the RunPod team. 🙏 The team is super dialed into the Discord server, and their helpful and supportive attitude made a significant difference in my experience. Kudos to the RunPod team for fostering such a positive and engaging community. ✨


The Lessons Learned

Overall, I give RunPod a highly positive experience. I did encounter a few challenges that I'd like to document and share here with others who may be facing the same:

  • Downloading Custom Models During Build Time: The Dockerfile in the blog post downloads a custom model using wget during build time. However, members of the RunPod Discord server advised against downloading models on the fly and recommended packaging the model into the container image instead. Serverless doesn't allow workers to run idle for too long, so downloading large files during build time can lead to file corruption.

  • Remote Build with Docker Hub: Don't do it. I use Docker Hub's automated build features to build images remotely, but I consistently encountered the build error message "no space left on device." The cause of this issue remains unclear (Docker support was not helpful), so I had to resort to building the image locally.

  • Resolving Looping Pods: If you encounter an issue where your pods are looping and not completing any jobs, try killing all workers, setting max workers to 0, waiting a moment, and then resetting max workers. This may help.

  • Testing Workers: For testing, set both min and max workers to 1 and reduce the timeout to a reasonable value (e.g., 10 seconds).


Understanding the Different Docker Images

There are a bunch of Stable Diffusion related images on RunPod's Docker Hub. One question I had that's been asked a few times on Discord is-- What's the difference between the "web-automatic-base" and "web-automatic" Docker images?


The primary difference is that the "web-automatic-base" image does not include checkpoints (ckpts), making it smaller in size. It is suitable for users who want to use custom checkpoints. The "web-automatic-base" image also lacks start scripts, so it won't run properly as a template unless you add those.


Here are the Stable Diffusion related images on Docker Hub.


Web Automatic
  • The "web-automatic" Docker image is a comprehensive image that includes both the core functionality for generating content using AI models and a set of pre-trained checkpoints (ckpts). These checkpoints allow users to immediately start generating content without the need to provide their own models. The "web-automatic" image is ideal for users who want a ready-to-use solution with pre-trained models included. Due to the inclusion of checkpoints, this image has a larger size compared to the "web-automatic-base" image.

  • Recent Image Versions: web-automatic-5.0.0, web-automatic-4.0.0, web-automatic-3.0.0, web-automatic-2.1.18, web-automatic-2.1.16, web-automatic-1.5.17, web-automatic-1.5.16, web-automatic-next


Web Automatic Base
  • The "web-automatic-base" Docker image is a streamlined version of the "web-automatic" image. It provides the core functionality and framework for generating content using AI models, but it does not include pre-trained checkpoints (ckpts). This image is suitable for users who plan to use their own checkpoints or who wish to have a smaller image size. It serves as a foundation for building custom AI applications.

  • Recent Image Versions: web-automatic-base-5.0.0, web-automatic-base-4.0.0


Fast Stable Diffusion
  • The "fast-stable-diffusion" Docker image provides access to Fast Stable Diffusion via a Colab notebook that guides users through the process of training and testing their models. The tool is designed for experimentation and creativity, allowing users to train models with various settings and generate unique visual content.

  • Recent Image Versions: fast-stable-diffusion-2.1.0, fast-stable-diffusion-2.0.0, fast-stable-diffusion-1.3.1, fast-stable-diffusion-1.3.0, fast-stable-diffusion-1.2.0, fast-stable-diffusion-1.1.0

  • Resources: RunPod Blog, Colab Notebook


Invoke
  • InvokeAI is a creative engine for Stable Diffusion models that enables users to generate visual media using AI-driven technologies. It is an open source project that offers a suite of image creation tools for both enthusiasts and professionals. Key features of InvokeAI 2.0 include: new web UI, cross platform, RealESRGAN for upscaling, inpainting, outpainting, etc.

  • Recent Image Versions: invoke-2.3.4, invoke-2.3.0

  • Resources: GitHub


Comfy
  • The "comfy" Docker image provides ComfyUI, a powerful and modular stable diffusion graphical user interface (GUI) that allows users to design and execute advanced stable diffusion workflows using a graph, nodes, and flowchart-based interface. It supports both SD1.x and SD2.x models and provides features such as asynchronous queue system, optimizations for efficient execution, support for loading various model checkpoints, textual inversion, area composition, inpainting, ControlNet, T2I-Adapter, upscaling models, unCLIP models, and more.

  • Recent Image Versions: comfy-2.1.0, comfy-2.0.0, comfy-1.0.0

  • Resources: GitHub


Comfy Base
  • The "comfy-base" Docker image is a streamlined version of the "comfy" Docker image, designed to provide the essential functionality of the ComfyUI stable diffusion graphical user interface (GUI) without including additional resources such as pre-trained model checkpoints. This lightweight image is suitable for users who wish to use their own model checkpoints or require a minimal setup for running ComfyUI workflows.

  • Recent Image Versions: comfy-base-2.1.0, comfy-base-2.0.0


In conclusion, my journey with custom serverless deployment was a strong positive. I'm thankful for the RunPod team's outstanding support and the knowledge gained. I hope my insights prove helpful to others exploring serverless technology. 🚀




3,377 views1 comment
bottom of page