Gradio Frontend Without Exposing Model Weights

Introduction

When deploying machine learning models on the web, it’s often essential to keep model weights private, especially when dealing with proprietary or large models such as diffusion generators or segmentation systems. This post shows how I built a Gradio frontend that talks to a secure backend API while hiding the model logic and weights from users.

Why Hide the Weights?

In a typical Gradio app, the model and interface are often hosted together. That creates a few problems:

• Users can inspect model files if the app is exposed publicly
• Heavy models can slow the UI
• It becomes harder to scale compute across separate machines

To solve this, I decoupled the frontend interface from the inference backend.

How It Works

I used a client-server architecture.

Frontend (Gradio UI):

• Users upload an image and select options
• Requests are added to a queue
• The UI checks status periodically until results are ready

Backend (API Server):

• Receives requests via Hugging Face’s gradio_client
• Processes one request at a time, which works well for GPU-heavy tasks
• Returns base64-encoded outputs to the frontend

Queue-Based Request System

Since the GPU backend can only handle one request at a time, I used a queue system to:

• Handle multiple concurrent users
• Provide estimated wait times
• Prevent overload and timeouts

Each request gets a request_id, queue position, and estimated wait time. Once processing finishes, the UI displays the output.

Sample Workflow

1. User uploads a photo and selects preferences.
2. The app encodes the image in base64 and submits it to the backend.
3. The backend runs inference and returns two output images: an overlay and a rendered result.
4. The frontend updates the UI when the job is complete.

Here is a simplified version of the backend API function:

def predict_api(image_b64: str, category: str, gender: str):
    # Actual model inference happens here - hidden from user
    overlay_img = run_segmentation_model(image_b64)
    bg_img = generate_background(category, gender)
    return image_b64, to_b64(overlay_img), to_b64(bg_img), "Done"

The frontend polls request_id every 2 seconds until completion.

Frontend Polling Pattern

import gradio as gr
import requests
import time

def submit_and_poll(image, category, gender):
    resp = requests.post(BACKEND_URL + "/queue", json={
        "image": encode_b64(image),
        "category": category,
        "gender": gender
    })
    request_id = resp.json()["request_id"]

    while True:
        status = requests.get(f"{BACKEND_URL}/status/{request_id}").json()
        if status["done"]:
            return status["overlay"], status["background"]
        time.sleep(2)

demo = gr.Interface(
    fn=submit_and_poll,
    inputs=[gr.Image(), gr.Dropdown(["casual", "formal"]), gr.Radio(["male", "female"])],
    outputs=[gr.Image(label="Overlay"), gr.Image(label="Result")]
)
demo.launch()

Benefits of This Setup

Security: The model weights and inference logic stay hidden on the server.

Scalability: The backend can move to a stronger GPU machine without changing the frontend.

Modularity: Models can be swapped or upgraded without changing the public UI.

Queue Control: Heavy workloads can be scheduled more safely without crashing the service.

When to Use This Pattern

This setup is a strong fit for:

• Commercial AI tools where model IP must stay protected
• Diffusion or segmentation models that are too large to host alongside the UI
• Multi-tenant deployments that need rate limiting
• Protected ML workflows in enterprise environments

Conclusion

By separating the Gradio interface from the model backend and implementing a queue, I built a flow that is more secure, more scalable, and easier to operate. The key idea is simple: Gradio does not need to be monolithic. It can work as a lightweight frontend in front of a hardened inference service.