使用 Gradio 添加 UI¶

Gradio 是一个开源 Python 库，允许开发者快速为 AI 模型构建基于 Web 的用户界面 (UI)。BentoML 提供了一个简单的 API 来集成 Gradio，以便通过其 UI 服务模型。

先决条件¶

此集成需要 FastAPI 和 Gradio。使用 pip 安装它们。

pip install fastapi gradio

基本用法¶

按照以下步骤将 Gradio 与 BentoML 服务集成。

首先准备一个 BentoML 服务。这是一个使用来自Hello world 指南的文本摘要模型的示例

import bentoml
import torch
from transformers import pipeline

EXAMPLE_INPUT = "Breaking News: In an astonishing turn of events, the small \
town of Willow Creek has been taken by storm as local resident Jerry Thompson's cat, \
Whiskers, performed what witnesses are calling a 'miraculous and gravity-defying leap.' \
Eyewitnesses report that Whiskers, an otherwise unremarkable tabby cat, jumped \
a record-breaking 20 feet into the air to catch a fly. The event, which took \
place in Thompson's backyard, is now being investigated by scientists for potential \
breaches in the laws of physics. Local authorities are considering a town festival \
to celebrate what is being hailed as 'The Leap of the Century."


@bentoml.service(resources={"cpu": "4"})
class Summarization:
    def __init__(self) -> None:
        device = "cuda" if torch.cuda.is_available() else "cpu"
        self.pipeline = pipeline("summarization", device=device)

    @bentoml.api(batchable=True)
    def summarize(self, texts: list[str]) -> list[str]:
        results = self.pipeline(texts)
        return [item["summary_text"] for item in results]

定义一个辅助函数来为服务创建 Gradio UI。它通过 get_current_service() 获取当前的服务实例，并调用其暴露的 API 方法。

def summarize_text(text: str) -> str:
    svc_instance = bentoml.get_current_service()
    return svc_instance.summarize([text])[0]

通过指定要包装的函数并定义输入和输出组件来设置 Gradio 接口。更多信息请参见Gradio 文档。

import gradio as gr

def summarize_text(text: str) -> str:
    svc_instance = bentoml.get_current_service()
    return svc_instance.summarize([text])[0]

# Configure a Gradio UI
io = gr.Interface(
    fn=summarize_text, # Wrap the UI around the function defined above
    inputs=[gr.Textbox(lines=5, label="Enter Text", value=EXAMPLE_INPUT)],
    outputs=[gr.Textbox(label="Summary Text")],
    title="Summarization",
    description="Enter text to get summarized text.",
)

@bentoml.service(resources={"cpu": "4"})
class Summarization:
      ...

使用 @bentoml.gradio.mount_gradio_app 装饰器将 Gradio UI (io) 挂载到自定义路径 (/ui)。这使得它作为服务 Web 服务器的一部分可访问
```
...

@bentoml.service(resources={"cpu": "4"})
@bentoml.gradio.mount_gradio_app(io, path="/ui")
class Summarization:
      ...
```
使用 bentoml serve 启动服务，并通过 https://:3000/ui 访问 Gradio UI。您还可以通过 https://:3000/ 调用 BentoML 的 API 端点 summarize。
```
bentoml serve
```

访问此示例以查看完整的演示代码。