Protocol-driven inline media for agent-generated images (and other element types) during chat streaming. Any backend that implements BaseProvider can emit images without PraisonAI-specific code in chat or the frontend.
BaseProvider
Before this feature, PraisonAIUI could render images (Message.add_image, ElementRenderer) but the wire protocol dropped media during agent runs. ImageAgent / tool results appeared as JSON in the tool-call panel instead of inline <img> previews.
Message.add_image
ElementRenderer
<img>
Modular layers — same pattern as A2UI surfaces:
| Layer | Module | Backend-agnostic? |
|-------|--------|-------------------|
| Detection | media_utils.py | Yes — normalises OpenAI, URL, b64, element dicts |
media_utils.py
| Protocol | provider.py — RunEventType.MESSAGE_ELEMENT | Yes |
provider.py
RunEventType.MESSAGE_ELEMENT
| Adapter | providers/__init__.py — PraisonAI bridge only | No — maps SDK/callback queue → protocol |
providers/__init__.py
| Transport | chat.py — broadcast + persist | Yes |
chat.py
| Storage | attachments.py — GET /api/chat/media/{id} | Yes — reuses attachment store for b64 |
attachments.py
GET /api/chat/media/{id}
| UI | chat.js, ElementRenderer | Yes — renders {type, url, alt} |
chat.js
{type, url, alt}
Detection and UI do not import PraisonAI. Only the provider adapter knows ImageAgent / OpenAI {data: [{url}]} shapes.
{data: [{url}]}
Defined in Python as RunEventType.MESSAGE_ELEMENT and mirrored in types.ts.
types.ts
Element shape reuses the existing message element schema (image, pdf, video, audio, file, code).
image
pdf
video
audio
file
code
Example app: examples/python/30-image-preview/app.py.
examples/python/30-image-preview/app.py
PraisonAIProvider maps queue events (image, message + elements) → MESSAGE_ELEMENT.
PraisonAIProvider
message
elements
MESSAGE_ELEMENT
When a tool completes, tool_completed_extra() in a2ui_utils.py calls build_media_extra() from media_utils. Supported shapes:
tool_completed_extra()
a2ui_utils.py
build_media_extra()
media_utils
| Input shape | Example |
|-------------|---------|
| OpenAI / ImageAgent | {"data": [{"url": "...", "b64_json": "...", "revised_prompt": "..."}]} |
{"data": [{"url": "...", "b64_json": "...", "revised_prompt": "..."}]}
| Explicit element | {"type": "image", "url": "..."} |
{"type": "image", "url": "..."}
| Plain URL string | "https://cdn.example.com/out.png" |
"https://cdn.example.com/out.png"
| Data URL | "data:image/png;base64,..." |
"data:image/png;base64,..."
| Nested elements | {"elements": [{"type": "video", "url": "..."}]} |
{"elements": [{"type": "video", "url": "..."}]}
chat.py broadcasts message_element for each detected element and attaches elements to the persisted assistant message.
message_element
If the registered agent's chat() returns an OpenAI-shaped dict, PraisonAIProvider._run_direct_mode runs extract_media_elements() and emits MESSAGE_ELEMENT events before RUN_COMPLETED.
chat()
PraisonAIProvider._run_direct_mode
extract_media_elements()
RUN_COMPLETED
Large b64_json payloads are not stored inline in message history. They are decoded and saved via AttachmentManager, then served at:
b64_json
AttachmentManager
If storage fails, the system falls back to a data:image/png;base64,... URL.
data:image/png;base64,...
| Surface | Handler |
|---------|---------|
| Dashboard chat (template) | chat.js — case 'message_element': appendMediaElement() |
case 'message_element': appendMediaElement()
| React chat | streamingStore.ts — pendingElements + ElementRenderer |
streamingStore.ts
pendingElements
| Tool call panel | Thumbnail from toolCall.elements when expanded |
toolCall.elements
| Session reload | elements on messages from /api/chat/history and /sessions/{id}/runs |
/api/chat/history
/sessions/{id}/runs
See also Message Elements for element properties and display modes.
_collect_element() in chat.py dedupes by URL so the same image is not stored twice when both a MESSAGE_ELEMENT event and a tool completion carry the same URL.
_collect_element()
POST /v1/images/generations
Coverage includes: detection normalisation, b64 fallback, provider callback bridge, _run_and_broadcast persistence, media serve route, ImageAgent dict responses, URL dedup.
_run_and_broadcast