chore: initial public snapshot for github upload
This commit is contained in:
Your Name
@@ -0,0 +1,316 @@
|
||||
# LiteLLM Dotprompt Manager
|
||||
|
||||
A powerful prompt management system for LiteLLM that supports [Google's Dotprompt specification](https://google.github.io/dotprompt/getting-started/). This allows you to manage your AI prompts in organized `.prompt` files with YAML frontmatter, Handlebars templating, and full integration with LiteLLM's completion API.
|
||||
|
||||
## Features
|
||||
|
||||
- **📁 File-based prompt management**: Organize prompts in `.prompt` files
|
||||
- **🎯 YAML frontmatter**: Define model, parameters, and schemas in file headers
|
||||
- **🔧 Handlebars templating**: Use `{{variable}}` syntax with Jinja2 backend
|
||||
- **✅ Input validation**: Automatic validation against defined schemas
|
||||
- **🔗 LiteLLM integration**: Works seamlessly with `litellm.completion()`
|
||||
- **💬 Smart message parsing**: Converts prompts to proper chat messages
|
||||
- **⚙️ Parameter extraction**: Automatically applies model settings from prompts
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Create a `.prompt` file
|
||||
|
||||
Create a file called `chat_assistant.prompt`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
model: gpt-4
|
||||
temperature: 0.7
|
||||
max_tokens: 150
|
||||
input:
|
||||
schema:
|
||||
user_message: string
|
||||
system_context?: string
|
||||
---
|
||||
|
||||
{% if system_context %}System: {{system_context}}
|
||||
|
||||
{% endif %}User: {{user_message}}
|
||||
```
|
||||
|
||||
### 2. Use with LiteLLM
|
||||
|
||||
```python
|
||||
import litellm
|
||||
|
||||
litellm.set_global_prompt_directory("path/to/your/prompts")
|
||||
|
||||
# Use with completion - the model prefix 'dotprompt/' tells LiteLLM to use prompt management
|
||||
response = litellm.completion(
|
||||
model="dotprompt/gpt-4", # The actual model comes from the .prompt file
|
||||
prompt_id="chat_assistant",
|
||||
prompt_variables={
|
||||
"user_message": "What is machine learning?",
|
||||
"system_context": "You are a helpful AI tutor."
|
||||
},
|
||||
# Any additional messages will be appended after the prompt
|
||||
messages=[{"role": "user", "content": "Please explain it simply."}]
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
## Prompt File Format
|
||||
|
||||
### Basic Structure
|
||||
|
||||
```yaml
|
||||
---
|
||||
# Model configuration
|
||||
model: gpt-4
|
||||
temperature: 0.7
|
||||
max_tokens: 500
|
||||
|
||||
# Input schema (optional)
|
||||
input:
|
||||
schema:
|
||||
name: string
|
||||
age: integer
|
||||
preferences?: array
|
||||
---
|
||||
|
||||
# Template content using Handlebars syntax
|
||||
Hello {{name}}!
|
||||
|
||||
{% if age >= 18 %}
|
||||
You're an adult, so here are some mature recommendations:
|
||||
{% else %}
|
||||
Here are some age-appropriate suggestions:
|
||||
{% endif %}
|
||||
|
||||
{% for pref in preferences %}
|
||||
- Based on your interest in {{pref}}, I recommend...
|
||||
{% endfor %}
|
||||
```
|
||||
|
||||
### Supported Frontmatter Fields
|
||||
|
||||
- **`model`**: The LLM model to use (e.g., `gpt-4`, `claude-3-sonnet`)
|
||||
- **`input.schema`**: Define expected input variables and their types
|
||||
- **`output.format`**: Expected output format (`json`, `text`, etc.)
|
||||
- **`output.schema`**: Structure of expected output
|
||||
|
||||
### Additional Parameters
|
||||
|
||||
- **`temperature`**: Model temperature (0.0 to 1.0)
|
||||
- **`max_tokens`**: Maximum tokens to generate
|
||||
- **`top_p`**: Nucleus sampling parameter (0.0 to 1.0)
|
||||
- **`frequency_penalty`**: Frequency penalty (0.0 to 1.0)
|
||||
- **`presence_penalty`**: Presence penalty (0.0 to 1.0)
|
||||
- any other parameters that are not model or schema-related will be treated as optional parameters to the model.
|
||||
|
||||
### Input Schema Types
|
||||
|
||||
- `string` or `str`: Text values
|
||||
- `integer` or `int`: Whole numbers
|
||||
- `float`: Decimal numbers
|
||||
- `boolean` or `bool`: True/false values
|
||||
- `array` or `list`: Lists of values
|
||||
- `object` or `dict`: Key-value objects
|
||||
|
||||
Use `?` suffix for optional fields: `name?: string`
|
||||
|
||||
## Message Format Conversion
|
||||
|
||||
The dotprompt manager intelligently converts your rendered prompts into proper chat messages:
|
||||
|
||||
### Simple Text → User Message
|
||||
```yaml
|
||||
---
|
||||
model: gpt-4
|
||||
---
|
||||
Tell me about {{topic}}.
|
||||
```
|
||||
Becomes: `[{"role": "user", "content": "Tell me about AI."}]`
|
||||
|
||||
### Role-Based Format → Multiple Messages
|
||||
```yaml
|
||||
---
|
||||
model: gpt-4
|
||||
---
|
||||
System: You are a {{role}}.
|
||||
|
||||
User: {{question}}
|
||||
```
|
||||
|
||||
Becomes:
|
||||
```python
|
||||
[
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "user", "content": "What is AI?"}
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
## Example Prompts
|
||||
|
||||
### Data Extraction
|
||||
```yaml
|
||||
# extract_info.prompt
|
||||
---
|
||||
model: gemini/gemini-1.5-pro
|
||||
input:
|
||||
schema:
|
||||
text: string
|
||||
output:
|
||||
format: json
|
||||
schema:
|
||||
title?: string
|
||||
summary: string
|
||||
tags: array
|
||||
---
|
||||
|
||||
Extract the requested information from the given text. Return JSON format.
|
||||
|
||||
Text: {{text}}
|
||||
```
|
||||
|
||||
### Code Assistant
|
||||
```yaml
|
||||
# code_helper.prompt
|
||||
---
|
||||
model: claude-3-5-sonnet-20241022
|
||||
temperature: 0.2
|
||||
max_tokens: 2000
|
||||
input:
|
||||
schema:
|
||||
language: string
|
||||
task: string
|
||||
code?: string
|
||||
---
|
||||
|
||||
You are an expert {{language}} programmer.
|
||||
|
||||
Task: {{task}}
|
||||
|
||||
{% if code %}
|
||||
Current code:
|
||||
```{{language}}
|
||||
{{code}}
|
||||
```
|
||||
{% endif %}
|
||||
|
||||
Please provide a complete, well-documented solution.
|
||||
```
|
||||
|
||||
### Multi-turn Conversation
|
||||
```yaml
|
||||
# conversation.prompt
|
||||
---
|
||||
model: gpt-4
|
||||
temperature: 0.8
|
||||
input:
|
||||
schema:
|
||||
personality: string
|
||||
context: string
|
||||
---
|
||||
|
||||
System: You are a {{personality}}. {{context}}
|
||||
|
||||
User: Let's start our conversation.
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### PromptManager
|
||||
|
||||
The core class for managing `.prompt` files.
|
||||
|
||||
#### Methods
|
||||
|
||||
- **`__init__(prompt_directory: str)`**: Initialize with directory path
|
||||
- **`render(prompt_id: str, variables: dict) -> str`**: Render prompt with variables
|
||||
- **`list_prompts() -> List[str]`**: Get all available prompt IDs
|
||||
- **`get_prompt(prompt_id: str) -> PromptTemplate`**: Get prompt template object
|
||||
- **`get_prompt_metadata(prompt_id: str) -> dict`**: Get prompt metadata
|
||||
- **`reload_prompts() -> None`**: Reload all prompts from directory
|
||||
- **`add_prompt(prompt_id: str, content: str, metadata: dict)`**: Add prompt programmatically
|
||||
|
||||
### DotpromptManager
|
||||
|
||||
LiteLLM integration class extending `PromptManagementBase`.
|
||||
|
||||
#### Methods
|
||||
|
||||
- **`__init__(prompt_directory: str)`**: Initialize with directory path
|
||||
- **`should_run_prompt_management(prompt_id: str, params: dict) -> bool`**: Check if prompt exists
|
||||
- **`set_prompt_directory(directory: str)`**: Change prompt directory
|
||||
- **`reload_prompts()`**: Reload prompts from directory
|
||||
|
||||
### PromptTemplate
|
||||
|
||||
Represents a single prompt with metadata.
|
||||
|
||||
#### Properties
|
||||
|
||||
- **`content: str`**: The prompt template content
|
||||
- **`metadata: dict`**: Full metadata from frontmatter
|
||||
- **`model: str`**: Specified model name
|
||||
- **`temperature: float`**: Model temperature
|
||||
- **`max_tokens: int`**: Token limit
|
||||
- **`input_schema: dict`**: Input validation schema
|
||||
- **`output_format: str`**: Expected output format
|
||||
- **`output_schema: dict`**: Output structure schema
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Organize by purpose**: Group related prompts in subdirectories
|
||||
2. **Use descriptive names**: `extract_user_info.prompt` vs `prompt1.prompt`
|
||||
3. **Define schemas**: Always specify input schemas for validation
|
||||
4. **Version control**: Store `.prompt` files in git for change tracking
|
||||
5. **Test prompts**: Use the test framework to validate prompt behavior
|
||||
6. **Keep templates focused**: One prompt should do one thing well
|
||||
7. **Use includes**: Break complex prompts into reusable components
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Prompt not found**: Ensure the `.prompt` file exists and has correct extension
|
||||
```python
|
||||
# Check available prompts
|
||||
from litellm.integrations.dotprompt import get_dotprompt_manager
|
||||
manager = get_dotprompt_manager()
|
||||
print(manager.prompt_manager.list_prompts())
|
||||
```
|
||||
|
||||
**Template errors**: Verify Handlebars syntax and variable names
|
||||
```python
|
||||
# Test rendering directly
|
||||
manager.prompt_manager.render("my_prompt", {"test": "value"})
|
||||
```
|
||||
|
||||
**Model not working**: Check that model name in frontmatter is correct
|
||||
```python
|
||||
# Check prompt metadata
|
||||
metadata = manager.prompt_manager.get_prompt_metadata("my_prompt")
|
||||
print(metadata)
|
||||
```
|
||||
|
||||
### Validation Errors
|
||||
|
||||
Input validation failures show helpful error messages:
|
||||
```
|
||||
ValueError: Invalid type for field 'age': expected int, got str
|
||||
```
|
||||
|
||||
Make sure your variables match the defined schema types.
|
||||
|
||||
## Contributing
|
||||
|
||||
The LiteLLM Dotprompt manager follows the [Dotprompt specification](https://google.github.io/dotprompt/) for maximum compatibility. When contributing:
|
||||
|
||||
1. Ensure compatibility with existing `.prompt` files
|
||||
2. Add tests for new features
|
||||
3. Update documentation
|
||||
4. Follow the existing code style
|
||||
|
||||
## License
|
||||
|
||||
This prompt management system is part of LiteLLM and follows the same license terms.
|
||||
@@ -0,0 +1,91 @@
|
||||
from typing import TYPE_CHECKING, Optional
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .prompt_manager import PromptManager, PromptTemplate
|
||||
from litellm.types.prompts.init_prompts import PromptLiteLLMParams, PromptSpec
|
||||
from litellm.integrations.custom_prompt_management import CustomPromptManagement
|
||||
|
||||
from litellm.types.prompts.init_prompts import SupportedPromptIntegrations
|
||||
|
||||
from .dotprompt_manager import DotpromptManager
|
||||
|
||||
# Global instances
|
||||
global_prompt_directory: Optional[str] = None
|
||||
global_prompt_manager: Optional["PromptManager"] = None
|
||||
|
||||
|
||||
def set_global_prompt_directory(directory: str) -> None:
|
||||
"""
|
||||
Set the global prompt directory for dotprompt files.
|
||||
|
||||
Args:
|
||||
directory: Path to directory containing .prompt files
|
||||
"""
|
||||
import litellm
|
||||
|
||||
litellm.global_prompt_directory = directory # type: ignore
|
||||
|
||||
|
||||
def _get_prompt_data_from_dotprompt_content(dotprompt_content: str) -> dict:
|
||||
"""
|
||||
Get the prompt data from the dotprompt content.
|
||||
|
||||
The UI stores prompts under `dotprompt_content` in the database. This function parses the content and returns the prompt data in the format expected by the prompt manager.
|
||||
"""
|
||||
from .prompt_manager import PromptManager
|
||||
|
||||
# Parse the dotprompt content to extract frontmatter and content
|
||||
temp_manager = PromptManager()
|
||||
metadata, content = temp_manager._parse_frontmatter(dotprompt_content)
|
||||
|
||||
# Convert to prompt_data format
|
||||
return {"content": content.strip(), "metadata": metadata}
|
||||
|
||||
|
||||
def prompt_initializer(
|
||||
litellm_params: "PromptLiteLLMParams", prompt_spec: "PromptSpec"
|
||||
) -> "CustomPromptManagement":
|
||||
"""
|
||||
Initialize a prompt from a .prompt file.
|
||||
"""
|
||||
prompt_directory = getattr(litellm_params, "prompt_directory", None)
|
||||
prompt_data = getattr(litellm_params, "prompt_data", None)
|
||||
prompt_id = getattr(litellm_params, "prompt_id", None)
|
||||
if prompt_directory:
|
||||
raise ValueError(
|
||||
"Cannot set prompt_directory when working with prompt_initializer. Needs to be a specific dotprompt file"
|
||||
)
|
||||
|
||||
prompt_file = getattr(litellm_params, "prompt_file", None)
|
||||
|
||||
# Handle dotprompt_content from database
|
||||
dotprompt_content = getattr(litellm_params, "dotprompt_content", None)
|
||||
if dotprompt_content and not prompt_data and not prompt_file:
|
||||
prompt_data = _get_prompt_data_from_dotprompt_content(dotprompt_content)
|
||||
|
||||
try:
|
||||
dot_prompt_manager = DotpromptManager(
|
||||
prompt_directory=prompt_directory,
|
||||
prompt_data=prompt_data,
|
||||
prompt_file=prompt_file,
|
||||
prompt_id=prompt_id,
|
||||
)
|
||||
|
||||
return dot_prompt_manager
|
||||
except Exception as e:
|
||||
raise e
|
||||
|
||||
|
||||
prompt_initializer_registry = {
|
||||
SupportedPromptIntegrations.DOT_PROMPT.value: prompt_initializer,
|
||||
}
|
||||
|
||||
# Export public API
|
||||
__all__ = [
|
||||
"PromptManager",
|
||||
"DotpromptManager",
|
||||
"PromptTemplate",
|
||||
"set_global_prompt_directory",
|
||||
"global_prompt_directory",
|
||||
"global_prompt_manager",
|
||||
]
|
||||
@@ -0,0 +1,378 @@
|
||||
"""
|
||||
Dotprompt manager that integrates with LiteLLM's prompt management system.
|
||||
Builds on top of PromptManagementBase to provide .prompt file support.
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, Union
|
||||
|
||||
from litellm.integrations.custom_prompt_management import CustomPromptManagement
|
||||
from litellm.integrations.prompt_management_base import PromptManagementClient
|
||||
from litellm.types.llms.openai import AllMessageValues
|
||||
from litellm.types.prompts.init_prompts import PromptSpec
|
||||
from litellm.types.utils import StandardCallbackDynamicParams
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
|
||||
else:
|
||||
LiteLLMLoggingObj = Any
|
||||
|
||||
from .prompt_manager import PromptManager, PromptTemplate
|
||||
|
||||
|
||||
class DotpromptManager(CustomPromptManagement):
|
||||
"""
|
||||
Dotprompt manager that integrates with LiteLLM's prompt management system.
|
||||
|
||||
This class enables using .prompt files with the litellm completion() function
|
||||
by implementing the PromptManagementBase interface.
|
||||
|
||||
Usage:
|
||||
# Set global prompt directory
|
||||
litellm.prompt_directory = "path/to/prompts"
|
||||
|
||||
# Use with completion
|
||||
response = litellm.completion(
|
||||
model="dotprompt/gpt-4",
|
||||
prompt_id="my_prompt",
|
||||
prompt_variables={"variable": "value"},
|
||||
messages=[{"role": "user", "content": "This will be combined with the prompt"}]
|
||||
)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt_directory: Optional[str] = None,
|
||||
prompt_file: Optional[str] = None,
|
||||
prompt_data: Optional[Union[dict, str]] = None,
|
||||
prompt_id: Optional[str] = None,
|
||||
):
|
||||
import litellm
|
||||
|
||||
self.prompt_directory = prompt_directory or litellm.global_prompt_directory
|
||||
# Support for JSON-based prompts stored in memory/database
|
||||
if isinstance(prompt_data, str):
|
||||
self.prompt_data = json.loads(prompt_data)
|
||||
else:
|
||||
self.prompt_data = prompt_data or {}
|
||||
|
||||
self._prompt_manager: Optional[PromptManager] = None
|
||||
self.prompt_file = prompt_file
|
||||
self.prompt_id = prompt_id
|
||||
|
||||
@property
|
||||
def integration_name(self) -> str:
|
||||
"""Integration name used in model names like 'dotprompt/gpt-4'."""
|
||||
return "dotprompt"
|
||||
|
||||
@property
|
||||
def prompt_manager(self) -> PromptManager:
|
||||
"""Lazy-load the prompt manager."""
|
||||
if self._prompt_manager is None:
|
||||
if (
|
||||
self.prompt_directory is None
|
||||
and not self.prompt_data
|
||||
and not self.prompt_file
|
||||
):
|
||||
raise ValueError(
|
||||
"Either prompt_directory or prompt_data must be set before using dotprompt manager. "
|
||||
"Set litellm.global_prompt_directory, initialize with prompt_directory parameter, or provide prompt_data."
|
||||
)
|
||||
self._prompt_manager = PromptManager(
|
||||
prompt_directory=self.prompt_directory,
|
||||
prompt_data=self.prompt_data,
|
||||
prompt_file=self.prompt_file,
|
||||
prompt_id=self.prompt_id,
|
||||
)
|
||||
return self._prompt_manager
|
||||
|
||||
def should_run_prompt_management(
|
||||
self,
|
||||
prompt_id: Optional[str],
|
||||
prompt_spec: Optional[PromptSpec],
|
||||
dynamic_callback_params: StandardCallbackDynamicParams,
|
||||
) -> bool:
|
||||
"""
|
||||
Determine if prompt management should run based on the prompt_id.
|
||||
|
||||
Returns True if the prompt_id exists in our prompt manager.
|
||||
"""
|
||||
if prompt_id is None:
|
||||
return False
|
||||
try:
|
||||
return prompt_id in self.prompt_manager.list_prompts()
|
||||
except Exception:
|
||||
# If there's any error accessing prompts, don't run prompt management
|
||||
return False
|
||||
|
||||
def _compile_prompt_helper(
|
||||
self,
|
||||
prompt_id: Optional[str],
|
||||
prompt_spec: Optional[PromptSpec],
|
||||
prompt_variables: Optional[dict],
|
||||
dynamic_callback_params: StandardCallbackDynamicParams,
|
||||
prompt_label: Optional[str] = None,
|
||||
prompt_version: Optional[int] = None,
|
||||
) -> PromptManagementClient:
|
||||
"""
|
||||
Compile a .prompt file into a PromptManagementClient structure.
|
||||
|
||||
This method:
|
||||
1. Loads the prompt template from the .prompt file (with optional version)
|
||||
2. Renders it with the provided variables
|
||||
3. Converts the rendered text into chat messages
|
||||
4. Extracts model and optional parameters from metadata
|
||||
"""
|
||||
|
||||
if prompt_id is None:
|
||||
raise ValueError("prompt_id is required for dotprompt manager")
|
||||
|
||||
try:
|
||||
# Get the prompt template (versioned or base)
|
||||
template = self.prompt_manager.get_prompt(
|
||||
prompt_id=prompt_id, version=prompt_version
|
||||
)
|
||||
if template is None:
|
||||
version_str = f" (version {prompt_version})" if prompt_version else ""
|
||||
raise ValueError(
|
||||
f"Prompt '{prompt_id}'{version_str} not found in prompt directory"
|
||||
)
|
||||
|
||||
# Render the template with variables (pass version for proper lookup)
|
||||
rendered_content = self.prompt_manager.render(
|
||||
prompt_id=prompt_id,
|
||||
prompt_variables=prompt_variables,
|
||||
version=prompt_version,
|
||||
)
|
||||
|
||||
# Convert rendered content to chat messages
|
||||
messages = self._convert_to_messages(rendered_content)
|
||||
|
||||
# Extract model from metadata (if specified)
|
||||
template_model = template.model
|
||||
|
||||
# Extract optional parameters from metadata
|
||||
optional_params = self._extract_optional_params(template)
|
||||
|
||||
return PromptManagementClient(
|
||||
prompt_id=prompt_id,
|
||||
prompt_template=messages,
|
||||
prompt_template_model=template_model,
|
||||
prompt_template_optional_params=optional_params,
|
||||
completed_messages=None,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
raise ValueError(f"Error compiling prompt '{prompt_id}': {e}")
|
||||
|
||||
async def async_compile_prompt_helper(
|
||||
self,
|
||||
prompt_id: Optional[str],
|
||||
prompt_variables: Optional[dict],
|
||||
dynamic_callback_params: StandardCallbackDynamicParams,
|
||||
prompt_spec: Optional[PromptSpec] = None,
|
||||
prompt_label: Optional[str] = None,
|
||||
prompt_version: Optional[int] = None,
|
||||
) -> PromptManagementClient:
|
||||
"""
|
||||
Async version of compile prompt helper. Since dotprompt operations are synchronous,
|
||||
this simply delegates to the sync version.
|
||||
"""
|
||||
if prompt_id is None:
|
||||
raise ValueError("prompt_id is required for dotprompt manager")
|
||||
|
||||
return self._compile_prompt_helper(
|
||||
prompt_id=prompt_id,
|
||||
prompt_spec=prompt_spec,
|
||||
prompt_variables=prompt_variables,
|
||||
dynamic_callback_params=dynamic_callback_params,
|
||||
prompt_label=prompt_label,
|
||||
prompt_version=prompt_version,
|
||||
)
|
||||
|
||||
def get_chat_completion_prompt(
|
||||
self,
|
||||
model: str,
|
||||
messages: List[AllMessageValues],
|
||||
non_default_params: dict,
|
||||
prompt_id: Optional[str],
|
||||
prompt_variables: Optional[dict],
|
||||
dynamic_callback_params: StandardCallbackDynamicParams,
|
||||
prompt_spec: Optional[PromptSpec] = None,
|
||||
prompt_label: Optional[str] = None,
|
||||
prompt_version: Optional[int] = None,
|
||||
ignore_prompt_manager_model: Optional[bool] = False,
|
||||
ignore_prompt_manager_optional_params: Optional[bool] = False,
|
||||
) -> Tuple[str, List[AllMessageValues], dict]:
|
||||
from litellm.integrations.prompt_management_base import PromptManagementBase
|
||||
|
||||
return PromptManagementBase.get_chat_completion_prompt(
|
||||
self,
|
||||
model,
|
||||
messages,
|
||||
non_default_params,
|
||||
prompt_id,
|
||||
prompt_variables,
|
||||
dynamic_callback_params,
|
||||
prompt_spec=prompt_spec,
|
||||
prompt_label=prompt_label,
|
||||
prompt_version=prompt_version,
|
||||
)
|
||||
|
||||
async def async_get_chat_completion_prompt(
|
||||
self,
|
||||
model: str,
|
||||
messages: List[AllMessageValues],
|
||||
non_default_params: dict,
|
||||
prompt_id: Optional[str],
|
||||
prompt_variables: Optional[dict],
|
||||
dynamic_callback_params: StandardCallbackDynamicParams,
|
||||
litellm_logging_obj: LiteLLMLoggingObj,
|
||||
prompt_spec: Optional[PromptSpec] = None,
|
||||
tools: Optional[List[Dict]] = None,
|
||||
prompt_label: Optional[str] = None,
|
||||
prompt_version: Optional[int] = None,
|
||||
ignore_prompt_manager_model: Optional[bool] = False,
|
||||
ignore_prompt_manager_optional_params: Optional[bool] = False,
|
||||
) -> Tuple[str, List[AllMessageValues], dict]:
|
||||
"""
|
||||
Async version - delegates to PromptManagementBase async implementation.
|
||||
"""
|
||||
from litellm.integrations.prompt_management_base import PromptManagementBase
|
||||
|
||||
return await PromptManagementBase.async_get_chat_completion_prompt(
|
||||
self,
|
||||
model,
|
||||
messages,
|
||||
non_default_params,
|
||||
prompt_id=prompt_id,
|
||||
prompt_variables=prompt_variables,
|
||||
litellm_logging_obj=litellm_logging_obj,
|
||||
dynamic_callback_params=dynamic_callback_params,
|
||||
prompt_spec=prompt_spec,
|
||||
tools=tools,
|
||||
prompt_label=prompt_label,
|
||||
prompt_version=prompt_version,
|
||||
ignore_prompt_manager_model=ignore_prompt_manager_model,
|
||||
ignore_prompt_manager_optional_params=ignore_prompt_manager_optional_params,
|
||||
)
|
||||
|
||||
def _convert_to_messages(self, rendered_content: str) -> List[AllMessageValues]:
|
||||
"""
|
||||
Convert rendered prompt content to chat messages.
|
||||
|
||||
This method supports multiple formats:
|
||||
1. Simple text -> converted to user message
|
||||
2. Text with role prefixes (System:, User:, Assistant:) -> parsed into separate messages
|
||||
3. Already formatted as a single message
|
||||
"""
|
||||
# Clean up the content
|
||||
content = rendered_content.strip()
|
||||
|
||||
# Try to parse role-based format (System: ..., User: ..., etc.)
|
||||
messages = []
|
||||
current_role = None
|
||||
current_content = []
|
||||
|
||||
lines = content.split("\n")
|
||||
|
||||
for line in lines:
|
||||
line = line.strip()
|
||||
|
||||
# Check for role prefixes
|
||||
if line.startswith("System:"):
|
||||
if current_role and current_content:
|
||||
messages.append(
|
||||
self._create_message(
|
||||
current_role, "\n".join(current_content).strip()
|
||||
)
|
||||
)
|
||||
current_role = "system"
|
||||
current_content = [line[7:].strip()] # Remove "System:" prefix
|
||||
elif line.startswith("User:"):
|
||||
if current_role and current_content:
|
||||
messages.append(
|
||||
self._create_message(
|
||||
current_role, "\n".join(current_content).strip()
|
||||
)
|
||||
)
|
||||
current_role = "user"
|
||||
current_content = [line[5:].strip()] # Remove "User:" prefix
|
||||
elif line.startswith("Assistant:"):
|
||||
if current_role and current_content:
|
||||
messages.append(
|
||||
self._create_message(
|
||||
current_role, "\n".join(current_content).strip()
|
||||
)
|
||||
)
|
||||
current_role = "assistant"
|
||||
current_content = [line[10:].strip()] # Remove "Assistant:" prefix
|
||||
else:
|
||||
# Continue current message content
|
||||
if current_role:
|
||||
current_content.append(line)
|
||||
else:
|
||||
# No role prefix found, treat as user message
|
||||
current_role = "user"
|
||||
current_content = [line]
|
||||
|
||||
# Add the last message
|
||||
if current_role and current_content:
|
||||
content_text = "\n".join(current_content).strip()
|
||||
if content_text: # Only add if there's actual content
|
||||
messages.append(self._create_message(current_role, content_text))
|
||||
|
||||
# If no messages were created, treat the entire content as a user message
|
||||
if not messages and content:
|
||||
messages.append(self._create_message("user", content))
|
||||
|
||||
return messages
|
||||
|
||||
def _create_message(self, role: str, content: str) -> AllMessageValues:
|
||||
"""Create a message with the specified role and content."""
|
||||
return {
|
||||
"role": role, # type: ignore
|
||||
"content": content,
|
||||
}
|
||||
|
||||
def _extract_optional_params(self, template: PromptTemplate) -> dict:
|
||||
"""
|
||||
Extract optional parameters from the prompt template metadata.
|
||||
|
||||
Includes parameters like temperature, max_tokens, etc.
|
||||
"""
|
||||
optional_params = {}
|
||||
|
||||
# Extract common parameters from metadata
|
||||
if template.optional_params is not None:
|
||||
optional_params.update(template.optional_params)
|
||||
|
||||
return optional_params
|
||||
|
||||
def set_prompt_directory(self, prompt_directory: str) -> None:
|
||||
"""Set the prompt directory and reload prompts."""
|
||||
self.prompt_directory = prompt_directory
|
||||
self._prompt_manager = None # Reset to force reload
|
||||
|
||||
def reload_prompts(self) -> None:
|
||||
"""Reload all prompts from the directory."""
|
||||
if self._prompt_manager:
|
||||
self._prompt_manager.reload_prompts()
|
||||
|
||||
def add_prompt_from_json(self, prompt_id: str, json_data: Dict[str, Any]) -> None:
|
||||
"""Add a prompt from JSON data."""
|
||||
content = json_data.get("content", "")
|
||||
metadata = json_data.get("metadata", {})
|
||||
self.prompt_manager.add_prompt(prompt_id, content, metadata)
|
||||
|
||||
def load_prompts_from_json(self, prompts_data: Dict[str, Dict[str, Any]]) -> None:
|
||||
"""Load multiple prompts from JSON data."""
|
||||
self.prompt_manager.load_prompts_from_json_data(prompts_data)
|
||||
|
||||
def get_prompts_as_json(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get all prompts in JSON format."""
|
||||
return self.prompt_manager.get_all_prompts_as_json()
|
||||
|
||||
def convert_prompt_file_to_json(self, file_path: str) -> Dict[str, Any]:
|
||||
"""Convert a .prompt file to JSON format."""
|
||||
return self.prompt_manager.prompt_file_to_json(file_path)
|
||||
@@ -0,0 +1,368 @@
|
||||
"""
|
||||
Based on Google's GenAI Kit dotprompt implementation: https://google.github.io/dotprompt/reference/frontmatter/
|
||||
"""
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple, Union
|
||||
|
||||
import yaml
|
||||
from jinja2 import DictLoader, Environment, select_autoescape
|
||||
|
||||
|
||||
class PromptTemplate:
|
||||
"""Represents a single prompt template with metadata and content."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
content: str,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
template_id: Optional[str] = None,
|
||||
):
|
||||
self.content = content
|
||||
self.metadata = metadata or {}
|
||||
self.template_id = template_id
|
||||
|
||||
# Extract common metadata fields
|
||||
restricted_keys = ["model", "input", "output"]
|
||||
self.model = self.metadata.get("model")
|
||||
self.input_schema = self.metadata.get("input", {}).get("schema", {})
|
||||
self.output_format = self.metadata.get("output", {}).get("format")
|
||||
self.output_schema = self.metadata.get("output", {}).get("schema", {})
|
||||
self.optional_params = {}
|
||||
for key in self.metadata.keys():
|
||||
if key not in restricted_keys:
|
||||
self.optional_params[key] = self.metadata[key]
|
||||
|
||||
def __repr__(self):
|
||||
return f"PromptTemplate(id='{self.template_id}', model='{self.model}')"
|
||||
|
||||
|
||||
class PromptManager:
|
||||
"""
|
||||
Manager for loading and rendering .prompt files following the Dotprompt specification.
|
||||
|
||||
Supports:
|
||||
- YAML frontmatter for metadata
|
||||
- Handlebars-style templating (using Jinja2)
|
||||
- Input/output schema validation
|
||||
- Model configuration
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt_id: Optional[str] = None,
|
||||
prompt_directory: Optional[str] = None,
|
||||
prompt_data: Optional[Dict[str, Dict[str, Any]]] = None,
|
||||
prompt_file: Optional[str] = None,
|
||||
):
|
||||
self.prompt_directory = Path(prompt_directory) if prompt_directory else None
|
||||
self.prompts: Dict[str, PromptTemplate] = {}
|
||||
self.prompt_file = prompt_file
|
||||
self.jinja_env = Environment(
|
||||
loader=DictLoader({}),
|
||||
autoescape=select_autoescape(["html", "xml"]),
|
||||
# Use Handlebars-style delimiters to match Dotprompt spec
|
||||
variable_start_string="{{",
|
||||
variable_end_string="}}",
|
||||
block_start_string="{%",
|
||||
block_end_string="%}",
|
||||
comment_start_string="{#",
|
||||
comment_end_string="#}",
|
||||
)
|
||||
|
||||
# Load prompts from directory if provided
|
||||
if self.prompt_directory:
|
||||
self._load_prompts()
|
||||
|
||||
if self.prompt_file:
|
||||
if not prompt_id:
|
||||
raise ValueError("prompt_id is required when prompt_file is provided")
|
||||
|
||||
template = self._load_prompt_file(self.prompt_file, prompt_id)
|
||||
self.prompts[prompt_id] = template
|
||||
|
||||
# Load prompts from JSON data if provided
|
||||
if prompt_data:
|
||||
self._load_prompts_from_json(prompt_data, prompt_id)
|
||||
|
||||
def _load_prompts(self) -> None:
|
||||
"""Load all .prompt files from the prompt directory."""
|
||||
if not self.prompt_directory or not self.prompt_directory.exists():
|
||||
raise ValueError(
|
||||
f"Prompt directory does not exist: {self.prompt_directory}"
|
||||
)
|
||||
|
||||
prompt_files = list(self.prompt_directory.glob("*.prompt"))
|
||||
|
||||
for prompt_file in prompt_files:
|
||||
try:
|
||||
prompt_id = prompt_file.stem # filename without extension
|
||||
template = self._load_prompt_file(prompt_file, prompt_id)
|
||||
self.prompts[prompt_id] = template
|
||||
# Optional: print(f"Loaded prompt: {prompt_id}")
|
||||
except Exception:
|
||||
# Optional: print(f"Error loading prompt file {prompt_file}")
|
||||
pass
|
||||
|
||||
def _load_prompts_from_json(
|
||||
self, prompt_data: Dict[str, Dict[str, Any]], prompt_id: Optional[str] = None
|
||||
) -> None:
|
||||
"""Load prompts from JSON data structure.
|
||||
|
||||
Expected format:
|
||||
{
|
||||
"prompt_id": {
|
||||
"content": "template content",
|
||||
"metadata": {"model": "gpt-4", "temperature": 0.7, ...}
|
||||
}
|
||||
}
|
||||
|
||||
or
|
||||
|
||||
{
|
||||
"content": "template content",
|
||||
"metadata": {"model": "gpt-4", "temperature": 0.7, ...}
|
||||
} + prompt_id
|
||||
"""
|
||||
if prompt_id:
|
||||
prompt_data = {prompt_id: prompt_data}
|
||||
|
||||
for prompt_id, prompt_info in prompt_data.items():
|
||||
try:
|
||||
content = prompt_info.get("content", "")
|
||||
metadata = prompt_info.get("metadata", {})
|
||||
|
||||
template = PromptTemplate(
|
||||
content=content,
|
||||
metadata=metadata,
|
||||
template_id=prompt_id,
|
||||
)
|
||||
self.prompts[prompt_id] = template
|
||||
except Exception:
|
||||
# Optional: print(f"Error loading prompt from JSON: {prompt_id}")
|
||||
pass
|
||||
|
||||
def _load_prompt_file(
|
||||
self, file_path: Union[str, Path], prompt_id: str
|
||||
) -> PromptTemplate:
|
||||
"""Load and parse a single .prompt file."""
|
||||
if isinstance(file_path, str):
|
||||
file_path = Path(file_path)
|
||||
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
|
||||
# Split frontmatter and content
|
||||
frontmatter, template_content = self._parse_frontmatter(content)
|
||||
|
||||
return PromptTemplate(
|
||||
content=template_content.strip(),
|
||||
metadata=frontmatter,
|
||||
template_id=prompt_id,
|
||||
)
|
||||
|
||||
def _parse_frontmatter(self, content: str) -> Tuple[Dict[str, Any], str]:
|
||||
"""Parse YAML frontmatter from prompt content."""
|
||||
# Match YAML frontmatter between --- delimiters
|
||||
frontmatter_pattern = r"^---\s*\n(.*?)\n---\s*\n(.*)$"
|
||||
match = re.match(frontmatter_pattern, content, re.DOTALL)
|
||||
|
||||
if match:
|
||||
frontmatter_yaml = match.group(1)
|
||||
template_content = match.group(2)
|
||||
|
||||
try:
|
||||
frontmatter = yaml.safe_load(frontmatter_yaml) or {}
|
||||
except yaml.YAMLError as e:
|
||||
raise ValueError(f"Invalid YAML frontmatter: {e}")
|
||||
else:
|
||||
# No frontmatter found, treat entire content as template
|
||||
frontmatter = {}
|
||||
template_content = content
|
||||
|
||||
return frontmatter, template_content
|
||||
|
||||
def render(
|
||||
self,
|
||||
prompt_id: str,
|
||||
prompt_variables: Optional[Dict[str, Any]] = None,
|
||||
version: Optional[int] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Render a prompt template with the given variables.
|
||||
|
||||
Args:
|
||||
prompt_id: The ID of the prompt template to render
|
||||
prompt_variables: Variables to substitute in the template
|
||||
version: Optional version number. If provided, looks for {prompt_id}.v{version}
|
||||
|
||||
Returns:
|
||||
The rendered prompt string
|
||||
|
||||
Raises:
|
||||
KeyError: If prompt_id is not found
|
||||
ValueError: If template rendering fails
|
||||
"""
|
||||
# Get the template (versioned or base)
|
||||
template = self.get_prompt(prompt_id=prompt_id, version=version)
|
||||
|
||||
if template is None:
|
||||
available_prompts = list(self.prompts.keys())
|
||||
version_str = f" (version {version})" if version else ""
|
||||
raise KeyError(
|
||||
f"Prompt '{prompt_id}'{version_str} not found. Available prompts: {available_prompts}"
|
||||
)
|
||||
|
||||
variables = prompt_variables or {}
|
||||
|
||||
# Validate input variables against schema if defined
|
||||
if template.input_schema:
|
||||
self._validate_input(variables, template.input_schema)
|
||||
|
||||
try:
|
||||
# Create Jinja2 template and render
|
||||
jinja_template = self.jinja_env.from_string(template.content)
|
||||
rendered = jinja_template.render(**variables)
|
||||
return rendered
|
||||
except Exception as e:
|
||||
raise ValueError(f"Error rendering template '{prompt_id}': {e}")
|
||||
|
||||
def _validate_input(
|
||||
self, variables: Dict[str, Any], schema: Dict[str, Any]
|
||||
) -> None:
|
||||
"""Basic validation of input variables against schema."""
|
||||
for field_name, field_type in schema.items():
|
||||
if field_name in variables:
|
||||
value = variables[field_name]
|
||||
expected_type = self._get_python_type(field_type)
|
||||
|
||||
if not isinstance(value, expected_type):
|
||||
raise ValueError(
|
||||
f"Invalid type for field '{field_name}': "
|
||||
f"expected {getattr(expected_type, '__name__', str(expected_type))}, got {type(value).__name__}"
|
||||
)
|
||||
|
||||
def _get_python_type(self, schema_type: str) -> Union[type, tuple]:
|
||||
"""Convert schema type string to Python type."""
|
||||
type_mapping: Dict[str, Union[type, tuple]] = {
|
||||
"string": str,
|
||||
"str": str,
|
||||
"number": (int, float),
|
||||
"integer": int,
|
||||
"int": int,
|
||||
"float": float,
|
||||
"boolean": bool,
|
||||
"bool": bool,
|
||||
"array": list,
|
||||
"list": list,
|
||||
"object": dict,
|
||||
"dict": dict,
|
||||
}
|
||||
|
||||
return type_mapping.get(schema_type.lower(), str) # type: ignore
|
||||
|
||||
def get_prompt(
|
||||
self, prompt_id: str, version: Optional[int] = None
|
||||
) -> Optional[PromptTemplate]:
|
||||
"""
|
||||
Get a prompt template by ID and optional version.
|
||||
|
||||
Args:
|
||||
prompt_id: The base prompt ID
|
||||
version: Optional version number. If provided, looks for {prompt_id}.v{version}
|
||||
|
||||
Returns:
|
||||
The prompt template if found, None otherwise
|
||||
"""
|
||||
if version is not None:
|
||||
# Try versioned prompt first: prompt_id.v{version}
|
||||
versioned_id = f"{prompt_id}.v{version}"
|
||||
if versioned_id in self.prompts:
|
||||
return self.prompts[versioned_id]
|
||||
|
||||
# Fall back to base prompt_id
|
||||
return self.prompts.get(prompt_id)
|
||||
|
||||
def list_prompts(self) -> List[str]:
|
||||
"""Get a list of all available prompt IDs."""
|
||||
return list(self.prompts.keys())
|
||||
|
||||
def get_prompt_metadata(self, prompt_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get metadata for a specific prompt."""
|
||||
template = self.prompts.get(prompt_id)
|
||||
return template.metadata if template else None
|
||||
|
||||
def reload_prompts(self) -> None:
|
||||
"""Reload all prompts from the directory (if directory was provided)."""
|
||||
self.prompts.clear()
|
||||
if self.prompt_directory:
|
||||
self._load_prompts()
|
||||
|
||||
def add_prompt(
|
||||
self, prompt_id: str, content: str, metadata: Optional[Dict[str, Any]] = None
|
||||
) -> None:
|
||||
"""Add a prompt template programmatically."""
|
||||
template = PromptTemplate(
|
||||
content=content, metadata=metadata or {}, template_id=prompt_id
|
||||
)
|
||||
self.prompts[prompt_id] = template
|
||||
|
||||
def prompt_file_to_json(self, file_path: Union[str, Path]) -> Dict[str, Any]:
|
||||
"""Convert a .prompt file to JSON format.
|
||||
|
||||
Args:
|
||||
file_path: Path to the .prompt file
|
||||
|
||||
Returns:
|
||||
Dictionary with 'content' and 'metadata' keys
|
||||
"""
|
||||
file_path = Path(file_path)
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
|
||||
# Parse frontmatter and content
|
||||
frontmatter, template_content = self._parse_frontmatter(content)
|
||||
|
||||
return {"content": template_content.strip(), "metadata": frontmatter}
|
||||
|
||||
def json_to_prompt_file(self, prompt_data: Dict[str, Any]) -> str:
|
||||
"""Convert JSON prompt data to .prompt file format.
|
||||
|
||||
Args:
|
||||
prompt_data: Dictionary with 'content' and 'metadata' keys
|
||||
|
||||
Returns:
|
||||
String content in .prompt file format
|
||||
"""
|
||||
content = prompt_data.get("content", "")
|
||||
metadata = prompt_data.get("metadata", {})
|
||||
|
||||
if not metadata:
|
||||
# No metadata, return just the content
|
||||
return content
|
||||
|
||||
# Convert metadata to YAML frontmatter
|
||||
import yaml
|
||||
|
||||
frontmatter_yaml = yaml.dump(metadata, default_flow_style=False)
|
||||
|
||||
return f"---\n{frontmatter_yaml}---\n{content}"
|
||||
|
||||
def get_all_prompts_as_json(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get all loaded prompts in JSON format.
|
||||
|
||||
Returns:
|
||||
Dictionary mapping prompt_id to prompt data
|
||||
"""
|
||||
result = {}
|
||||
for prompt_id, template in self.prompts.items():
|
||||
result[prompt_id] = {
|
||||
"content": template.content,
|
||||
"metadata": template.metadata,
|
||||
}
|
||||
return result
|
||||
|
||||
def load_prompts_from_json_data(
|
||||
self, prompt_data: Dict[str, Dict[str, Any]]
|
||||
) -> None:
|
||||
"""Load additional prompts from JSON data (merges with existing prompts)."""
|
||||
self._load_prompts_from_json(prompt_data)
|
||||
Reference in New Issue
Block a user