Jiki API Reference

This API reference documents the key components of Jiki's architecture. It's organized by functional areas to help you understand how the different pieces fit together.

Core Components

This section covers the central components of Jiki that form the backbone of the orchestration framework.

Factory Function

The factory function is the primary entry point for most users. It simplifies creating a properly configured orchestrator with sensible defaults.

The Jiki() Factory Pattern

The Jiki() function follows the factory pattern, creating and configuring all necessary components behind the scenes:

Initialization Flow:
- Sets up a TraceLogger if tracing is enabled
- Configures the appropriate MCP transport (stdio or SSE)
- Creates a JikiClient instance for tool/resource management
- Discovers or loads tool configurations
- Initializes a language model wrapper (via LiteLLM)
- Creates and returns a fully configured JikiOrchestrator
Usage Examples:

# Basic usage with auto-discovery
orchestrator = Jiki(
    auto_discover_tools=True,
    mcp_script_path="servers/calculator_server.py"
)

# Advanced configuration
orchestrator = Jiki(
    model="openai/gpt-4o",
    tools="path/to/tools.json",
    mcp_mode="sse",
    mcp_url="http://localhost:8000",
    trace=True,
    trace_dir="custom_traces",
    prompt_builder=MyCustomPromptBuilder(),
    sampler_config=SamplerConfig(temperature=0.7)
)

Factory function to create and configure a JikiOrchestrator instance.

This factory infers the model loading mechanism: - If hf_model_path and model_arch are provided, it uses VerlCompatibleModel to load a local Hugging Face model compatible with the verl library. - Otherwise, it uses LiteLLMModel with litellm_model_name.

Parameters:

Name	Type	Description	Default
`litellm_model_name`	`Optional[str]`	Model name for LiteLLM (used if HF params are not provided). Default: "anthropic/claude-3-sonnet-20240229".	`'anthropic/claude-3-sonnet-20240229'`
`hf_model_path`	`Optional[str]`	Path to Hugging Face model weights. If provided, triggers `verl` loader.	`None`
`hf_tokenizer_path`	`Optional[str]`	Path to Hugging Face tokenizer (optional, defaults to hf_model_path).	`None`
`model_arch`	`Optional[str]`	Model architecture for verl registry (required if hf_model_path is provided).	`None`
`hf_model_kwargs`	`Optional[Dict[str, Any]]`	Optional kwargs for HuggingFace `from_pretrained`.	`None`
`tools`	`Optional[Union[str, List[Dict[str, Any]]]]`	Tool configuration (path to JSON, list of dicts, or None).	`None`
`auto_discover_tools`	`bool`	If True, discover tools from the MCP endpoint.	`False`
`mcp_mode`	`str`	Transport mode for MCP client ('stdio' or 'sse').	`'stdio'`
`mcp_script_path`	`Optional[str]`	Path to the script for stdio MCP transport.	`None`
`mcp_url`	`Optional[str]`	URL for the SSE MCP endpoint.	`None`
`trace`	`bool`	Enable/disable interaction tracing.	`True`
`trace_dir`	`Optional[str]`	Directory to save trace logs.	`'interaction_traces'`
`conversation_root_manager`	`Optional[IConversationRootManager]`	Optional custom manager for conversation state.	`None`
`prompt_builder`	`Optional[Any]`	Optional custom prompt builder.	`None`
`sampler_config`	`Optional[ISamplerConfig]`	Optional custom sampler configuration.	`None`

Returns:

Type	Description
`JikiOrchestrator`	An initialized JikiOrchestrator instance.

Raises:

Type	Description
`ValueError`	If configuration is ambiguous or invalid.
`ImportError`	If required packages for the inferred model loader are missing.
`RuntimeError`	If model loading or tool discovery fails.

Orchestrator

JikiOrchestrator Architecture and Functionality

The JikiOrchestrator is the central component responsible for coordinating the entire interaction lifecycle within Jiki. It manages communication between the language model (LLM), the tool execution layer (MCP client), and the conversation state.

Key responsibilities include component management, orchestrating the conversation flow, and handling state persistence. The orchestrator wraps the selected language model (model) and holds references to the MCP client (mcp_client) for tool operations, a PromptBuilder for formatting LLM inputs, and an optional TraceLogger for debugging. It manages the tool configurations (tools_config) and builds an internal map (_tools_map) for efficient validation.

Internally, the orchestrator maintains the conversation history as a list of messages (_messages). When processing user input (process_user_input), it distinguishes between the first turn (where it may fetch resources via mcp_client.list_resources() and builds an initial system prompt using the prompt_builder) and subsequent turns (where it simply appends the user message). Before sending messages to the LLM, it uses the jiki.utils.context.trim_context utility to ensure the history fits within the model's context window, relying on jiki.utils.token.count_tokens for measurement. The core LLM interaction, including streaming responses and intercepting tool calls, is delegated to the jiki.utils.streaming.generate_and_intercept utility function. This function uses callbacks (_handle_tool_call) provided by the orchestrator to manage tool execution.

The _handle_tool_call method is responsible for parsing the LLM's tool call request (using jiki.utils.parsing.extract_tool_call and jiki.utils.tool.parse_tool_call_content), validating it against the tool schema map (_tools_map via jiki.utils.tool.validate_tool_call), executing the validated call via the mcp_client.execute_tool_call, and formatting the result (as <mcp_tool_result>...) to be injected back into the conversation history. It also records successful calls in _last_tool_calls.

For state persistence, the JikiOrchestrator itself implements snapshot() and resume() methods, allowing the current conversation state (primarily _messages and _last_tool_calls) to be saved and restored externally. It also accepts an optional conversation_root_manager during initialization for custom state handling (see Roots and Conversation State Management section). Core functionality is exposed through methods like process() for simple interactions and process_detailed() for retrieving structured results including tool call information.

Usage Example

# Process a simple query
result = orchestrator.process("What is 2 + 2?")
print(result)

# Get detailed response with tool calls
detailed = orchestrator.process_detailed("What is the square root of 144?")
print(f"Result: {detailed.result}")
print(f"Tool calls: {detailed.tool_calls}")

# Save conversation state
state = orchestrator.snapshot()
# ... later ...
orchestrator.resume(state)

jiki.orchestrator.JikiOrchestrator

JikiOrchestrator(model, mcp_client, tools_config, logger=None, prompt_builder=None, conversation_root_manager=None)

Central orchestration engine for Jiki, managing LLM messages, tool/resource calls, and conversation context.

Typically instantiated via the Jiki() factory function for ease of use.

Example (using factory): >>> from jiki import Jiki >>> orchestrator = Jiki( ... model="anthropic/claude-3-haiku-20240307", ... auto_discover_tools=True, ... mcp_script_path="servers/calculator_server.py" ... ) >>> result = orchestrator.process("What is 2 + 3?") >>> print(result)

Initialize the Jiki Orchestrator.

Parameters:

Name	Type	Description	Default
`model`	`Any`	LLM model wrapper instance (e.g., LiteLLMModel).	required
`mcp_client`	`IMCPClient`	Tool client implementing the IMCPClient interface.	required
`tools_config`	`List[Dict[str, Any]]`	List of available tool schemas (dictionaries).	required
`logger`	`Optional[TraceLogger]`	Optional TraceLogger instance for recording interactions.	`None`
`prompt_builder`	`Optional[IPromptBuilder]`	Optional custom prompt builder implementing IPromptBuilder.	`None`
`conversation_root_manager`	`Optional[IConversationRootManager]`	Optional manager for conversation state persistence.	`None`

Functions

build_initial_prompt

build_initial_prompt(user_input, resources_config=None)

Build the initial prompt for the LLM, including user input and available tools.

create_available_tools_block

create_available_tools_block()

Format a block describing the available tools (e.g., ... ).

process_user_input `async`

process_user_input(user_input, max_tokens_ctx=6000)

Orchestrate a single user query, returning the final answer.

resume

resume(snapshot)

Restore conversation state from a snapshot dict.

snapshot

snapshot()

Capture the current conversation state as a snapshot dict.

Detailed Response

Working with Detailed Responses

The DetailedResponse and ToolCall classes provide structured access to the results of an interaction, particularly when tools are involved. Instead of just getting the final string output, you can inspect the sequence of operations.

The DetailedResponse object contains the overall result (the final text response from the LLM), a list of tool_calls made during the processing, and potentially the raw interaction traces if tracing was enabled. Each item in the tool_calls list is a ToolCall object.

A ToolCall object represents a single tool invocation and holds the tool_name that was called, the arguments dictionary passed to it, and the result returned by the tool execution.

This detailed structure is useful for various purposes, such as debugging (by analyzing which tools were used and with what parameters), extracting specific structured data returned by tools, building user interfaces that visualize the tool execution flow, or creating automated test cases for tool integrations.

Usage Example:

detailed = orchestrator.process_detailed("What is 15 * 8?")

# Access the string result
print(f"Final answer: {detailed.result}")

# Examine tool calls
for call in detailed.tool_calls:
    print(f"Tool: {call.tool_name}")
    print(f"Arguments: {call.arguments}")
    print(f"Result: {call.result}")

jiki.models.response.DetailedResponse

DetailedResponse(result, tool_calls=None, traces=None)

Enhanced response object with additional information about tool calls and traces.

jiki.models.response.ToolCall

ToolCall(tool, arguments, result)

Represents a single tool call.

MCP Client Components

These components handle communication with Model Context Protocol (MCP) tool servers.

Jiki Client

JikiClient Architecture and Role

The JikiClient acts as the primary interface between the Jiki orchestrator and MCP-compatible tool servers. It implements the full Model Context Protocol (MCP) to manage the lifecycle of communication and interaction with external tools and resources.

Its core responsibilities include managing the underlying transport layer, handling various tool and resource operations, managing conversation state via roots, and facilitating debugging through interaction tracing. The client uses fastmcp.Client internally to establish connections, supporting different transport mechanisms like stdio for local scripts and SSE or WebSockets for network communication, managing the connection lifecycle within asynchronous contexts.

For tool interactions, the JikiClient discovers available tools (discover_tools()) and executes them with specified arguments (execute_tool_call()), processing and formatting the results for the orchestrator. It similarly handles resource discovery (list_resources()) and retrieval (read_resource()).

In stateful conversations, the client manages root URIs (list_roots(), send_roots_list_changed()) to synchronize context with the MCP server. For debugging, it captures all MCP messages, including server-side logs, making these traces available via get_interaction_traces(). The client also maintains backward compatibility through the EnhancedMCPClient layer, handling deprecations gracefully.

While typically instantiated by the main Jiki() factory function, the JikiClient can be used directly for more specialized scenarios.

Usage Example:

# Creating a client directly (usually done by Jiki() factory)
client = JikiClient("servers/calculator_server.py")

# Initialize and discover tools
await client.initialize()
tools = await client.discover_tools()

# Execute a tool call
result = await client.execute_tool_call("add", {"a": 5, "b": 3})
print(result)  # "8"

# Get traces for debugging
traces = client.get_interaction_traces()

jiki.mcp_client.JikiClient

JikiClient(transport_source, roots=None)

Bases: IMCPClient

The standard, full-featured MCP client for Jiki, built using fastmcp. (Formerly known as FullMCPClient and EnhancedMCPClient)

Handles the complete MCP lifecycle using the fastmcp.Client: - Transport management (inferred by fastmcp.Client). - Initialization handshake (implicit in async with Client(...)). - Tool discovery (discover_tools -> client.list_tools). - Tool execution (execute_tool_call -> client.call_tool). - Resource listing and reading (list_resources -> client.list_resources, etc.). - Roots listing and change notification (send_roots_list_changed). - Interaction tracing.

This is the default client used by Jiki and is recommended for most use cases. It directly uses the fastmcp library.

Reference: https://gofastmcp.com/clients/client

Initialize the Jiki MCP client using fastmcp.

:param transport_source: Source for the transport connection. This can be: - Path to a server script (.py, .js) for stdio. - URL (http/s for SSE, ws/s for WebSocket). - A fastmcp.server.FastMCP instance for in-memory transport. fastmcp.Client will infer the correct transport. Reference: https://gofastmcp.com/clients/transports :param roots: Optional list of root URIs or a callable returning them for the roots capability. Reference: https://gofastmcp.com/clients/client#roots

Functions

discover_tools `async`

discover_tools()

Discover tools using the client.list_tools() method.

execute_tool_call `async`

execute_tool_call(tool_name, arguments)

Execute a tool call using the client.call_tool method with tracing.

get_interaction_traces

get_interaction_traces()

Return all logged interaction traces for debugging and analysis.

initialize `async`

initialize(*args, **kwargs)

Marks the client as initialized (for compatibility with callers). The actual MCP handshake happens implicitly when the first async with Client(...) block is entered in other methods.

list_resources `async`

list_resources()

List resources using client.list_resources.

list_roots `async`

list_roots()

Return the list of roots configured for this client.

The MCP specification defines roots/list as a client capability – therefore FastMCP servers (and the fastmcp.Client library) do not expose a dedicated RPC for listing roots. Instead, the client communicates its available roots to the server during the handshake via the roots/set_roots capability.

This helper surfaces that locally‑configured information so that higher‑ level application code (examples, orchestrators, etc.) can still query the active root list in a uniform way without having to keep an external reference.

Returns

List[Dict[str, Any]] A list of dictionaries with the keys uri and name. If the root objects contain additional attributes those are ignored for now to keep the structure simple and implementation‑agnostic.

read_resource `async`

read_resource(uri)

Read resource using client.read_resource.

send_roots_list_changed `async`

send_roots_list_changed()

Notify server of roots list change via client.send_roots_list_changed.

MCP Client Transports

The connection between the JikiClient and the MCP server is managed by a Transport layer. This layer handles the specifics of how communication occurs, whether it's launching a local script or connecting to a network service. Jiki leverages the transport system provided by the underlying fastmcp library.

While fastmcp supports various transport types [https://gofastmcp.com/clients/transports], the Jiki() factory function provides convenient configuration for the most common ones used with Jiki:

Stdio (PythonStdioTransport):
- Use Case: Running a Python-based MCP server as a local subprocess. Communication happens via the subprocess's standard input and standard output.
- Configuration: Specify the path to the server script using the mcp_script_path argument in Jiki(). This is the default if mcp_mode is not set or set to 'stdio'.
- Details: Jiki uses fastmcp.client.transports.PythonStdioTransport for this, as seen in jiki.transports.factory.py. Ideal for development or simple self-contained tools.
SSE (SSETransport):
- Use Case: Connecting to a persistent MCP server running over a network via HTTP/S using Server-Sent Events.
- Configuration: Set mcp_mode="sse" and provide the server's URL via the mcp_url argument in Jiki() (defaults to http://localhost:6277/mcp if not provided, according to jiki.transports.factory.py).
- Details: Uses fastmcp.client.transports.SSETransport. Suitable for connecting to deployed MCP services. See https://gofastmcp.com/clients/transports#sse-server-sent-events for more on SSE transport.

Other FastMCP Transports:

The underlying fastmcp.Client used by JikiClient can also potentially work with other transports like WebSockets (WSTransport) or connect directly to in-process servers (FastMCPTransport) for testing, though these might require more direct configuration of the JikiClient rather than using the main Jiki() factory function's simpler parameters. Refer to the FastMCP documentation [https://gofastmcp.com/clients/transports] for the full list and capabilities.

Client Interfaces

Purpose and Structure

The client interfaces, IToolClient and IMCPClient (defined in jiki.tool_client), establish the essential contracts for how the Jiki orchestrator interacts with tool execution backends.

IToolClient: Represents the base interface, requiring methods for fundamental tool operations like discovery (discover_tools) and execution (execute_tool_call).
IMCPClient: Extends IToolClient, adding methods specific to the full Model Context Protocol (MCP), including resource listing/reading (via IResourceManager inheritance) and roots management (list_roots, send_roots_list_changed). This is the interface expected by the JikiOrchestrator.

Standard Usage vs. Custom Implementations:

For most use cases involving standard MCP servers, the built-in jiki.mcp_client.JikiClient is the recommended implementation. It leverages the robust fastmcp.Client library [https://gofastmcp.com/clients/client] and handles various transports (Stdio, SSE, WebSockets) and the MCP protocol details automatically. Customization for standard clients typically involves configuring the transport_source and roots when initializing JikiClient (often done indirectly via Jiki() factory parameters).

Why Build a Custom Client?

Implementing IMCPClient (or IToolClient for basic needs) yourself is an advanced task, typically only necessary in specific situations where JikiClient is unsuitable, such as:

Integrating Non-MCP Systems: Connecting Jiki to a backend that uses a completely different communication protocol (e.g., a custom REST API, gRPC service) and cannot be exposed via an MCP server.
Specialized Communication Logic: Requiring highly custom logic for connection management, authentication, error handling, retries, or caching that goes beyond the capabilities or configuration options of fastmcp.Client.
Mocking for Testing: Creating mock or stub clients for unit or integration testing the orchestrator without needing a live backend.
Performance-Critical Scenarios: In rare cases where the overhead of fastmcp.Client might be prohibitive, requiring a bare-metal implementation (though fastmcp itself is designed to be efficient).

How to Build a Custom Client (Conceptual Steps):

Define a Class: Create a Python class that explicitly inherits from jiki.tool_client.IMCPClient (or IToolClient).
Implement Methods: Implement all the async methods defined by the chosen interface(s). This involves writing the code to handle the underlying communication (e.g., making HTTP requests, calling library functions, interacting with a mock state).
Handle State: Manage any necessary connection state, authentication tokens, etc., within your class instance.
Return Correct Types: Ensure each method returns data in the format expected by the interface definition (e.g., discover_tools returns a list of dictionaries representing tool schemas).
Integrate: Pass an instance of your custom client class to the JikiOrchestrator during its initialization.

Example Skeleton (Conceptual)

from jiki.tool_client import IMCPClient
from typing import List, Dict, Any

# Implementing a custom client (Conceptual - requires actual logic)
class MyNonMCPClient(IMCPClient):
    def __init__(self, api_endpoint: str):
        self.endpoint = api_endpoint
        # Add necessary state like authentication tokens, session objects etc.

    async def discover_tools(self) -> List[Dict[str, Any]]:
        # Logic to fetch tool definitions from the custom backend (e.g., via REST)
        # and translate them into the expected MCP-like schema format.
        print(f"Custom client discovering tools from {self.endpoint}...")
        # ... implementation ...
        return [{"tool_name": "custom_action", "description": "Performs a custom action", "arguments": {}, "required": []}]

    async def execute_tool_call(self, tool_name: str, arguments: dict) -> str:
        # Logic to call the specific tool/action on the custom backend
        # (e.g., make a POST request) and return the result as a string.
        print(f"Custom client executing '{tool_name}' on {self.endpoint}...")
        # ... implementation ...
        return f"Result from custom action '{tool_name}'"

    async def list_resources(self) -> List[Dict[str, Any]]:
        # Logic for listing resources, if applicable to the custom backend.
        print(f"Custom client listing resources from {self.endpoint}...")
        # ... implementation ...
        return [] # Or return actual resources if backend supports them

    async def read_resource(self, uri: str) -> List[Dict[str, Any]]:
        # Logic for reading a resource, if applicable.
        print(f"Custom client reading resource '{uri}' from {self.endpoint}...")
        # ... implementation ...
        return [] # Or return actual content

    async def list_roots(self) -> List[Dict[str, Any]]:
        # If the custom backend has a concept similar to roots, implement here.
        print(f"Custom client listing roots...")
        # ... implementation ...
        return []

    async def send_roots_list_changed(self) -> None:
        # If the custom backend needs notifications about context changes, implement here.
        print(f"Custom client handling roots changed...")
        # ... implementation ...
        pass

# Usage:
# custom_client = MyNonMCPClient("http://my-custom-api.com")
# orchestrator = JikiOrchestrator(client=custom_client, ...) # Assuming direct Orchestrator init

Bases: IToolClient, IResourceManager, Protocol

Protocol defining the comprehensive interface for a full-featured MCP client within Jiki.

This interface combines several key MCP capabilities into a single contract: - Tool Discovery & Execution (inherited from IToolClient) - Resource Listing & Reading (inherited from IResourceManager) - Roots Listing & Change Notification (defined here)

By depending on IMCPClient, components like JikiOrchestrator can leverage the full spectrum of standard MCP client-side interactions without being tied to a specific implementation.

Benefits: - Unified Interface: Provides a single point of interaction for all standard MCP client operations needed by the orchestrator. - Complete Abstraction: Fully decouples the orchestrator from the specifics of the underlying client library (like fastmcp) and transport mechanism. - Extensibility: Allows for alternative full MCP client implementations.

Implementation: - The primary implementation provided by Jiki is JikiClient (see jiki/mcp_client.py), which integrates fastmcp.Client with additional logic for handshakes, tracing, and protocol adherence. - This protocol represents the expected interface for the client passed to JikiOrchestrator during its initialization.

Bases: Protocol

Protocol defining the standard interface for discovering and executing tools via MCP.

This interface abstracts the core functionalities required by the Jiki orchestrator to interact with an external tool server. These core functionalities are:

Discovering tools: Identifying available tools and their schemas.
Executing tools: Invoking a specific tool with given arguments.

By depending on IToolClient, the orchestrator can work with any implementation that fulfills this contract, decoupling it from specific client implementations like JikiClient.

Benefits: - Modularity: Allows different tool client strategies or backends to be plugged in. - Testability: Enables using mock or stub tool clients during testing. - Clear Contract: Explicitly defines the essential interactions for tool handling.

Implementation: - The primary implementation provided by Jiki is JikiClient (see jiki/mcp_client.py), which handles the full MCP handshake and protocol. - The methods defined here (discover_tools, execute_tool_call) represent the minimal subset required by the orchestrator for basic tool use.

discover_tools `async`

discover_tools()

Discover available tools from the MCP server. Returns a list of tool schema dictionaries.

execute_tool_call `async`

execute_tool_call(tool_name, arguments)

Invoke a tool by name with the provided arguments. Returns the tool's raw string result (or JSON-encoded value).

Configuration and Customization

These components allow customizing Jiki's behavior for advanced use cases.

Sampling Configuration

Sampling parameters allow fine-grained control over how the underlying Language Model (LLM) generates text during interactions. They influence the creativity, determinism, length, and stopping conditions of the responses.

Note: This SamplerConfig controls the parameters for Language Model calls made directly by the Jiki orchestrator when processing user input and generating responses. It does not configure how Jiki might handle the separate Model Context Protocol feature where an MCP server requests an LLM completion from the client (via sampling/createMessage, see https://modelcontextprotocol.io/docs/concepts/sampling).

Jiki provides the jiki.sampling.SamplerConfig dataclass to specify these settings:

temperature (float, default 1.0): Controls randomness. Lower values (e.g., 0.1) make the output more focused and deterministic, while higher values increase diversity and creativity.
top_p (float, default 1.0): Nucleus sampling. Restricts generation to tokens comprising the top 'p' probability mass. Lower values (e.g., 0.8) further restrict the LLM's choices, often leading to more factual or less surprising text.
max_tokens (Optional[int], default None): Sets a hard limit on the maximum number of tokens to be generated in a single response.
stop (Optional[List[str]], default None): A list of text sequences. If the LLM generates any of these sequences, generation stops immediately.

To apply custom sampling settings, you instantiate SamplerConfig with your desired values and pass this instance to the main Jiki() factory function using the sampler_config argument. The orchestrator will then use these parameters when invoking the LLM.

Usage Example

from jiki import Jiki, SamplerConfig

# Define custom sampling settings (e.g., low temperature for less randomness)
custom_sampler = SamplerConfig(temperature=0.2, max_tokens=100)

# Pass the config to the Jiki factory
orchestrator = Jiki(
    sampler_config=custom_sampler,
    # Other necessary parameters like mcp_script_path or tools...
    auto_discover_tools=True,
    mcp_script_path="servers/calculator_server.py"
)

# Subsequent calls will use the specified sampling parameters
result = orchestrator.process("Explain the concept of temperature in LLMs briefly.")
print(result)

jiki.sampling.SamplerConfig `dataclass`

SamplerConfig(temperature=1.0, top_p=1.0, max_tokens=None, stop=None)

Default dataclass for LLM sampling configuration.

Controls how the underlying LLM samples tokens

temperature (float): sampling randomness (lower = more deterministic).
top_p (float): nucleus sampling probability (lower = restrict options).
max_tokens (Optional[int]): cap on tokens generated.
stop (Optional[List[str]]): stop sequences to terminate generation early.

Example

config = SamplerConfig( temperature=0.7, top_p=0.8, max_tokens=300, stop=["###"] )

To apply in an LLM API call

params = config.to_dict() response = llm_api.create( model="gpt-4", messages=messages, **params )

Functions

to_dict

to_dict()

Convert to dict, omitting None values.

Bases: Protocol

Interface for LLM sampling configuration parameters, following MCP sampling guidelines.

See Model Context Protocol Sampling spec: https://modelcontextprotocol.io/docs/concepts/sampling

Sampling parameters control model behavior at runtime

temperature: randomness in sampling (0.0 = deterministic, 1.0 = default)
top_p: nucleus sampling threshold (1.0 = disabled)
max_tokens: maximum number of tokens to generate (optional)
stop: list of strings where generation should stop (optional)

Example

class MySamplerConfig: temperature = 0.5 top_p = 0.9 max_tokens = 200 stop = ["\n"]

def to_dict(self):
    return {
        "temperature": self.temperature,
        "top_p": self.top_p,
        "max_tokens": self.max_tokens,
        "stop": self.stop
    }

to_dict

to_dict()

Convert sampling parameters to a dict for LLM APIs.

Prompt Building

Prompt Builder Pattern and Customization

Prompt builders are essential components that control how Jiki structures and formats the input sent to the language model (LLM). They are responsible for assembling system instructions, tool schemas, resource information, and conversation history into a coherent prompt.

Note: Jiki's PromptBuilder is responsible for assembling the overall context sent to the LLM on the client-side. This is distinct from the concept of server-side 'Prompts' defined in the Model Context Protocol (often via @mcp.prompt in FastMCP servers, see https://gofastmcp.com/servers/prompts), which are reusable message templates that an MCP server can generate upon request from a client. The PromptBuilder integrates the results of tool calls and the conversation history, potentially including messages generated by server-side prompts if the client requested them.

The core contract is defined by the IPromptBuilder interface, which specifies methods for generating the tool schema block (create_available_tools_block), the resource information block (create_available_resources_block), and constructing the complete initial system prompt (build_initial_prompt).

Jiki provides a DefaultPromptBuilder implementation that adheres to the Model Context Protocol (MCP) specifications, ensuring standard, compatible formatting. However, the prompt builder pattern allows for significant customization. Developers can create their own implementations of IPromptBuilder to tailor the prompts for specific needs, such as modifying the system instructions for different application domains, changing how tool schemas are presented to optimize for particular LLMs, or injecting custom contextual information or few-shot examples into the prompt.

Usage Example:

# Custom prompt builder
class MyPromptBuilder(IPromptBuilder):
    def create_available_tools_block(self, tools_config):
        # Custom tool block formatting
        ...

    def create_available_resources_block(self, resources_config):
        # Custom resources block formatting
        ...

    def build_initial_prompt(self, user_input, tools_config, resources_config=None):
        # Custom prompt assembly with domain-specific instructions
        ...

# Use with orchestrator
orchestrator = Jiki(
    prompt_builder=MyPromptBuilder(),
    # other parameters...
)

Bases: Protocol

Interface for building MCP prompts, including available tools and resources blocks, and the initial system prompt.

This follows the Model Context Protocol prompt guidelines

Expose tools with a block
Expose resources with a block
Include user instructions and query in a single prompt

See MCP spec on prompts: https://modelcontextprotocol.io/docs/concepts/prompts

Usage Example

builder = DefaultPromptBuilder() tools_block = builder.create_available_tools_block(tools_config) resources_block = builder.create_available_resources_block(resources_config) prompt = builder.build_initial_prompt( user_input="List all files.", tools_config=tools_config, resources_config=resources_config )

build_initial_prompt

build_initial_prompt(user_input, tools_config, resources_config=None)

Construct the full system prompt for the LLM, embedding: 1. Instructions for tool/resource usage. 2. The user's initial question. 3. Available tools block. 4. Available resources block (optional).

Parameters:

Name	Type	Description	Default
`user_input`	`str`	The question or instruction from the user (str).	required
`tools_config`	`List[Dict[str, Any]]`	Schema list for available tools.	required
`resources_config`	`Optional[List[Dict[str, Any]]]`	Optional schema list for resources.	`None`

Returns: A multi-line string representing the complete system prompt.

create_available_resources_block

create_available_resources_block(resources_config)

Generate the section for the prompt.

Parameters:

Name	Type	Description	Default
`resources_config`	`List[Dict[str, Any]]`	List of dicts, each with keys: - uri (str) - name (str) - description (str) - mimeType (str)	required

Returns: A formatted string block with resources represented in JSON.

create_available_tools_block

create_available_tools_block(tools_config)

Generate the section for the prompt.

Parameters:

Name	Type	Description	Default
`tools_config`	`List[Dict[str, Any]]`	List of dicts, each with keys: - tool_name (str) - description (str) - arguments (dict of {param: schema}) - optional 'required' list	required

Returns: A formatted string block with indenting and JSON payload.

class DynamicPromptBuilder(IPromptBuilder):
    def __init__(self):
        # Builders might store state if needed, though how it's populated
        # depends on how the Orchestrator uses the builder instance.
        self.conversation_mode = "general" 

    def create_available_tools_block(self, tools_config):
        # Standard tool block generation (implementation omitted)
        # ...
        return "<!-- Tools Block -->\n"

    def create_available_resources_block(self, resources_config):
        # Standard resource block generation (implementation omitted)
        # ...
        return "<!-- Resources Block -->\n"

    def build_initial_prompt(self, user_input, tools_config, resources_config=None, history=None):
        # NOTE: Assumes 'history' (list of messages) is passed or accessible.
        # This detail is not explicitly confirmed in the current reference.

        system_message_content = "You are a helpful assistant.\n"

        # --- Dynamic Logic Example ---
        # Check user input or history for keywords to change system prompt
        if "write code" in user_input.lower():
            system_message_content += "Focus on providing accurate and efficient code solutions.\n"
            self.conversation_mode = "coding" # Example of changing state
        elif history and any("customer support" in msg.get("content", "").lower() for msg in history if msg["role"] == "user"):
             system_message_content += "Adopt a polite and helpful customer support persona.\n"
             self.conversation_mode = "support"
        else:
            self.conversation_mode = "general"
        # --- End Dynamic Logic ---

        tools_block = self.create_available_tools_block(tools_config)
        resources_block = self.create_available_resources_block(resources_config)

        # Combine parts into the initial system prompt message dictionary
        system_prompt = {
            "role": "system",
            "content": f"{system_message_content}\n{tools_block}\n{resources_block}"
        }

        # The orchestrator would then typically append the history (if any) 
        # and the latest user_input message after this system prompt.
        # This method *itself* usually just returns the system prompt part.
        return system_prompt 

# Usage with orchestrator:
# dynamic_builder = DynamicPromptBuilder()
# orchestrator = Jiki(
#     prompt_builder=dynamic_builder,
#     # other parameters...
# )

jiki.prompts.prompt_builder.DefaultPromptBuilder

Standard PromptBuilder implementation using jiki.utils.prompt utilities.

This default builder preserves the built-in examples, instruction text, and JSON formatting rules.

Example

builder = DefaultPromptBuilder() prompt = builder.build_initial_prompt( user_input="What is the current time?", tools_config=tools_config, resources_config=resources_config )

Roots and Conversation State Management

The Model Context Protocol (MCP) includes the concept of "Roots", which allows a client application (like Jiki) to inform the MCP server about its current contextual boundaries or accessible resources. The server can leverage this information to tailor its behavior. (See MCP Docs on Roots).

While MCP Roots relate to informing the server about context, Jiki also needs a mechanism to save and restore the client-side conversation state (message history, recent tool calls) for persistence across sessions or requests. Jiki abstracts this client-side state management through the IConversationRootManager interface.

Integration: An implementation of IConversationRootManager can be provided to the JikiOrchestrator via the optional conversation_root_manager argument in its constructor (__init__).

Functionality: * Implementations of this interface are responsible for defining what client-side state needs saving and loading. * The interface requires implementing snapshot() to serialize the relevant state into a dictionary and resume() to restore the state from such a dictionary.

Default Behavior: If no custom conversation_root_manager is provided to the orchestrator, the orchestrator instance itself acts as the default manager. Its built-in snapshot() and resume() methods handle saving and loading the internal message list (_messages) and the list of tool calls from the last turn (_last_tool_calls).

Usage: The snapshot() and resume() methods (whether default or custom) are intended to be called externally by the application using the orchestrator instance. For example, a web application might call snapshot() after processing a request to save the state to a database and resume() at the start of the next request to load it. They are not automatically called by the orchestrator during its internal processing loop.

Distinction from MCP Roots: While a custom IConversationRootManager could potentially include data relevant to MCP Roots within its snapshot, the interface's primary role in Jiki's structure is client-side conversation state persistence. Communicating MCP Roots to the server is typically a responsibility of the IMCPClient implementation.

Bases: Protocol

Protocol for snapshotting conversation roots and resuming from saved contexts.

Specifications: https://modelcontextprotocol.io/docs/concepts/architecture#rootmanager

resume

resume(snapshot)

Restore conversation state from a snapshot dict.

snapshot

snapshot()

Capture the current conversation state as a snapshot dict.

Tools and Resources

These components handle the definition and management of executable actions (Tools) and informational assets (Resources) available during LLM interactions, following the Model Context Protocol (MCP) concepts.

Tools: Enabling Actions

Tools are a core concept in MCP, representing executable functions or capabilities exposed by a server. They empower the LLM, guided by the orchestrator, to interact with external systems, perform calculations, fetch dynamic data, or execute specific actions. Each tool is defined with a unique name, a description (to guide the LLM on its usage), and an input schema (typically JSON Schema) specifying the parameters the tool expects. This allows the orchestrator to validate requests and the LLM to correctly format its tool calls.

In Jiki, the jiki.tools.tool.Tool class encapsulates this definition. Tool configurations are often loaded from external sources like JSON files using helper functions.

jiki.tools.tool.Tool

Tool(name, description, arguments)

Tool class for programmatic tool definition.

Initialize a new tool.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the tool	required
`description`	`str`	A description of what the tool does	required
`arguments`	`dict`	A dictionary of argument specifications	required

Functions

to_dict

to_dict()

Convert the tool to a configuration dictionary.

Returns:

Name	Type	Description
`dict`		Tool configuration dictionary

Load tool configuration from a JSON file and return as a list of tool schemas.

Resources: Providing Context

Resources, distinct from tools, represent data assets or contextual information made available to the LLM. They are typically passive information sources rather than executable actions. Examples include relevant documentation snippets, user profiles, project files, or templates that can inform the LLM's responses or actions. Resources are generally identified by a URI, and their content can be retrieved by the client/orchestrator as needed during the conversation.

In Jiki, the management of these resources (discovery, retrieval) is abstracted through the jiki.resources.resource_manager.IResourceManager interface. Implementations of this interface handle how resources are located and their content fetched.

Bases: Protocol

Protocol for listing and reading MCP resources from a server.

Specifications: https://modelcontextprotocol.io/docs/concepts/resources

list_resources `async`

list_resources()

Retrieve all available resources as a list of metadata dicts.

Each dict includes keys: 'uri', 'name', 'description', 'mimeType'.

read_resource `async`

read_resource(uri)

Read content for a given resource URI. Returns a list of content blocks.

Each block dict includes keys: 'uri', 'mimeType', 'text'.

Logging and Debugging

Jiki provides mechanisms for detailed tracing of interactions and basic debugging output, primarily through the TraceLogger.

Structured Interaction Tracing

The main component for logging is the jiki.logging.TraceLogger. Its primary purpose is to record detailed, structured information about conversations, including prompts, LLM responses, tool calls, and results. Crucially, it aims to capture this information in a format consistent with the Model Context Protocol (MCP) interaction trace standards. This structured format makes the logs suitable for detailed analysis, debugging complex interaction flows, and potentially generating training data for downstream tasks like reinforcement learning.

The TraceLogger works by accumulating complete interaction traces. Each trace typically represents a full turn or significant segment of the conversation. The log_complete_trace method is used to record these complete traces, often incorporating intermediate events logged via log_event. These accumulated traces are stored in memory and can be persisted explicitly by calling the save_all_traces method, which writes them to a JSON or JSONL file.

Basic Debug Output

For simpler, less structured debugging needs, the TraceLogger also provides a debug method. This method currently offers basic functionality, simply printing the provided debug message directly to the standard error stream (stderr). This is distinct from the structured tracing mechanism.

jiki.logging.TraceLogger

TraceLogger(log_dir='interaction_traces')

Logger for recording structured conversation events and interaction traces. This logger is designed to be general-purpose. Downstream systems (like an RL trainer) can use this to log rich interaction data from Jiki.

Functions

debug

debug(message, **kwargs)

Log a debug message (currently prints to stderr).

get_current_traces

get_current_traces()

Get a copy of all accumulated traces from the current session.

Returns:

Type	Description
`List[Dict[str, Any]]`	List[Dict[str, Any]]: A list of trace dictionaries.

log_complete_trace

log_complete_trace(trace_data)

Log a complete interaction trace. The trace_data dictionary is augmented with a timestamp, a default reward field (if not present), and any accumulated events, then stored.

Parameters:

Name	Type	Description	Default
`trace_data`	`Dict[str, Any]`	Dictionary containing the primary trace information.	required

log_event

log_event(event)

Append a structured event to the internal events list. These events are typically aggregated into a complete trace.

save_all_traces

save_all_traces(filepath=None)

Save all accumulated traces to a file. Defaults to a timestamped .jsonl file in the log_dir.

Parameters:

Name	Type	Description	Default
`filepath`	`Optional[str]`	Optional path to save the traces. If None, a default is used.	`None`

Utility Functions

Jiki provides several utility functions to assist with common tasks involved in processing LLM interactions, managing context, and preparing output.

`clean_output`

This function is designed to post-process the raw text generated by the LLM before it is presented to the end-user. It removes internal Jiki/MCP tags (such as <mcp_tool_call>, <mcp_tool_result>, <mcp_available_tools>, and <Assistant_Thought>) that are used during the interaction logic but are not meant for final display. It also normalizes whitespace by trimming leading/trailing spaces and collapsing multiple consecutive newlines into double newlines for better readability.

Remove MCP-related tags and normalize whitespace for final assistant display.

`trim_context`

LLMs have finite context windows. This utility helps manage the conversation history to ensure it fits within a specified token limit (max_tokens). It operates directly (in-place) on the list of message dictionaries. Its strategy is to always preserve the first message (typically the system prompt) and then remove older messages (starting from the second message) one by one until the total token count, as measured by the provided num_tokens function (often jiki.utils.token.count_tokens), is below the max_tokens threshold. It guarantees that at least two messages (the system prompt and the most recent message) remain.

Trim the conversation context in-place to ensure it fits within max_tokens. Always preserve the first system message and keep at least two messages.

`count_tokens`

Accurately estimating the number of tokens used by the conversation history is crucial for context management. This function calculates the token count for a given list of messages, specific to a particular model (model_name). It leverages the tiktoken library when available for precise counting (especially for OpenAI models), including per-message overhead. If tiktoken is not installed or the model is unknown to it, it falls back to a simple character-based heuristic (approximately 4 characters per token). The output of this function is typically used as the input to the trim_context utility's num_tokens argument.

Count the number of tokens in a list of messages for a given model. Uses tiktoken when available, otherwise falls back to a character-based heuristic.

Complete Module Reference

This reference document has detailed the core components, interfaces, and configuration options available in the Jiki framework. For specifics on individual classes and functions, please refer to the relevant sections above. For a comprehensive view of all modules and their contents, exploring the source code directly is recommended.

Jiki API Reference

Core Components

Factory Function

The Jiki() Factory Pattern

Orchestrator

JikiOrchestrator Architecture and Functionality

Usage Example

jiki.orchestrator.JikiOrchestrator

Functions

build_initial_prompt

create_available_tools_block

process_user_input async

resume

snapshot

Detailed Response

Working with Detailed Responses

Usage Example:

jiki.models.response.DetailedResponse

jiki.models.response.ToolCall

MCP Client Components

Jiki Client

JikiClient Architecture and Role

Usage Example:

jiki.mcp_client.JikiClient

Functions

discover_tools async

execute_tool_call async

get_interaction_traces

initialize async

list_resources async

list_roots async

Returns

read_resource async

send_roots_list_changed async

MCP Client Transports

Client Interfaces

Purpose and Structure

Example Skeleton (Conceptual)

discover_tools async

execute_tool_call async

Configuration and Customization

Sampling Configuration

Usage Example

jiki.sampling.SamplerConfig dataclass

Functions

to_dict

to_dict

Prompt Building

Prompt Builder Pattern and Customization

Usage Example:

build_initial_prompt

create_available_resources_block

create_available_tools_block

jiki.prompts.prompt_builder.DefaultPromptBuilder

Roots and Conversation State Management

resume

snapshot

Tools and Resources

Tools: Enabling Actions

jiki.tools.tool.Tool

Functions

to_dict

Resources: Providing Context

list_resources async

read_resource async

Logging and Debugging

Structured Interaction Tracing

Basic Debug Output

jiki.logging.TraceLogger

Functions

debug

get_current_traces

log_complete_trace

log_event

save_all_traces

Utility Functions

clean_output

trim_context

count_tokens

Complete Module Reference

process_user_input `async`

discover_tools `async`

execute_tool_call `async`

initialize `async`

list_resources `async`

list_roots `async`

read_resource `async`

send_roots_list_changed `async`

discover_tools `async`

execute_tool_call `async`

jiki.sampling.SamplerConfig `dataclass`

list_resources `async`

read_resource `async`

`clean_output`

`trim_context`

`count_tokens`