Skip to content

server

BaseServer

Bases: ABC

The base class for all servers.

Servers are responsible for communicating with the underlying model.

model_name abstractmethod property 

model_name: str

The name of the model used by the server.

close abstractmethod 

close()

Close the server.

Source code in src/appl/core/server.py 
126
127
128
129
@abstractmethod
def close(self):
    """Close the server."""
    raise NotImplementedError

create 

create(
    args: GenArgs, gen_id: str, **kwargs: Any
) -> CompletionResponse

Create a CompletionResponse from the model with given arguments.

Parameters:

  • args (GenArgs) –

    The arguments for generating the response

  • gen_id (str) –

    The ID of the generation

  • **kwargs (Any, default: {} ) –

    Additional keyword arguments

Returns: The response from the model.

Source code in src/appl/core/server.py 
112
113
114
115
116
117
118
119
120
121
122
123
124
def create(self, args: GenArgs, gen_id: str, **kwargs: Any) -> CompletionResponse:
    """Create a CompletionResponse from the model with given arguments.

    Args:
        args: The arguments for generating the response
        gen_id: The ID of the generation
        **kwargs: Additional keyword arguments
    Returns:
        The response from the model.
    """
    create_args = self._get_create_args(args, **kwargs)
    results = self._create(gen_id=gen_id, **create_args)
    return results

DummyServer

Bases: BaseServer

A dummy server for testing purposes.

create 

create(
    args: GenArgs, gen_id: str, **kwargs: Any
) -> CompletionResponse

Create a CompletionResponse from the model with given arguments.

Parameters:

  • args (GenArgs) –

    The arguments for generating the response

  • gen_id (str) –

    The ID of the generation

  • **kwargs (Any, default: {} ) –

    Additional keyword arguments

Returns: The response from the model.

Source code in src/appl/core/server.py 
112
113
114
115
116
117
118
119
120
121
122
123
124
def create(self, args: GenArgs, gen_id: str, **kwargs: Any) -> CompletionResponse:
    """Create a CompletionResponse from the model with given arguments.

    Args:
        args: The arguments for generating the response
        gen_id: The ID of the generation
        **kwargs: Additional keyword arguments
    Returns:
        The response from the model.
    """
    create_args = self._get_create_args(args, **kwargs)
    results = self._create(gen_id=gen_id, **create_args)
    return results

GenArgs

Bases: BaseModel

Common arguments for generating a response from a model.

preprocess 

preprocess(
    convert_func: Callable, is_openai: bool = False
) -> dict

Convert the GenArgs into a dictionary for creating the response.

Source code in src/appl/core/server.py 
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def preprocess(self, convert_func: Callable, is_openai: bool = False) -> dict:
    """Convert the GenArgs into a dictionary for creating the response."""
    # build dict, filter out the None values
    args = self.model_dump(exclude_none=True)

    # messages
    args["messages"] = convert_func(self.messages)

    # format the tool
    tools = self.tools
    tool_format = args.pop("tool_format")
    if len(tools):
        if tool_format == "auto":
            tool_format = "openai" if is_openai else "str"
        formatted_tools = []
        for tool in tools:
            tool_str: Any = None
            if tool_format == "openai":
                tool_str = tool.openai_schema
            else:  # TODO: supports more formats
                tool_str = str(tool)
            formatted_tools.append(tool_str)
        args["tools"] = formatted_tools
    else:
        args.pop("tools", None)
    return args