janus.llm.models_info#

Classes#

`JanusModelProtocol`	Base class for protocol classes.
`JanusModel`	A unit of work that can be invoked, batched, streamed, transformed and composed.

Functions#

`get_available_model_names`()
`load_model`(model_id)

Module Contents#

janus.llm.models_info.log#

janus.llm.models_info.ModelType#

class janus.llm.models_info.JanusModelProtocol#

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto(Protocol[T]):
    def meth(self) -> T:
        ...

model_id: str#

model_type_name: str#

token_limit: int#

input_token_cost: float#

output_token_cost: float#

prompt_engine: type[janus.prompts.prompt.PromptEngine]#

get_num_tokens(text)#

Parameters:: text (str) –
Return type:: int

class janus.llm.models_info.JanusModel#

Bases: langchain_core.runnables.Runnable, JanusModelProtocol

A unit of work that can be invoked, batched, streamed, transformed and composed.

invoke/ainvoke: Transforms a single input into an output.
batch/abatch: Efficiently transforms multiple inputs into outputs.
stream/astream: Streams output from a single input as it’s produced.
astream_log: Streams output and selected intermediate results from an input.

Built-in optimizations:

Batch: By default, batch runs invoke() in parallel using a thread pool executor. Override to optimize batching.
Async: Methods with “a” suffix are asynchronous. By default, they execute the sync counterpart using asyncio’s thread pool. Override for native async.

All methods accept an optional config argument, which can be used to configure execution, add tags and metadata for tracing and debugging etc.

Runnables expose schematic information about their input, output and config via the input_schema property, the output_schema property and config_schema method.

LCEL and Composition#

The LangChain Expression Language (LCEL) is a declarative way to compose Runnables into chains. Any chain constructed this way will automatically have sync, async, batch, and streaming support.

The main composition primitives are RunnableSequence and RunnableParallel.

RunnableSequence invokes a series of runnables sequentially, with one Runnable’s output serving as the next’s input. Construct using the | operator or by passing a list of runnables to RunnableSequence.

RunnableParallel invokes runnables concurrently, providing the same input to each. Construct it using a dict literal within a sequence or by passing a dict to RunnableParallel.

For example,

from langchain_core.runnables import RunnableLambda

# A RunnableSequence constructed using the `|` operator
sequence = RunnableLambda(lambda x: x + 1) | RunnableLambda(lambda x: x * 2)
sequence.invoke(1) # 4
sequence.batch([1, 2, 3]) # [4, 6, 8]


# A sequence that contains a RunnableParallel constructed using a dict literal
sequence = RunnableLambda(lambda x: x + 1) | {
    'mul_2': RunnableLambda(lambda x: x * 2),
    'mul_5': RunnableLambda(lambda x: x * 5)
}
sequence.invoke(1) # {'mul_2': 4, 'mul_5': 10}

Standard Methods#

All Runnables expose additional methods that can be used to modify their behavior (e.g., add a retry policy, add lifecycle listeners, make them configurable, etc.).

These methods will work on any Runnable, including Runnable chains constructed by composing other Runnables. See the individual methods for details.

For example,

from langchain_core.runnables import RunnableLambda

import random

def add_one(x: int) -> int:
    return x + 1


def buggy_double(y: int) -> int:
    '''Buggy code that will fail 70% of the time'''
    if random.random() > 0.3:
        print('This code failed, and will probably be retried!')  # noqa: T201
        raise ValueError('Triggered buggy code')
    return y * 2

sequence = (
    RunnableLambda(add_one) |
    RunnableLambda(buggy_double).with_retry( # Retry on failure
        stop_after_attempt=10,
        wait_exponential_jitter=False
    )
)

print(sequence.input_schema.schema()) # Show inferred input schema
print(sequence.output_schema.schema()) # Show inferred output schema
print(sequence.invoke(2)) # invoke the sequence (note the retry above!!)

Debugging and tracing#

As the chains get longer, it can be useful to be able to see intermediate results to debug and trace the chain.

You can set the global debug flag to True to enable debug output for all chains:

from langchain_core.globals import set_debug
set_debug(True)

Alternatively, you can pass existing or custom callbacks to any given chain:

from langchain_core.tracers import ConsoleCallbackHandler

chain.invoke(
    ...,
    config={'callbacks': [ConsoleCallbackHandler()]}
)

For a UI (and much more) checkout LangSmith: https://docs.smith.langchain.com/

janus.llm.models_info.openai_models = ['gpt-4o', 'gpt-4o-mini', 'gpt-4', 'gpt-4-turbo', 'gpt-4-turbo-preview', 'gpt-3.5-turbo',...#

janus.llm.models_info.azure_models = ['gpt-4o', 'gpt-4o-mini', 'gpt-3.5-turbo-16k']#

janus.llm.models_info.claude_models = ['bedrock-claude-v2', 'bedrock-claude-instant-v1', 'bedrock-claude-haiku',...#

janus.llm.models_info.llama2_models = ['bedrock-llama2-70b', 'bedrock-llama2-70b-chat', 'bedrock-llama2-13b', 'bedrock-llama2-13b-chat']#

janus.llm.models_info.llama3_models = ['bedrock-llama3-8b-instruct', 'bedrock-llama3-70b-instruct']#

janus.llm.models_info.titan_models = ['bedrock-titan-text-lite', 'bedrock-titan-text-express', 'bedrock-jurassic-2-mid',...#

janus.llm.models_info.cohere_models = ['bedrock-command-r-plus']#

janus.llm.models_info.mistral_models = ['bedrock-mistral-7b-instruct', 'bedrock-mistral-large', 'bedrock-mixtral']#

janus.llm.models_info.bedrock_models#

janus.llm.models_info.all_models#

janus.llm.models_info.MODEL_TYPE_CONSTRUCTORS: dict[str, ModelType]#

janus.llm.models_info.MODEL_PROMPT_ENGINES: dict[str, Callable[Ellipsis, janus.prompts.prompt.PromptEngine]]#

janus.llm.models_info.MODEL_ID_TO_LONG_ID#

janus.llm.models_info.MODEL_DEFAULT_ARGUMENTS: dict[str, dict[str, str]]#

janus.llm.models_info.DEFAULT_MODELS#

janus.llm.models_info.MODEL_CONFIG_DIR#

janus.llm.models_info.MODEL_TYPES: dict[str, janus.prompts.prompt.PromptEngine]#

janus.llm.models_info.TOKEN_LIMITS: dict[str, int]#

janus.llm.models_info.get_available_model_names()#

Return type:: list[str]

janus.llm.models_info.load_model(model_id)#

Return type:: load_model.JanusModel

`log`
`ModelType`
`openai_models`
`azure_models`
`claude_models`
`llama2_models`
`llama3_models`
`titan_models`
`cohere_models`
`mistral_models`
`bedrock_models`
`all_models`
`MODEL_TYPE_CONSTRUCTORS`
`MODEL_PROMPT_ENGINES`
`MODEL_ID_TO_LONG_ID`
`MODEL_DEFAULT_ARGUMENTS`
`DEFAULT_MODELS`
`MODEL_CONFIG_DIR`
`MODEL_TYPES`
`TOKEN_LIMITS`