janus.parsers.eval_parser

Attributes

log

Classes

Eval	Mixin to provide __str__, __repr__, and __pretty__ methods. See #884 for more details.
EvaluationParser	Base class to parse the output of an LLM call.

Module Contents

janus.parsers.eval_parser.log

class janus.parsers.eval_parser.Eval(**data)

Bases: langchain_core.pydantic_v1.BaseModel

Mixin to provide __str__, __repr__, and __pretty__ methods. See #884 for more details.

__pretty__ is used by [devtools](https://python-devtools.helpmanual.io/) to provide human readable representations of objects.

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters:

data (Any) –

syntax: float

style: float

completeness: float

correctness: float

score_is_valid(v)

Parameters:

v (float | int) –

class janus.parsers.eval_parser.EvaluationParser

Bases: janus.parsers.parser.JanusParser, langchain.output_parsers.PydanticOutputParser

Base class to parse the output of an LLM call.

Output parsers help structure language model responses.

Example

class BooleanOutputParser(BaseOutputParser[bool]):
    true_val: str = "YES"
    false_val: str = "NO"

    def parse(self, text: str) -> bool:
        cleaned_text = text.strip().upper()
        if cleaned_text not in (self.true_val.upper(), self.false_val.upper()):
            raise OutputParserException(
                f"BooleanOutputParser expected output value to either be "
                f"{self.true_val} or {self.false_val} (case-insensitive). "
                f"Received {cleaned_text}."
            )
        return cleaned_text == self.true_val.upper()

    @property
    def _type(self) -> str:
        return "boolean_output_parser"

parse(text)

Parse a single string model output into some structure.

Parameters:

text (str | langchain_core.messages.BaseMessage) – String output of a language model.

Returns:

Structured output.

Return type:

str

parse_combined_output(text)

Parse the JSON object, convert keys to lowercase, filter out unexpected keys, and average the values

Parameters:

text (str) – The output text from the LLM.

Returns:

A parsed version of the text.

Return type:

str