janus.converter.evaluate

Attributes

log

Classes

Evaluator	Evaluator
RequirementEvaluator	INCOSE Requirement Evaluator
InlineCommentEvaluator	Inline Comment Evaluator

Module Contents

janus.converter.evaluate.log

class janus.converter.evaluate.Evaluator(**kwargs)

Bases: janus.converter.converter.Converter

Evaluator

A class that performs an LLM self evaluation” “on an input target, with an associated prompt.

Current valid evaluation types: [‘incose’, ‘comments’]

Initialize the Evaluator class

Parameters:

• model – The LLM to use for translation. If an OpenAI model, the OPENAI_API_KEY environment variable must be set.

• model_arguments – Additional arguments to pass to the LLM constructor.

• max_prompts – The maximum number of prompts to try before giving up.

class janus.converter.evaluate.RequirementEvaluator(eval_items_per_request=None, **kwargs)

Bases: Evaluator

INCOSE Requirement Evaluator

A class that performs an LLM self evaluation on an input target, with an associated prompt.

The evaluation prompts are for Incose Evaluations

Initialize the Evaluator class

Parameters:

• model – The LLM to use for translation. If an OpenAI model, the OPENAI_API_KEY environment variable must be set.

• model_arguments – Additional arguments to pass to the LLM constructor.

• max_prompts – The maximum number of prompts to try before giving up.

• eval_items_per_request (int | None) –

class janus.converter.evaluate.InlineCommentEvaluator(eval_items_per_request=None, **kwargs)

Bases: Evaluator

Inline Comment Evaluator

A class that performs an LLM self evaluation on inline comments, with an associated prompt.

Initialize the Evaluator class

Parameters:

• model – The LLM to use for translation. If an OpenAI model, the OPENAI_API_KEY environment variable must be set.

• model_arguments – Additional arguments to pass to the LLM constructor.

• max_prompts – The maximum number of prompts to try before giving up.

• eval_items_per_request (int | None) –