janus.metrics.metric#

Attributes#

log

Functions#

metric([name, help, use_reference])

Returns a decorator to add a given metric to the cli

apply_function_pairs(pairs, function, progress, ...)

apply_function_strings(strings, function, progress, ...)

Module Contents#

janus.metrics.metric.log#
janus.metrics.metric.metric(name=None, help=None, use_reference=True)#

Returns a decorator to add a given metric to the cli

Metrics must follow the format (src_str, cmp_str, **other_params)

Parameters:

  • name (None | str) – The name of the metric. If None, the function name is used.

  • help (None | str) – The help text for the metric.

  • use_reference (bool) – Whether the metric requires a reference string.

Returns:

The decorator function.

Return type:

Callable

janus.metrics.metric.apply_function_pairs(pairs, function, progress, language, llm, token_limit, model_cost, *args, **kwargs)#
janus.metrics.metric.apply_function_strings(strings, function, progress, language, llm, token_limit, model_cost, *args, **kwargs)#