`llmp.components.optimizer.examples`.ExampleOptimizer¶

class llmp.components.optimizer.examples.ExampleOptimizer(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None, display_progress: bool = True, max_examples_per_prompt: int = 4, debug: bool = False, **kwargs)[source]¶

Example Optimizer for a given job.

The ExampleOptimizer is used to optimize the examples for a given job.

job¶: JobRecord

debug¶: bool

display_progress¶: bool

progress_bar_config¶: dict

MIN_EXAMPLES¶

int

Type: int

TEST_SIZE¶

int

Type: int

PROMPT_SAMPLE_SIZE¶

int

Type: int

SELECT_MODE¶

str

Type: str

INSTRUCTION_TEST_SIZE¶

int

Type: int

RUN_PER_SAMPLE¶

int

Type: int

Attributes

`INSTRUCTION_TEST_SIZE`
`MIN_EXAMPLES`
`PROMPT_SAMPLE_SIZE`
`RUN_PER_SAMPLE`
`SELECT_MODE`
`TEST_SIZE`
`test_set_ids`

Methods

`__init__`(job[, test_set, display_progress, ...])
`evaluate`(job_settings, **kwargs)
`examples_to_prompt`(examples)
`get_progress_bar`(length, description[, sub, ...])
`optimize`([mode, metric])	Optimize the prompts and examples for a specific job.
`prepare_job`()

__init__(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None, display_progress: bool = True, max_examples_per_prompt: int = 4, debug: bool = False, **kwargs)[source]¶

evaluate(job_settings: list[dict], **kwargs)[source]¶

examples_to_prompt(examples: list[llmp.data_model.example_record.ExampleRecord])[source]¶

get_progress_bar(length: int, description: str, sub: bool = False, desc_length: int = 40, leave: bool = True)¶

optimize(mode: str = 'random', metric: str = 'accuracy')[source]¶: Optimize the prompts and examples for a specific job.

prepare_job()[source]¶

llmp.components.optimizer.examples.ExampleOptimizer¶

`llmp.components.optimizer.examples`.ExampleOptimizer¶