llmp.components.optimizer.instructions.InstructionOptimizer

class llmp.components.optimizer.instructions.InstructionOptimizer(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None, display_progress: bool = True, debug: bool = False, **kwargs)

Instruction Optimizer for a given job.

The InstructionOptimizer is used to optimize the instructions for a given job. It runs an TestSet of Examples with different instructions and returns the best instruction.

Attributes

INSTRUCTION_TEST_SIZE
MIN_EXAMPLES
PROMPT_SAMPLE_SIZE
RUN_PER_SAMPLE
SELECT_MODE
TEST_SIZE
test_set_ids

Methods

__init__(job[, test_set, display_progress, ...])
evaluate(job_settings[, num_workers])
examples_to_prompt(examples)
get_progress_bar(length, description[, sub, ...])
optimize([mode, metric])	Optimize the prompts and examples for a specific job.
prepare_job()

__init__(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None, display_progress: bool = True, debug: bool = False, **kwargs)

evaluate(job_settings: list[dict], num_workers: int = 5, **kwargs)

examples_to_prompt(examples: list[llmp.data_model.example_record.ExampleRecord])

get_progress_bar(length: int, description: str, sub: bool = False, desc_length: int = 40, leave: bool = True)

optimize(mode: str = 'random', metric: str = 'accuracy')

Optimize the prompts and examples for a specific job.

prepare_job()