llmp.components.optimizer.optimizer.Optimizer

class llmp.components.optimizer.optimizer.Optimizer(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None)

Attributes

INSTRUCTION_TEST_SIZE
MIN_EXAMPLES
PROMPT_SAMPLE_SIZE
RUN_PER_SAMPLE
SELECT_MODE
TEST_SIZE
test_set_ids

Methods

__init__(job[, test_set])
evaluate(job_settings[, num_workers])
get_progress_bar(length, description[, sub, ...])
optimize([mode, metric])	Optimize the prompts and examples for a specific job.
prepare_job()

__init__(job: JobRecord, test_set: list[llmp.data_model.example_record.ExampleRecord] = None)

evaluate(job_settings: list[dict], num_workers: int = 5, **kwargs)

get_progress_bar(length: int, description: str, sub: bool = False, desc_length: int = 40, leave: bool = True)

optimize(mode: str = 'random', metric: str = 'accuracy')

Optimize the prompts and examples for a specific job.

prepare_job()