llmp.components.generator.concurrent.MultiThreadingAsyncGenerator

class llmp.components.generator.concurrent.MultiThreadingAsyncGenerator(job: JobRecord, job_settings: list[dict], num_runs: int = 5)

Run SequentialAsyncGenerator in multiple threads.

For each thread, a SequentialAsyncGenerator is created with different job settings and runs for each sample from input_data for “num_runs” times.

Initialize the generator with a job and job settings.

Parameters

• job (JobRecord) – the job to be executed

• job_settings (dict) – the job settings to be used

• **kwargs – any - passed to load_engine_from_job

Attributes

verification_type

Return the validation type for the generator.

Methods

__init__(job, job_settings[, num_runs])	Initialize the generator with a job and job settings.
generate(input_data, **kwargs)	Generate text based on the job and input object.
log_generation(input_object, ...)	Log the generation result.

__init__(job: JobRecord, job_settings: list[dict], num_runs: int = 5)

Initialize the generator with a job and job settings.

Parameters

• job (JobRecord) – the job to be executed

• job_settings (dict) – the job settings to be used

• **kwargs – any - passed to load_engine_from_job

generate(input_data: list[dict], **kwargs) → list[list[Tuple[dict, dict]]]

Generate text based on the job and input object.

log_generation(input_object: dict, generated_object: dict, run_metrics: dict, **kwargs) → Optional[Event]

Log the generation result.

Parameters

• input_object (dict) – the input object

• generated_object (dict) – the generated object

• run_metrics (dict) – the run metrics

• **kwargs – any - passed to JobRecord.log_generation