llmp.components.generator.concurrent.SequentialAsyncGenerator2

class llmp.components.generator.concurrent.SequentialAsyncGenerator2(job: JobRecord, job_settings: dict = None, num_runs: int = 5)

A faster version of SequentialAsyncGenerator.

Runs all inputs in parallel but is prone to RateLimitingErrors.

Initialize the generator with a job and job settings.

Parameters

• job (JobRecord) – the job to be executed

• job_settings (dict) – the job settings to be used

• **kwargs – any - passed to load_engine_from_job

Attributes

verification_type

Return the validation type for the generator.

Methods

__init__(job[, job_settings, num_runs])	Initialize the generator with a job and job settings.
generate(input_data, **kwargs)	Generate text based on the job and input object.
log_generation(input_object, ...)	Log the generation result.

__init__(job: JobRecord, job_settings: dict = None, num_runs: int = 5)

Initialize the generator with a job and job settings.

Parameters

• job (JobRecord) – the job to be executed

• job_settings (dict) – the job settings to be used

• **kwargs – any - passed to load_engine_from_job

generate(input_data: list[dict], **kwargs) → list[list[Tuple[dict, dict]]]

Generate text based on the job and input object.

log_generation(input_object: dict, generated_object: dict, run_metrics: dict, **kwargs) → Optional[Event]

Log the generation result.

Parameters

• input_object (dict) – the input object

• generated_object (dict) – the generated object

• run_metrics (dict) – the run metrics

• **kwargs – any - passed to JobRecord.log_generation