llmp.components.generator.consensus.MajorVoteGenerator

class llmp.components.generator.consensus.MajorVoteGenerator(job: JobRecord, job_settings: dict = None, num_votes: int = 10, mode: VerificationType = VerificationType.MAJORITY_VOTE, return_event_log: bool = False, **kwargs)

Execute a job with a specific input multiple times and return the majority vote output.

Initialize the generator with a job and job settings.

Parameters

• job – JobRecord - the job to be executed

• job_settings – dict - the job settings to be used

• num_votes – int - number of votes to be collected

• mode – VerificationType - the verification type to be used (majority vote, majority grade, human verified)

• return_event_log – bool - whether to return the event log

• **kwargs – any - passed to AsyncGenerator

Examples

>>> from llmp.components.generator import MajorVoteGenerator
>>> from llmp.data_model import JobRecord
>>> from llmp.data_model.job_record import load_job_from_file

Attributes

verification_type

Return the validation type for the generator.

Methods

__init__(job[, job_settings, num_votes, ...])	Initialize the generator with a job and job settings.
generate(inputs, **kwargs)	Generate an output from input data
log_generation(input_object, ...)	Log the generation result.

__init__(job: JobRecord, job_settings: dict = None, num_votes: int = 10, mode: VerificationType = VerificationType.MAJORITY_VOTE, return_event_log: bool = False, **kwargs)

Initialize the generator with a job and job settings.

Parameters

• job – JobRecord - the job to be executed

• job_settings – dict - the job settings to be used

• num_votes – int - number of votes to be collected

• mode – VerificationType - the verification type to be used (majority vote, majority grade, human verified)

• return_event_log – bool - whether to return the event log

• **kwargs – any - passed to AsyncGenerator

Examples

>>> from llmp.components.generator import MajorVoteGenerator
>>> from llmp.data_model import JobRecord
>>> from llmp.data_model.job_record import load_job_from_file

generate(inputs: dict, **kwargs) → Union[dict, Tuple[dict, Event]]

Generate an output from input data

Runs an AsyncGenerator with a specific input multiple times and returns the majority vote output.

Parameters

• inputs (dict) – Input data for the job. With prompt placeholder as keys

• **kwargs –

Returns

Union[dict, Tuple[dict, Event]]

log_generation(input_object: dict, generated_object: dict, run_metrics: dict, **kwargs) → Optional[Event]

Log the generation result.

Parameters

• input_object (dict) – the input object

• generated_object (dict) – the generated object

• run_metrics (dict) – the run metrics

• **kwargs – any - passed to JobRecord.log_generation