llmp.components.example_manager.ExampleManager

class llmp.components.example_manager.ExampleManager(job: JobRecord, debug: bool = False)[source]

Initialize the ExampleManager with a job.

Parameters

  • job (JobRecord) – The job to manage the examples for.

  • debug (bool, optional) – Enable debug mode. Defaults to False.

Attributes

records

Methods

__init__(job[, debug])

Initialize the ExampleManager with a job.

add_example(record, **kwargs)

Add an example to the job.

create_examples([num_examples])

Create a given number of examples for a given job.

delete_example(job_id, example_idx)

Delete an example from a given job.

fill_examples([total_count])

Fill the example records for a given job.

get_examples(job_id)

Return all examples for a given job.

get_possible_sets(set_size[, exclude_ids])

Return all possible sets of examples.

get_test_set(test_size[, mode, exclude_ids])

Return a test set of examples.

to_engine(job_id)

Return a list of examples for a given job.

update_example(job_id, example_idx, example)

Update an example for a given job.
__init__(job: JobRecord, debug: bool = False)[source]

Initialize the ExampleManager with a job.

Parameters

  • job (JobRecord) – The job to manage the examples for.

  • debug (bool, optional) – Enable debug mode. Defaults to False.

add_example(record: ExampleRecord, **kwargs) → None[source]

Add an example to the job.

Parameters

  • record (ExampleRecord) – The example to add.

  • **kwargs – Additional keyword arguments passed to self.job.add_example().

create_examples(num_examples: int = 20, **kwargs) → None[source]

Create a given number of examples for a given job.

Created examples are added to the job.

Parameters

  • num_examples (int, optional) – The number of examples to create. Defaults to 20.

  • **kwargs – Additional keyword arguments passed to ExampleGenerator.generate().

Example

>>> manager = ExampleManager(job)
>>> manager.create_examples(20)
delete_example(job_id: UUID4, example_idx: int) → None[source]

Delete an example from a given job.

Parameters

  • job_id (UUID4) – The job id.

  • example_idx (int) – The example index.

fill_examples(total_count: int = 20, **kwargs) → None[source]

Fill the example records for a given job.

Parameters

  • total_count (int, optional) – The total number of examples to create. Defaults to 20.

  • **kwargs – Additional keyword arguments passed to self.create_examples().

get_examples(job_id: UUID4) → list[dict][source]

Return all examples for a given job.

Parameters

job_id (UUID4) – The job id.

Returns

A list of example records.

Return type

list[dict]

get_possible_sets(set_size: int, exclude_ids: list[str] = None) → list[tuple[llmp.data_model.example_record.ExampleRecord]][source]

Return all possible sets of examples.

Is used in ExampleOptimizer to test all possible combinations of existing examples for a given set_size.

Parameters

  • set_size (int) – The size of the sets.

  • exclude_ids (list[str], optional) – A list of example ids to exclude. Defaults to None.

Returns

A list of example records.

Return type

list[tuple[ExampleRecord]]

get_test_set(test_size: int, mode: str = TestSetMode.RANDOM, exclude_ids: list[str] = None) → list[llmp.data_model.example_record.ExampleRecord][source]

Return a test set of examples.

to_engine(job_id: UUID4) → list[dict][source]

Return a list of examples for a given job.

Parameters

job_id (UUID4) – The job id.

Returns

A list of example records.

Return type

list[dict]

update_example(job_id: UUID4, example_idx: int, example: dict) → None[source]

Update an example for a given job.

Parameters

  • job_id (UUID4) – The job id.

  • example_idx (int) – The example index.

  • example (dict) – The example to update.