llmp.services.job_manager.JobManager

class llmp.services.job_manager.JobManager(config: dict)[source]

Methods

__init__(config)

create_job(job_name, **kwargs)

Create a new job.

delete_job(signature)

Delete a specific job.

evaluate_job_performance(job_id)

Evaluate the performance metrics for a specific job.

generate_examples_for_job(job_id, target_count)

Generate additional examples for a specific job.

generate_instruction(job, **kwargs)

Generate an instruction for a specific job.

generate_output(job, input_data[, ...])

Generate output for a specific input.

get_job(signature)

Retrieve details for a specific job.

get_job_by_input_output_model(input_model, ...)

Retrieve a job by input/output model.

get_job_metrics(job_id)

Retrieve metrics for a specific job.

human_verify_example(example)

Submit an example for human verification and get the result.

log_action(action, job_id)

Log a specific action related to a job.

optimize_job(*args, **kwargs)

Run the optimization process for a job, including generating examples and refining instructions.

update_job(job)

Update details of a specific job.
__init__(config: dict)[source]
create_job(job_name: str, **kwargs) → JobRecord[source]

Create a new job.

Generate a new job with the provided signature and kwargs. If not provided, the instruction will be generated automatically.

delete_job(signature: str)[source]

Delete a specific job.

evaluate_job_performance(job_id: str) → Dict[str, float][source]

Evaluate the performance metrics for a specific job.

generate_examples_for_job(job_id: str, target_count: int)[source]

Generate additional examples for a specific job.

generate_instruction(job: JobRecord, **kwargs) → str[source]

Generate an instruction for a specific job.

generate_output(job: JobRecord, input_data: dict, generator_type: str = 'default', **kwargs)[source]

Generate output for a specific input.

get_job(signature: str) → JobRecord[source]

Retrieve details for a specific job.

get_job_by_input_output_model(input_model: str, output_model: str) → JobRecord[source]

Retrieve a job by input/output model.

get_job_metrics(job_id: str) → Dict[str, Any][source]

Retrieve metrics for a specific job.

human_verify_example(example: ExampleRecord) → bool[source]

Submit an example for human verification and get the result.

log_action(action: str, job_id: str)[source]

Log a specific action related to a job.

optimize_job(*args, **kwargs)[source]

Run the optimization process for a job, including generating examples and refining instructions.

update_job(job: JobRecord)[source]

Update details of a specific job.