Skip to main content

Create Evaluation

POST 
/evaluations

Creates a new comparison table document Raises: HTTPException: description Returns: description

Request​

Body

required

    app_id App Id (string)required
    variant_ids string[]required
    evaluators_configs string[]required
    testset_id Testset Id (string)required

    rate_limit

    object

    required

    batch_size Batch Size (integer)required
    max_retries Max Retries (integer)required
    retry_delay Retry Delay (integer)required
    delay_between_batches Delay Between Batches (integer)required

    lm_providers_keys

    object

    property name* string
    correct_answer_column Correct Answer Column (string)

Responses​

Successful Response

Schema

  • Array [

    • id Id (string)required
    app_id App Id (string)required
    user_id User Id (string)required
    user_username User Username (string)required
    variant_ids string[]required
    variant_names string[]required
    variant_revision_ids string[]required
    revisions string[]required
    testset_id Testset Id (string)
    testset_name Testset Name (string)

    status

    object

    required

    type Type (string)required
    value Value

    error

    object

    message Message (string)required
    stacktrace Stacktrace (string)

    aggregated_results

    object[]

    required

  • Array [

    • evaluator_config

    object

    required

    anyOf

    id Id (string)required
    name Name (string)required
    evaluator_key Evaluator Key (string)required
    settings_values object
    created_at date-timerequired
    updated_at date-timerequired

    result

    object

    required

    type Type (string)required
    value Value

    error

    object

    message Message (string)required
    stacktrace Stacktrace (string)

  • ]

    • average_cost

    object

    type Type (string)required
    value Value

    error

    object

    message Message (string)required
    stacktrace Stacktrace (string)

    total_cost

    object

    type Type (string)required
    value Value

    error

    object

    message Message (string)required
    stacktrace Stacktrace (string)

    average_latency

    object

    type Type (string)required
    value Value

    error

    object

    message Message (string)required
    stacktrace Stacktrace (string)
    created_at date-timerequired
    updated_at date-timerequired

  • ]

Loading...