Baltor Get started
Examples

From the metric to the submission

A tabular competition, broken into six steps from the metric to the submission. Each step searches the library, downloads only the file it chose and puts it where the harness reads it.

Recorded starter-catalogue results, not a live search. The current library can return different versions. Always use the digest returned by a fresh search before downloading. About these recorded results.

The task

Predict the target of a tabular competition from train.csv, check that the score holds, and write submission.csv for test.csv.

  1. Fix the metric
  2. Audit the splits
  3. Train a baseline
  4. Find where it fails
  5. Check the score holds
  6. Write the submission

Step 1 of 6

Fix the metric

Write down the competition's metric and the validation plan before any model runs.

Recorded starter-catalogue results

  1. Search

    search: freeze the success metric before measuring

    1. freeze_the_success_metric_before_measuringskill · MIT · 2.8 KB · sha256 33872b98…Chosen
    2. orient_on_a_task_and_write_its_contractsskill · MIT · 2.9 KB · sha256 a1240b9b…
  2. Download

    download freeze_the_success_metric_before_measuring

  3. Placed where the harness reads it

    Example layout

    step-1-metric/
    ├── competition.md            the task's file
    ├── train.csv                 the task's file
    └── .claude/skills/
        └── freeze-the-success-metric-before-measuring/
            └── SKILL.md          the skill this step chose

    Writes validation-plan.json for the next steps.

Step 2 of 6

Audit the splits

Check the training and test files for leakage, duplicates and label errors.

Recorded starter-catalogue results

  1. Search

    search: audit data splits for errors and leakage

    1. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 46d3ab24…Chosen
    2. analyse_errors_by_segment_and_clusterskill · MIT · 2.6 KB · sha256 9550ecde…
  2. Download

    download audit_data_splits_for_errors_and_leakage

  3. Placed where the harness reads it

    Example layout

    step-2-splits/
    ├── train.csv                 the task's file
    ├── test.csv                  the task's file
    ├── validation-plan.json      from step 1
    └── .claude/skills/
        └── audit-data-splits-for-errors-and-leakage/
            └── SKILL.md          the skill this step chose

    Writes folds.csv for the next steps.

Step 3 of 6

Train a baseline

Fit a first model and read the gap between training and validation scores.

Recorded starter-catalogue results

  1. Search

    search: read the train validation gap

    1. read_the_train_validation_gapskill · MIT · 2.8 KB · sha256 523aba19…Chosen
    2. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 46d3ab24…
  2. Download

    download read_the_train_validation_gap

  3. Placed where the harness reads it

    Example layout

    step-3-baseline/
    ├── train.csv                 the task's file
    ├── folds.csv                 from step 2
    └── .claude/skills/
        └── read-the-train-validation-gap/
            └── SKILL.md          the skill this step chose

    Writes baseline-scores.json for the next steps.

Step 4 of 6

Find where it fails

Group the validation errors by segment and look for repeated failures.

Recorded starter-catalogue results

  1. Search

    search: analyse errors by segment

    1. analyse_errors_by_segment_and_clusterskill · MIT · 2.6 KB · sha256 9550ecde…Chosen
    2. audit_data_splits_for_errors_and_leakageskill · MIT · 2.7 KB · sha256 46d3ab24…
  2. Download

    download analyse_errors_by_segment_and_cluster

  3. Placed where the harness reads it

    Example layout

    step-4-errors/
    ├── baseline-scores.json      from step 3
    ├── folds.csv                 from step 2
    └── .claude/skills/
        └── analyse-errors-by-segment-and-cluster/
            └── SKILL.md          the skill this step chose

    Writes error-report.md for the next steps.

Step 5 of 6

Check the score holds

Rerun across seeds and folds, and keep only a result that holds.

Recorded starter-catalogue results

  1. Search

    search: check that a result is stable and generalizes

    1. check_that_a_result_is_stable_and_generalizesskill · MIT · 2.7 KB · sha256 ac18ebbb…Chosen
    2. check_for_existing_work_before_buildingskill · MIT · 3.0 KB · sha256 e3613cd6…
  2. Download

    download check_that_a_result_is_stable_and_generalizes

  3. Placed where the harness reads it

    Example layout

    step-5-stability/
    ├── train.csv                 the task's file
    ├── folds.csv                 from step 2
    ├── error-report.md           from step 4
    └── .claude/skills/
        └── check-that-a-result-is-stable-and-generalizes/
            └── SKILL.md          the skill this step chose

    Writes stability.json for the next steps.

Step 6 of 6

Write the submission

Predict the test file and write the submission with a short report.

Recorded starter-catalogue results

  1. Search

    search: hand over a result its consumer can use

    1. hand_over_a_result_its_consumer_can_useskill · MIT · 2.7 KB · sha256 2a2726d3…Chosen
    2. check_that_a_result_is_stable_and_generalizesskill · MIT · 2.7 KB · sha256 ac18ebbb…
  2. Download

    download hand_over_a_result_its_consumer_can_use

  3. Placed where the harness reads it

    Example layout

    step-6-submission/
    ├── test.csv                  the task's file
    ├── stability.json            from step 5
    └── .claude/skills/
        └── hand-over-a-result-its-consumer-can-use/
            └── SKILL.md          the skill this step chose

    Writes submission.csv, the result of the task.

Recorded searches

These searches reproduce the packaged starter catalogue identified above. They are recorded examples; a fresh search can return different results and versions.

Your agent chooses

Here each step takes the first result, and each folder shows where the harness reads the skill. This page shows how each step is set up, not a competition score.

More demonstrations

See a data cleanup task step by step, or how a long task runs overnight.

Bring your own competition

Pro, $49 a month. Cancel any time.

Get started