Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/build_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ jobs:
run: |
mkdir docker_build
git clone [email protected]:allenai/helios.git docker_build/helios
git -C docker_build/helios checkout f0a63f190b0f99d9c503249daf7e3e47bbd4792a
echo "helios @ /opt/rslearn_projects/docker_build/helios/" >> requirements-extra.txt

# Same thing for olmoearth_run repository.
Expand All @@ -74,7 +75,7 @@ jobs:
- name: Clone olmoearth_run and update requirements-extra.txt.
run: |
git clone [email protected]:allenai/olmoearth_run.git docker_build/olmoearth_run
echo "earth-system-run @ /opt/rslearn_projects/docker_build/olmoearth_run/" >> requirements-extra.txt
echo "olmoearth_run @ /opt/rslearn_projects/docker_build/olmoearth_run/" >> requirements-extra.txt

- name: Build and push Docker image
id: build-push
Expand Down
38 changes: 0 additions & 38 deletions esrun_data/sample/esrun.yaml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Local esrun for Nandi:
Local olmoearth_run for Nandi:
```bash
export EXTRA_FILES_PATH=/weka/dfive-default/helios/checkpoints
export DATASET_PATH=/weka/dfive-default/yawenz/datasets/scratch_v5/dataset_0
Expand All @@ -7,6 +7,6 @@ export TRAINER_DATA_PATH=/weka/dfive-default/yawenz/test/nandi
export WANDB_PROJECT=2025_10_03_nandi_crop_type
export WANDB_NAME=nandi_crop_type_segment_helios_base_S2_S1_ts_ws4_ps1_bs8_add_annotations_2
export WANDB_ENTITY=eai-ai2
python -m rslp.main esrun esrun --config_path esrun_data/nandi/ --scratch_path /weka/dfive-default/yawenz/datasets/scratch_v5/ --checkpoint_path /weka/dfive-default/yawenz/test/checkpoints/last_rewritten.ckpt
python -m rslp.main olmoearth_run olmoearth_run --config_path olmoearth_run_data/nandi/ --scratch_path /weka/dfive-default/yawenz/datasets/scratch_v5/ --checkpoint_path /weka/dfive-default/yawenz/test/checkpoints/last_rewritten.ckpt
```
Note that the original `task_name` has been converted from `crop_type_classification` to `class` in the checkpoint, and this checkpoint is also available at `gs://earth-system-run-dev/models/4edd1efb-b645-44c3-8d7a-5cc2abbbcc46/stage_0/checkpoint.ckpt`.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ inference_results_config:

partition_strategies:
partition_request_geometry:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 0.25

prepare_window_geometries:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 64
output_projection:
Expand All @@ -55,10 +55,10 @@ partition_strategies:

postprocessing_strategies:
process_dataset:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_partition:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_window:
class_path: esrun.runner.tools.postprocessors.noop_raster.NoopRaster
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster
File renamed without changes.
88 changes: 44 additions & 44 deletions esrun_data/sample/README.md → olmoearth_run_data/sample/README.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@
},
"id": null,
"properties": {
"es_annotations_task_id": "164679b9-04ed-5b35-b438-9677104067fc",
"es_end_time": "2024-02-24 05:57:04+00:00",
"es_start_time": "2024-02-24 05:57:00+00:00"
"oe_annotations_task_id": "164679b9-04ed-5b35-b438-9677104067fc",
"oe_end_time": "2024-02-24 05:57:04+00:00",
"oe_start_time": "2024-02-24 05:57:00+00:00"
},
"type": "Feature"
}
Expand Down
File renamed without changes.
File renamed without changes.
38 changes: 38 additions & 0 deletions olmoearth_run_data/sample/olmoearth_run.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
partition_strategies:
partition_request_geometry:
class_path: olmoearth_run.runner.tools.partitioners.noop_partitioner.NoopPartitioner
init_args:

prepare_window_geometries:
class_path: olmoearth_run.runner.tools.partitioners.fixed_window_partitioner.FixedWindowPartitioner
init_args:
window_size: 128 # intended to be a pixel value

postprocessing_strategies:
process_dataset:
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster

process_partition:
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster

process_window:
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster

window_prep:
sampler:
class_path: olmoearth_run.runner.tools.samplers.noop_sampler.NoopSampler
labeled_window_preparer:
class_path: olmoearth_run.runner.tools.labeled_window_preparers.point_to_pixel_window_preparer.PointToPixelWindowPreparer
init_args:
window_resolution: 10.0
data_splitter:
class_path: olmoearth_run.runner.tools.data_splitters.random_data_splitter.RandomDataSplitter
init_args:
train_prop: 0.8
val_prop: 0.2
test_prop: 0.0
seed: 42
label_layer: "labels"
label_property: "category"
group_name: "post_random_split"
split_property: "split"
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
partition_strategies:
partition_request_geometry:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 0.15
output_projection:
Expand All @@ -12,7 +12,7 @@ partition_strategies:
use_utm: true

prepare_window_geometries:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 2048
output_projection:
Expand All @@ -25,10 +25,10 @@ partition_strategies:

postprocessing_strategies:
process_dataset:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_partition:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_window:
class_path: esrun.runner.tools.postprocessors.noop_raster.NoopRaster
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
partition_strategies:
partition_request_geometry:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 0.15
output_projection:
Expand All @@ -12,7 +12,7 @@ partition_strategies:
use_utm: true

prepare_window_geometries:
class_path: esrun.runner.tools.partitioners.grid_partitioner.GridPartitioner
class_path: olmoearth_run.runner.tools.partitioners.grid_partitioner.GridPartitioner
init_args:
grid_size: 2048
output_projection:
Expand All @@ -25,13 +25,13 @@ partition_strategies:

postprocessing_strategies:
process_dataset:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_partition:
class_path: esrun.runner.tools.postprocessors.combine_geotiff.CombineGeotiff
class_path: olmoearth_run.runner.tools.postprocessors.combine_geotiff.CombineGeotiff

process_window:
class_path: esrun.runner.tools.postprocessors.noop_raster.NoopRaster
class_path: olmoearth_run.runner.tools.postprocessors.noop_raster.NoopRaster

inference_results_config:
data_type: RASTER
2 changes: 1 addition & 1 deletion requirements-helios.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
helios @ git+https://github.com/allenai/helios.git@main
helios @ git+https://github.com/allenai/helios.git@f0a63f190b0f99d9c503249daf7e3e47bbd4792a
2 changes: 1 addition & 1 deletion requirements-olmoearth_run.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
earth-system-run @ git+https://github.com/allenai/olmoearth_run.git@develop
olmoearth_run @ git+https://github.com/allenai/olmoearth_run.git@develop
35 changes: 0 additions & 35 deletions rslp/esrun/README.md

This file was deleted.

9 changes: 0 additions & 9 deletions rslp/esrun/__init__.py

This file was deleted.

2 changes: 1 addition & 1 deletion rslp/nandi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,4 +98,4 @@ Added AEF embeddings into `20250625` dataset.

### 2025-09-16

Worked on ES-run inference, see `esrun_data/nandi` for more details.
Worked on olmoearth_run inference, see `olmoearth_run_data/nandi` for more details.
35 changes: 35 additions & 0 deletions rslp/olmoearth_run/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
Here is example:

```
python -m rslp.main olmoearth_run olmoearth_run --config_path olmoearth_run_data/satlas/solar_farm/ --scratch_path /tmp/scratch/
```

So in `olmoearth_run/satlas/solar_farm/` we have:

- `dataset.json`: the rslearn dataset configuration file.
- `model.yaml`: the rslearn model configuration file.
- `olmoearth_run.yaml`: new YAML file containing oerun pre/post processing config.
- `prediction_request_geometry.geojson`: the GeoJSON input to the olmoearth_run partition and window generation.


In the `olmoearth_run_data/sample` directory, we can also run training window preparation, which
depends on:

- `dataset.json`: the rslearn dataset configuration file.
- `olmoearth_run.yaml`: new YAML file containiner the window_prep config
- `annotation_features.geojson`: annotation geojson FeatureCollection exported from Studio
- `annotation_task_features.geojson`: the Studio task geojson Features corresponding to the above

Run with:

```
python -m rslp.main olmoearth_run prepare_labeled_windows \
--project_path olmoearth_run_data/sample \
--scratch_path /tmp/scratch
```

to produce a new dataset at:

```
/tmp/scratch/dataset
```
9 changes: 9 additions & 0 deletions rslp/olmoearth_run/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""olmoearth_run pipeline."""

from .olmoearth_run import olmoearth_run, one_stage, prepare_labeled_windows

workflows = {
"olmoearth_run": olmoearth_run,
"one_stage": one_stage,
"prepare_labeled_windows": prepare_labeled_windows,
}
Loading
Loading