Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add graph creation functionality using weather-model-graphs #83

Open
3 tasks
sadamov opened this issue Oct 28, 2024 · 1 comment
Open
3 tasks

Add graph creation functionality using weather-model-graphs #83

sadamov opened this issue Oct 28, 2024 · 1 comment
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed

Comments

@sadamov
Copy link
Collaborator

sadamov commented Oct 28, 2024

Overview

Suggestion to add functionality to create and save different graph architectures using the weather-model-graphs package. The implementation includes support for three graph types (currently):

  • Keisler graphs (single-scale mesh)
  • GraphCast (multi-scale)
  • Oskarsson hierarchical graphs

Reference
#66 (comment)

New Features

  • Script to generate graphs from datastore xy coordinates
  • Integration with weather-model-graphs package
  • Support for multiple datastore types (MDP, NPYFilesMEPS)
  • Graph serialization to PyTorch format

Implementation Details

from pathlib import Path
import weather_model_graphs as wmg
from neural_lam.datastore import init_datastore

def create_graphs(datastore):
    xy = datastore.get_xy("state", stacked=False)
    graphs = {
        'keisler': wmg.create.archetype.create_keisler_graph(xy_grid=xy),
        'graphcast': wmg.create.archetype.create_graphcast_graph(xy_grid=xy),
        'hierarchical': wmg.create.archetype.create_oscarsson_hierarchical_graph(xy_grid=xy)
    }
    return graphs

Known Issues

  1. Graph serialization with wmg.save.to_pyg() not working correctly
  2. Need to determine correct edge/node feature naming convention

TODO

  • Expand/simplify PyG serialization
  • Adapt tests for graph creation with wgm
  • Document graph properties and usage

Questions

  1. What's the preferred format for storing generated graphs?
  2. Should we include visualization tools?
  3. How closely do we want to interate wgm into nl (slang for neural-lam, obviously)
@sadamov sadamov added documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed labels Oct 28, 2024
@joeloskarsson
Copy link
Collaborator

joeloskarsson commented Oct 31, 2024

I will take this on, as there is a need to more tightly integrate neural-lam and wmg for #84 . Or, in particular, neural-lam will have to call wmg with specific arguments, so we can add a dependency from neural-lam to wmg and then graph-creation scripts in neural-lam that call wmg.

For #84 this will be needed to impose a specific ordering between interior grid nodes and boundary grid nodes. By calling wmg directly from neural-lam we can assure that these calls always place all interior nodes before all boundary nodes.

@joeloskarsson joeloskarsson self-assigned this Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants