Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDRSv2.1 processing script could be more efficient #38

Open
kasra-keshavarz opened this issue Feb 14, 2024 · 2 comments
Open

RDRSv2.1 processing script could be more efficient #38

kasra-keshavarz opened this issue Feb 14, 2024 · 2 comments
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@kasra-keshavarz
Copy link
Collaborator

Reported by: Sujata Budhathoki @sujata91

The process time for RDRSv2.1 is inefficient as it takes nearly 3 hours for any domain. This could be much more efficient and be done in a few minutes for any spatial/temporal domain of interest.

Solutions that come to mind:

  1. first subsetting needs to be done and then the modifications on the lon value take place,
  2. spatial subsetting to be done using ncks index values, rather than cdo's sellonlatbox method.
@kasra-keshavarz kasra-keshavarz added documentation Improvements or additions to documentation enhancement New feature or request labels Feb 14, 2024
@kasra-keshavarz kasra-keshavarz self-assigned this Feb 14, 2024
@ShervanGharari
Copy link

one makes sense, I thought datatool was designed this way (first subsetting of variables and then subsetting of the domain).
for the second, I believe we looked into that but decided to continue with cdo, if the decision is to go back to the index, I would suggest having the index search in a fully separate function so WRF and all the other datasets also use the same function and are easier to control. Complications can arise from the order of lat, lon, etc ( I believe the reason why we stayed with cdo).
I would also rename the issue to a more general task possibility of subsetting for domain based on index. This is general and not RDRS specific in my point of view.
Given the fact that the efficiency is mentioned here, I would like to turn this back to @sujata91 on the efficiency of MESH model in reading the input files as well. I believe this is communicated well with MESH team but no solid action is performed.

@kasra-keshavarz
Copy link
Collaborator Author

Partially resolved with #47

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants