Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update information for new/updated grids #49

Closed
Esteban82 opened this issue Aug 4, 2023 · 11 comments
Closed

Update information for new/updated grids #49

Esteban82 opened this issue Aug 4, 2023 · 11 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@Esteban82
Copy link
Member

This is mainly a recordatory:

We should update the webpage for the:

  • New grids: moon and other planets
  • Updated grids: update the information.

I am thinking of adding the grid's version in the title (in this page). I would like to easly see what version are available.

@Esteban82 Esteban82 changed the title Update information Update information for new/updated grids Aug 4, 2023
@Esteban82 Esteban82 self-assigned this Aug 4, 2023
@Esteban82 Esteban82 added the documentation Improvements or additions to documentation label Aug 4, 2023
@seisman
Copy link
Member

seisman commented Aug 5, 2023

Don't forget to update the changelog (https://www.generic-mapping-tools.org/remote-datasets/changes.html).

@Esteban82
Copy link
Member Author

For the record.
Check that the file sizes is correct in the tables. Use the info of pixel-registered grids when both grids exists.
Ideally, make a script that update this from the gmt_data_server.txt file.

@Esteban82
Copy link
Member Author

I think that the docs could be improve. Like adding the versions of the grids.

@PaulWessel
Copy link
Member

There are several problems we have:

When we release an updated SRTM15+ then suddenly a bunch of tests fail. Now we have to update many tens of PostScript files. And this would happen 1-2 times a year. Not a good situation since the failures are not useful (just shows the data have changed). Possible remedies:

  1. Have a reference set (the original tiles?) in a safe place that tests can use exclusively. Now the problem is will those be mirrored on other servers and take up space? Or just pull from oceania? I think CI tests uses oceania, @seisman (?), so would not matter? If only on oceania in a parallel directory then we can use CMake settings or similar to force use of the reference-server data just for the tests. The reference-server could either have the original tiles cerated for teh first release or it could just have those used by tests and doc scripts. First solution saves space and the other takes bookkeeping. I have the space!

  2. While we can see if any of the mirrors are down or up, how can we tell if they have been updated. I think @joa-quim told us he has not updated the europe server in Portugal due to lack of space and expertise? What about the others?

@Esteban82
Copy link
Member Author

  1. Have a reference set (the original tiles?) in a safe place that tests can use exclusively.

I have the impression that for this reason we have this dir /export/gmtserver/gmt/data/server/earth/earth_relief2.1. (based on the SRTM15+_v2.1).

@PaulWessel
Copy link
Member

Hm, OK. I will play with that. May want to move it at some point but I will test locally first.

@Esteban82
Copy link
Member Author

When we release an updated SRTM15+ then suddenly a bunch of tests fail.

So, should I wait to update the SRTM15+ data?

@seisman
Copy link
Member

seisman commented Aug 23, 2023

  1. Have a reference set (the original tiles?) in a safe place that tests can use exclusively. Now the problem is will those be mirrored on other servers and take up space? Or just pull from oceania? I think CI tests uses oceania, @seisman (?), so would not matter? If only on oceania in a parallel directory then we can use CMake settings or similar to force use of the reference-server data just for the tests. The reference-server could either have the original tiles cerated for teh first release or it could just have those used by tests and doc scripts. First solution saves space and the other takes bookkeeping. I have the space!

This sounds a good idea. We can have a directory like "gmtci" in parallel with the data (mapped to the oceanic server) and test (mapped to the test server) directory. Other mirrors won't know the existence of this new server.

  1. While we can see if any of the mirrors are down or up, how can we tell if they have been updated. I think @joa-quim told us he has not updated the europe server in Portugal due to lack of space and expertise? What about the others?

We can have a GitHub Actions, that checks the dates of the file gmt_data_server.txt on all mirrors. If the file on a mirror is older than the file on the oceanic server by a few days/weeks/months, then it's very likely the mirror is out-of-dated.

@seisman
Copy link
Member

seisman commented Aug 23, 2023

  1. Have a reference set (the original tiles?) in a safe place that tests can use exclusively.

I have the impression that for this reason we have this dir /export/gmtserver/gmt/data/server/earth/earth_relief2.1. (based on the SRTM15+_v2.1).

I dont' think so. The data in the earth_relief2.1 directory is never used and actually should be removed.

@Esteban82
Copy link
Member Author

Esteban82 commented Aug 23, 2023

Here I asked about earth_relief2.1 (GenericMappingTools/gmtserver-admin#159).

@Esteban82
Copy link
Member Author

Done by #102.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

3 participants