Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mpi on candide #661

Open
wants to merge 284 commits into
base: develop
Choose a base branch
from
Open

Mpi on candide #661

wants to merge 284 commits into from

Conversation

martinkilbinger
Copy link
Contributor

@martinkilbinger martinkilbinger commented Oct 31, 2023 •
edited
Loading

Summary

A number of fixes are proposed to get MPI running again (on candide):

  • Version upgrades (gcc -> 9.3.0, openmpi -> 5.0.0, numpy -> 1.22)
  • MPI job script update
  • Some small fixes (module_config_sec was not communicated to clients)
  • Module name fixes in example config file
  • Minor other fixes in modules

Need to wait for pysap and subsequent mccd update, see CEA-COSMIC/pysap#187

Reviewer Checklist

Reviewers should tick the following boxes before approving and merging the PR.

  • The PR targets the develop branch
  • The PR is assigned to the developer
  • The PR has appropriate labels
  • The PR is included in appropriate projects and/or milestones
  • The PR includes a clear description of the proposed changes
  • If the PR addresses an open issue the description includes "closes #"
  • The code and documentation style match the current standards
  • Documentation has been added/updated consistently with the code
  • All CI tests are passing
  • API docs have been built and checked at least once (if relevant)
  • All changed files have been checked and comments provided to the developer
  • All of the reviewer's comments have been satisfactorily addressed by the developer

@martinkilbinger martinkilbinger added the bug Something isn't working label Oct 31, 2023
@martinkilbinger martinkilbinger added this to the P3 re-run 2023 milestone Oct 31, 2023
@martinkilbinger martinkilbinger self-assigned this Oct 31, 2023
@martinkilbinger martinkilbinger mentioned this pull request Nov 8, 2023
Open
@martinkilbinger
Testing dockerfile with only conda env installed
07cafb4
@martinkilbinger
Fixing numpy and astropy versions
92a999c
@martinkilbinger
installation on canfar working: removed most versions from yml file; …
2b6600d 
…added init_canfar.sh script
@martinkilbinger
Dockerfile for image 0:7, 0:8
b6e5c97
@martinkilbinger
removed pysap check for example runner
d3aeccb
@martinkilbinger
Installation of sextractor and psfex with conda
bc66565
@martinkilbinger
script to call curl with canfar contained updated, adding NCORE as ar…
0b3aac6 
…gument
@martinkilbinger
added init scripts for canfar; job_sp added n_smp for further jobs
69317cb
@martinkilbinger
script to call curl for canfar container: added resources
d35bd69
@martinkilbinger
testing input numbers
d93a60e
@martinkilbinger
removed debug msg
fb83a27
@martinkilbinger
numpy error fixed with version upgrade
e33972b
@martinkilbinger
Merge branch 'science_portal_run' of github.com:martinkilbinger/shape…
edbec76 
…pipe-1 into science_portal_run
@martinkilbinger
Getting ready for exclusive-one-tile processing
9aa16be
@martinkilbinger
Added command line option to specify exclusive ID for processing
e5d0335
@martinkilbinger
Merge remote-tracking branch 'origin/exclusive' into science_portal_run
49d8880
@martinkilbinger
curl script exclusive ID
d02347e
@martinkilbinger
Dockerfile conda -> source activate
12e6b63
@martinkilbinger
Merge remote-tracking branch 'origin/docker_image' into docker_image
7d20699 
Conflicts:
	Dockerfile
	environment.yml
	scripts/sh/init_canfar.sh
@martinkilbinger
Merge branch 'docker_image' of github.com:martinkilbinger/shapepipe-1…
1d3df1a 
… into docker_image
@martinkilbinger
comment added
b05f2a5
@martinkilbinger
Updated Dockerfile
45942d5
@martinkilbinger
print msg if empty vignets
c0e5435
@martinkilbinger
doc
a7203a3
@martinkilbinger
cleaning up and adding doc strings to vignet maker
7f842a3
@martinkilbinger
added clarifying comments to vignet maker
ee4ef4f
@martinkilbinger
cleaned up curl script
1c8b4cd
@martinkilbinger
canfar scripts adopted to new env; added debug
3210b88
@martinkilbinger
shapepipe_run script adopted to new canfar env
ae97110
@martinkilbinger
setools matplotlib Agg to avoid display error
ebb40b1
@martinkilbinger
ngmix: catching missing keys in postage stamp sqlite database files
ca026ef
@martinkilbinger
combin runs script: added tile_detection
126be96
@martinkilbinger
Merge branch 'v1.4' of github.com:martinkilbinger/shapepipe-1 into v1.4
f60a5f0
@martinkilbinger
cleaned up convery pix script
08d7a7c
@martinkilbinger
summary added special, n_sh as variable
8b4619c
@martinkilbinger
debugging curl canfar
e5c8046
@martinkilbinger
clarified doc on merging star cat
c8f657a
@martinkilbinger
Added checks and improved messages for keys in FITS files
e0eb0f4
@martinkilbinger
debugging curl canfar
ba3a0d8
@martinkilbinger
Pi psfex HDU number changed
ca7f3e5
@martinkilbinger
cosmetic changes
e219400
@martinkilbinger
doc canfar: options sp_local=0,1
34edaaf
@martinkilbinger
split image config more flexible
96ec184
@martinkilbinger
init_curl script: allowing for individual -j 2, 8 (sp_local=1)
45384c9
@martinkilbinger
Merge remote-tracking branch 'upstream/develop' into 651-conda-activa…
925ae5b 
…te-fix-MK
@martinkilbinger
Merge branch '651-conda-activate-fix-MK' into v1.4
c6be9cc
@martinkilbinger
added vos
f1ed5a8
@martinkilbinger
Merge remote-tracking branch 'origin/v1.4' into tiles_202402
f79e0bf
@martinkilbinger
Merge remote-tracking branch 'origin/tiles_202402' into v1.4
e2ec9fa
@martinkilbinger
Merge remote-tracking branch 'origin/v1.4' into cosmos
161d3ca
@martinkilbinger
Merge remote-tracking branch 'origin/v1.4' into mpi
27dc9d2
@martinkilbinger
file_handler updated
805fe03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sfarrens sfarrens Awaiting requested review from sfarrens

Assignees

@martinkilbinger martinkilbinger

Labels
bug Something isn't working
Projects
ShapePipe Dev
Status: To do
Milestone
ShapePipe v1.1.0 Release
Development

Successfully merging this pull request may close these issues.
2 participants
@martinkilbinger @sfarrens