Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the BIDS-like saving to align with template directory #77

Merged
merged 56 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
448b3b3
Update features to save in individual files
ibevers Jul 1, 2024
7825f9f
Fix dir doesn't exist bug
ibevers Jul 1, 2024
c13f4bf
Update feature saving to conform to BIDS and use pathlib
ibevers Jul 7, 2024
a59a307
Add to-do and automated style fixes
ibevers Aug 21, 2024
6e94648
Move data to src/b2aiprep
ibevers Aug 21, 2024
ab7883f
Add workflow for automatically updating bids-like template CHANGES.md
ibevers Aug 21, 2024
27a3710
Change summer_school to bids_like
ibevers Aug 22, 2024
50ec90b
Add copying of README.md and CHANGES.md from template
ibevers Aug 22, 2024
d614903
Move directory initialization to separ
ibevers Aug 22, 2024
37657b4
Update dataset_description.json
ibevers Aug 23, 2024
a93eaf1
Add dataset_description.json to the data saving
ibevers Aug 23, 2024
98647cb
Add RedCap column names file to the data dir
ibevers Aug 23, 2024
ab63fad
Add first iteration of participants.json
ibevers Aug 23, 2024
8ec4d6b
Reorganize files
ibevers Aug 23, 2024
09e5630
Remove redundancy in import
ibevers Aug 23, 2024
ffcb3ae
Add create participants.tsv functionality
ibevers Aug 26, 2024
3388b44
Add template phenotype JSON files
ibevers Aug 26, 2024
78abe11
Add phenotype tsv files to template
ibevers Aug 26, 2024
52078c3
Replace flake8 with ruff
ibevers Aug 26, 2024
18e6da6
Add utils and util tests
ibevers Aug 26, 2024
2880a90
Move bids-like template to resources and add phenotype files to template
ibevers Aug 26, 2024
e79206d
Add ruff fixes to notebooks
ibevers Aug 26, 2024
0286159
Add construct all tsvs from jsons function to utils with test
ibevers Aug 26, 2024
017ab5f
Add phenotype tsv file generation
ibevers Aug 26, 2024
f945852
Remove duplicated template directory
ibevers Aug 27, 2024
354dc6d
Clean up and clarify comment
ibevers Aug 28, 2024
5d51cfe
Move recording tasks and acoustic tasks to audio dir
ibevers Aug 28, 2024
8fbb2c5
Remove beh directory and redundant per-participant files
ibevers Aug 28, 2024
17df9a6
Replace task with recording name where applicable
ibevers Aug 28, 2024
d9ce3b5
Remove redundant information from audio file names
ibevers Aug 28, 2024
0268b77
Update JSON suffix to include 'metadata'
ibevers Aug 28, 2024
ace683a
Add sessions.tsv and remove session jsons
ibevers Aug 28, 2024
0e389fa
Merge in bids-like-updates
ibevers Aug 29, 2024
43b5bbb
Add separate feature saving changes manually
ibevers Aug 29, 2024
95ee66a
Fix senselab integration bugs
ibevers Aug 29, 2024
de7d5b4
Fix incorrect import
ibevers Aug 29, 2024
aafa96a
Fix out of date path in test
ibevers Aug 29, 2024
c8a9080
Fix transcript saving
ibevers Aug 29, 2024
8442083
Make default transcription whisper tiny
ibevers Aug 29, 2024
a158087
Add parameter for transcription model size
ibevers Aug 30, 2024
fa37d34
Add n_cores as parameter for feature extraction
ibevers Aug 30, 2024
41891eb
Add batching for feature extraction
ibevers Aug 30, 2024
287ec2d
Remove argument parsing from prepare.py
ibevers Aug 30, 2024
c0e7b87
Remove TODO comment
ibevers Sep 11, 2024
4e63af1
Add 1000 row synthetic data
ibevers Sep 12, 2024
3794ab9
Add test for each bids_like_data function
ibevers Sep 12, 2024
bd36e71
Add CLI test
ibevers Sep 13, 2024
be43d6c
Add descriptions to participants.json
ibevers Sep 13, 2024
073e11a
Sort participants.json
ibevers Sep 16, 2024
ed7c5b0
Update test_prepare.py
ibevers Sep 16, 2024
c2dd9f5
Update phenotype descriptions
ibevers Sep 16, 2024
7dcc29e
Require senselab>=0.12.0
ibevers Sep 16, 2024
2fb2d0e
Add flag to exclude sensitive data: .wav and transcripts
ibevers Sep 16, 2024
71582c9
Add license and update dataset_description.json
ibevers Sep 16, 2024
d4a4563
Update test to use with_sensitive parameter
ibevers Sep 16, 2024
c928e92
Move data directory out of src
ibevers Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/bids_changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Update BIDS-like data template dir CHANGES.md

on:
push:
paths:
- 'src/b2aiprep/data'
workflow_dispatch: # This allows manual triggering of the workflow

jobs:
update-changelog:
runs-on: ubuntu-latest

steps:
# Step 1: Check out the repository
- uses: actions/checkout@v3

# Step 2: Set up Node.js environment (required for auto-changelog)
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '16'

# Step 3: Install auto-changelog or any other changelog tool
- name: Install auto-changelog
run: npm install -g auto-changelog

# Step 4: Generate the changelog
- name: Generate changelog
run: auto-changelog -p

# Step 5: Commit and push the updated CHANGES.md file
- name: Commit and push changes
run: |
git config user.name "GitHub Actions"
git config user.email "[email protected]"
git add src/b2aiprep/data/b2ai-data-bids-like-template/CHANGES.md
git commit -m "Update b2ai-data-bids-like-template/CHANGES.md"
git push origin HEAD:main
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -146,5 +146,5 @@ cython_debug/
# guide data
guide/data/

.DS_Store
*.DS_Store
cache-wf/
15 changes: 8 additions & 7 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,30 @@ ci:

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v4.6.0
hooks:
- id: check-added-large-files
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 24.3.0
rev: 24.8.0
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.6.2 # Replace with the latest version available
hooks:
- id: flake8
exclude: ^(src/b2aiprep/_version\.py|versioneer\.py)$
- id: ruff
args: ["--fix"]
exclude: ^(src/b2aiprep/_version\.py|versioneer\.py)$
- repo: https://github.com/PyCQA/isort
rev: 5.13.2
hooks:
- id: isort
exclude: ^(src/b2aiprep/_version\.py|versioneer\.py)$
args: ["--profile", "black"]
- repo: https://github.com/codespell-project/codespell
rev: v2.2.6
rev: v2.3.0
hooks:
- id: codespell
args: ["--skip=*.json"]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
{
"resourceType": "QuestionnaireResponse",
"id": "01401050-e9dd-486e-b0d4-0001ee7f861d-airwaystenosis",
"questionnaire": "https://kind-lab.github.io/vbai-fhir/Questionnaire-drespairwaystenosisschema",
"status": "completed",
"item": [
{
"linkId": "record_id",
"answer": [
{
"valueString": "01401050-e9dd-486e-b0d4-0001ee7f861d"
}
]
},
{
"linkId": "diagnosis_as_pft"
},
{
"linkId": "diagnosis_as_gsd"
},
{
"linkId": "diagnosis_as_gsd_yes___bronchoscopy"
},
{
"linkId": "diagnosis_as_gsd_yes___ct"
},
{
"linkId": "diagnosis_as_ds"
},
{
"linkId": "diagnosis_as_ds_bvfi_gs"
},
{
"linkId": "diagnosis_as_etiology"
},
{
"linkId": "diagnosis_as_etiology_auto_immune"
},
{
"linkId": "diagnosis_as_ds_mcg"
},
{
"linkId": "diagnosis_as_ds_bc"
},
{
"linkId": "diagnosis_as_ds_agw"
},
{
"linkId": "diagnosis_as_ds_ac"
},
{
"linkId": "diagnosis_as_ds_ac_grade_4"
},
{
"linkId": "diagnosis_as_ds_ods"
},
{
"linkId": "diagnosis_as_ds_eps"
},
{
"linkId": "diagnosis_as_treatment___no_treatment"
},
{
"linkId": "diagnosis_as_treatment___surgery"
},
{
"linkId": "diagnosis_as_treatment___in_office_treatment"
},
{
"linkId": "diagnosis_as_treatment_surgery___endoscopic_intervention",
"answer": [
{
"valueBoolean": false
}
]
},
{
"linkId": "diagnosis_as_treatment_surgery___endoscopic_excision"
},
{
"linkId": "diagnosis_as_treatment_surgery___laryngotracheal_reconstruction"
},
{
"linkId": "diagnosis_as_treatment_surgery___laryngotracheoplasty"
},
{
"linkId": "diagnosis_as_treatment_surgery___tracheostomy"
},
{
"linkId": "diagnosis_as_treatment_surgery___other"
},
{
"linkId": "diagnosis_as_treatment_in_office___steroid_injections_sole"
},
{
"linkId": "diagnosis_as_treatment_in_office___steroid_injections_adjunct"
},
{
"linkId": "diagnosis_as_treatment_in_office___dilation"
},
{
"linkId": "diagnosis_as_treatment_in_office___laser"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
{
"resourceType": "QuestionnaireResponse",
"id": "01401050-e9dd-486e-b0d4-0001ee7f861d-als",
"questionnaire": "https://kind-lab.github.io/vbai-fhir/Questionnaire-dneuroamyotrophiclateralsclerosisalsschema",
"status": "completed",
"item": [
{
"linkId": "record_id",
"answer": [
{
"valueString": "01401050-e9dd-486e-b0d4-0001ee7f861d"
}
]
},
{
"linkId": "diagnosis_als_gsd_category_1"
},
{
"linkId": "diagnosis_als_gsd_category_2"
},
{
"linkId": "diagnosis_als_gsd_category_3"
},
{
"linkId": "diagnosis_als_gsd_calculation"
},
{
"linkId": "diagnosis_als_disease_subtype"
},
{
"linkId": "diagnosis_als_disease_subtype_familial"
},
{
"linkId": "diagnosis_als_etiology___genetic_factors"
},
{
"linkId": "diagnosis_als_etiology___environmental_factors"
},
{
"linkId": "diagnosis_als_etiology_genetic_factors___c9orf72"
},
{
"linkId": "diagnosis_als_etiology_genetic_factors___sod1"
},
{
"linkId": "diagnosis_als_etiology_genetic_factors___other"
},
{
"linkId": "diagnosis_als_cp___upper_motor_neuron"
},
{
"linkId": "diagnosis_als_cp___lower_motor_neuron"
},
{
"linkId": "diagnosis_als_cp___bulbar_symptoms"
},
{
"linkId": "diagnosis_als_cp___respiratory_involvement"
},
{
"linkId": "diagnosis_als_dp"
},
{
"linkId": "diagnosis_als_na_alsfrs_r_score"
},
{
"linkId": "diagnosis_als_na_fvc_percentage"
},
{
"linkId": "diagnosis_als_treatment_and_management"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes___medications"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes___supportive_care"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___riluzole"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___edaravone"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___sodium"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___tofersen"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___symptomatic_treatment"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_medications___other"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___physical_therapy",
"answer": [
{
"valueBoolean": false
}
]
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___occupational_therapy"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___speech_therapy"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___respiratory_support"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___nutrition_support",
"answer": [
{
"valueBoolean": false
}
]
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___psychotherapy"
},
{
"linkId": "diagnosis_als_treatment_and_management_yes_supportive_care___other"
}
]
}
Loading
Loading