Merge branch 'main' into fix/incorrect-data-for-ui-patterns-user

.github/pull_request_template.md

@@ -4,7 +4,7 @@
 
 ## Related
 
-* [FP-123](https://jira.tacc.utexas.edu/browse/FP-123)
+* [WP-XYZ](https://jira.tacc.utexas.edu/browse/WP-XYZ)
 
 ## Changes
 

.github/workflows/build.yml

@@ -0,0 +1,46 @@
+name: Core-Portal Builds
+
+on:
+  push:
+    branches:
+      - [ main, feat/Tapis-v3 ]
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      -
+        name: Get shortsha and branch name
+        uses: actions/checkout@v3
+        id: vars
+        run: |
+          if [ -z "$EVENT_SHA" ]; then SHORT_SHA=${GITHUB_SHA::8}; else SHORT_SHA=${EVENT_SHA::8}; fi
+          echo ::set-output name=sha_short::${SHORT_SHA}
+          BRANCH_SHORT=`git symbolic-ref --short HEAD | sed 's/[^[:alnum:]\.\_\-]/-/g'`
+          echo ::set-output name=branch_short::${BRANCH_SHORT}
+        env:
+          EVENT_SHA: ${{ github.event.client_payload.sha }}
+      - name: Print vars
+        run: |
+          echo $SHORTSHA
+          echo $BRANCH_SHORT
+        env:
+          SHORTSHA: ${{ steps.vars.outputs.sha_short }}
+          BRANCH_SHORT: ${{ steps.vars.outputs.branch_short }}
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      -
+        name: Login to Docker Hub
+        uses: docker/login-action@v2
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      -
+        name: Build and push
+        uses: docker/build-push-action@v3
+        with:
+          context: ./server/conf/docker
+          file: ./server/conf/docker/Dockerfile
+          push: true
+          tags: taccwma/core-portal:${{ steps.vars.outputs.branch_short }},taccwma/core-portal:${{ steps.vars.outputs.sha_short }}

.github/workflows/main.yml

@@ -18,12 +18,12 @@ jobs:
     - name: Setup Python
       uses: actions/setup-python@v4
       with:
-        python-version: 3.7
+        python-version: 3.11
 
     - name: Setup Poetry
-      uses: abatilo/actions-poetry@v2.0.0
+      uses: abatilo/actions-poetry@v2
       with:
-        poetry-version: 1.2.0
+        poetry-version: 1.4.2
 
     - uses: actions/cache@v3
       with:
@@ -60,12 +60,12 @@ jobs:
     - name: Setup Python
       uses: actions/setup-python@v4
       with:
-        python-version: 3.7
+        python-version: 3.11
 
     - name: Setup Poetry
-      uses: abatilo/actions-poetry@v2.0.0
+      uses: abatilo/actions-poetry@v2
       with:
-        poetry-version: 1.2.0
+        poetry-version: 1.4.2
 
     - uses: actions/cache@v3
       with:

.gitignore

@@ -55,6 +55,7 @@ db.sqlite3*
 
 # ignore various Python things
 __pycache__/
+.python-version
 *.py[cod]
 *$py.class
 *.so

CHANGELOG.md

@@ -7,6 +7,144 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [3.0.1] - 2023-06-29: Fix Onboarding Websockets
+
+### Changed
+
+- Update pull_request_template.md
+
+### Fixed
+
+- fix(docker): use latest not tapisv3 image (#818)
+- Fix SetupEvent not sending websocket notifications (#820)
+
+## [3.0.0] - 2023-06-26: Tapis v3; Upgrade Django to 3.2
+
+### Added
+
+- TV3-143: Shared Workspace migration scripts (#803)
+- TV3-175: token endpoint (#809)
+- TV3-155, TV3-156: Customizable Tapis v2 Jobs View (#789)
+- TV3-78: Tapis v3 import-apps util (#786)
+- TV3-105: Historical jobs in V3 (#744)
+- FP-319: Job History Search (#756)
+- TV3-55: Job (re)submission backend (#734)
+
+### Changed
+
+- Update Readme (#814), (#815)
+- TV3-176: Revised parameterSet view in AppForm (#812)
+- TV3-173, TV3-174: Regex validation; Dynamic parameterSet (#810)
+- TV3-92: support field type from notes in app definition (#802)
+- TV3-162: Upgrade Django to 3.2 (#798)
+- TV3-169: Replace tapis profile info with TAS info (#801)
+- TV3-167: Update TODOv3 comments (#796)
+- TV3-166: Remove Defuct/Unused Settings (#794), (#776)
+- TV3-161: Shared Workspaces Push Keys (#790)
+- TV3-157: No longer save ssh keypairs in database (#788)
+- TV3-47: Tapis v3 Search (#781)
+- TV3-129: Update Request New Allocation Link (#784)
+- TV3-130: manage account changes (#782)
+- TV3-25: Tapis v3 Shared Workspaces (#771)
+- TV3-79: Tapis v3 Postits (#775)
+- TV3-87: Tapis v3 Compress/Extract Toolbar Utility Apps (#774)
+- TV3-43: Tapis v3 Files Handler (#769)
+- TV3-125: Remove Feedback link (#773)
+- TV3-137: Queue validation for v3 systems (#770)
+- TV3-109 Concatenate homeDir in DataFiles Breadcrumbs (#765)
+- TV3-98: enable app unit tests (#768)
+- TV3-102: Implement relative homeDir listings for Data Files systems (#755)
+- TV3-44: Tapis v3 Notifications (#745)
+- TV3-82: Tapis v3 Push Keys (#739)
+- Tapis v3 long-live client (#747)
+- TV3-51: Tapis v3 app form frontend (#730)
+- TV3-57: Replace JobSubmission model with tags (#736)
+- TV3-80: update token refresh middleware (#732)
+- TV3-67: Tapis v3 App licenses (#726)
+- TV3-52: Tapis v3 Systems Onboarding (#713)
+- TV3-42: Tapis v3 Jobs Listing (#705)
+- TV3-70: Tapis v3 Apps Tray (#720)
+- TV3-50: Tapis v3 apps operations (#711)
+- TV3-2: Tapis v3 Auth (#664)
+
+### Fixed
+
+- TV3-168: Tapis Client UI Delay (#804)
+- TV3-141, TV3-171: Fix Navigation in Copy/Move Modal & Incorrect System Names (#793)
+- TV3-172: Prevent excessive refetching of user roles (#805)
+- TV3-152: Fix workspace jupyter mounts (#800)
+- WP-46: Fix Filter Inconsistencies (#795)
+- TV3-163: Add 10s delay to workaround tapis-issue (#791)
+- TV3-154: Treat 500 errors from file listings as need for keys (#787)
+- TV3-107, TV3-120: Fix incorrect and missing systems (#767)
+- TV3-102: Properly Handle Job Output (#760)
+
+## [2.24.0] - 2023-06-22: Bugfixes; BM preview; Custom stylesheets
+
+### Added
+
+- BM-45: Add preview for Brainmap files (.nii, .nii.gz) (#772)
+- Allow per-project portal stylesheets (#799)
+
+### Changed
+
+- FP-1986: Restrict APCD submissions access (#766)
+- Update local CA certs (#783)
+- Remove Maverick from sysmon and from sysmon fixtures (#808)
+
+### Fixed
+
+- FP-1952: tighter ES network settings (#752)
+- Pin cms version; update cms settings; fix es_domain (#759)
+- Remove Data Submissions from Sidebar (#761)
+- FP-1986: Restrict APCD submissions access (#766)
+- WI-39: remove MFAStep from onboarding (#811)
+
+## [2.23.0] - 2023-01-18: Minor changes and bugfixes
+
+### Changed
+
+- FP-1951: Configurable Ticket Attachment Size (#754)
+- FP-1939: Cleanup searchbar and sidebar components (#737)
+- FP-1943: Remove retired Longhorn from system monitor (#738)
+- TV3-93: Tapis v2 verbose historical job data (#748)
+
+### Fixed
+
+- FP-1928: Fix Data Files Dropdown CSS (#735)
+
+## [2.22.1] - 2022-11-15: Increase CustomMessageTemplate length
+
+### Fixed
+
+- FP-1922: Increase CustomMessageTemplate max_length to 1000 characters
+
+## [2.22.0] - 2022-11-15: Common Searchbar; Common Sidebar; TAP Sysmon
+
+### Added
+
+- FP-1902: Common Searchbar component (#724)
+
+### Changed
+
+- FP-1839: Feedback link in portal is configurable (#707)
+- FP-1537: Download modal adjustments (#716)
+- FP-1877: Replace alert with inline (#717)
+- FP-1886: Remove My Account from pulldown at the top right (#723)
+- Quick: Move prettier config to .prettierrc file (#725)
+- FP-1901: Replace system monitor endpoint with TAP (#729)
+- FP-1560: Use _common Sidebar (#731)
+
+### Fixed
+
+- FP-1668: Added () to accepted characters and changed error message to reflect (#709)
+- FP-1669: Fix back button in copy modal (#710)
+- FP-1859: Fix upload icon color (#714)
+- FP-1584: Populating source portal name in ticket data (#718)
+- FP-1629: Allow helper text to display with error message while creating a job (#719)
+- FP-1861: Remove app icon underline on hover (#721)
+- FP-1860: Match css text color for 'Up to 500mb' text (#728)
+
 ## [2.21.0] - 2022-10-06: Bugfixes for Allocations, Systems, Roles
 
 ### Changed
@@ -748,7 +886,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [1.0.0] - 2020-02-28
 v1.0.0 Production release as of Feb 28, 2020.
 
-[unreleased]: https://github.com/TACC/Core-Portal/compare/v2.21.0...HEAD
+[unreleased]: https://github.com/TACC/Core-Portal/compare/v3.0.1...HEAD
+[3.0.1]: https://github.com/TACC/Core-Portal/releases/tag/v3.0.1
+[3.0.0]: https://github.com/TACC/Core-Portal/releases/tag/v3.0.0
+[2.24.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.24.0
+[2.23.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.23.0
+[2.22.1]: https://github.com/TACC/Core-Portal/releases/tag/v2.22.1
+[2.22.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.22.0
 [2.21.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.21.0
 [2.20.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.20.0
 [2.19.0]: https://github.com/TACC/Core-Portal/releases/tag/v2.19.0

Makefile

@@ -5,7 +5,7 @@ DOCKER_IMAGE_LATEST := $(DOCKERHUB_REPO):latest
 
 ####
 # `DOCKER_IMAGE_BRANCH` tag is the git tag for the commit if it exists, else the branch on which the commit exists
-DOCKER_IMAGE_BRANCH := $(DOCKERHUB_REPO):$(shell git describe --exact-match --tags 2> /dev/null || git symbolic-ref --short HEAD)
+DOCKER_IMAGE_BRANCH := $(DOCKERHUB_REPO):$(shell git describe --exact-match --tags 2> /dev/null || git symbolic-ref --short HEAD | sed 's/[^[:alnum:]\.\_\-]/-/g')
 
 .PHONY: build
 build:
@@ -14,12 +14,12 @@ build:
 .PHONY: build-full
 build-full:
 	docker build -t $(DOCKER_IMAGE) --target production -f ./server/conf/docker/Dockerfile .
-	# docker tag $(DOCKER_IMAGE) $(DOCKER_IMAGE_BRANCH) # Note: Currently broken for branches with slashes
+	docker tag $(DOCKER_IMAGE) $(DOCKER_IMAGE_BRANCH) # Note: Special chars replaced with dashes
 
 .PHONY: publish
 publish:
 	docker push $(DOCKER_IMAGE)
-	# docker push $(DOCKER_IMAGE_BRANCH)
+	docker push $(DOCKER_IMAGE_BRANCH)
 
 .PHONY: publish-latest
 publish-latest:

README.md

@@ -38,7 +38,8 @@ NOTE: This may require a computer restart to take effect.
 2. With Default Keychains > login selected, choose File > Import Items... from the menu.
 3. Navigate to `./server/conf/nginx/certificates`
 4. Select `ca.pem`
-5. Search for CEP and double click on the certificate
+5. Under the "All" or "Certificates" tab,\
+    Search for CEP and double click on the certificate
 6. In the Trust section, find the "When using this certificate" dropdown and select "Always Trust"
 7. Close the window to save.
 
@@ -72,10 +73,10 @@ NOTE: This may require a computer restart to take effect.
   1. Add a record to your local `hosts` file for `127.0.0.1 cep.test`
       - `sudo vim /etc/hosts`
 
-     _WARNING: This name **must** match the **agave callback URL** defined for the client in `settings_secret.py` for `_AGAVE_TENANT_ID`._
+  2. Do this step after going through the server and client code configuration steps in next section.
+
+     Direct your browser to `https://cep.test`. This will display the django CMS default page. To login to the portal, point your browser to `https://cep.test/login`.
 
-  2. Direct your browser to `https://cep.test`. This will display the django CMS default page. To login to the portal, point your browser to `https://cep.test/login`.
-
      _NOTE: If when navigating to `https://cep.test` you see a "Server not found" error while on the VPN, follow these steps and try again:_
       1. Open the Network app utility
       2. Select network connection you’re on (wifi, ethernet, etc)
@@ -132,9 +133,9 @@ OR
     npm ci
     npm run build
 
--  _Note: During local development you can also use `npm run dev` to set a livereload watch on your local system that will update the portal code in real-time. Again, make sure that you are using NodeJS LTS and not an earlier version. You will also need the port 3000 available locally._
-
+-  _Notes: During local development you can also use `npm run dev` to set a live reload watch on your local system that will update the portal code in real-time. Again, make sure that you are using NodeJS LTS and not an earlier version. You will also need the port 3000 available locally._
 
+-  _Notes: If your settings.DEBUG is set to true, you will have to use `npm run dev` to have a functional app. In DEBUG setting, the requests are handled via [vite][3]._
 #### Initialize the application in the `core_portal_django` container:
 
     docker exec -it core_portal_django /bin/bash
@@ -160,7 +161,7 @@ Requirements:
 - At least one page in CMS (see above).
 - At least [15% of free disk space](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html).
 - For Mac/Windows
-    - At least 4GB of RAM allocated to Docker (see Docker Desktop > Preferences > Resources > Advanced).
+    - At least 4GB of RAM allocated to Docker (see Docker Desktop > Settings > Resources > Advanced).
 - For Linux (Locally)
     - Run `sudo sysctl -w vm.max_map_count=2146999999` (The minimum required by [ES](https://www.elastic.co/guide/en/elasticsearch/reference/master/_maximum_map_count_check.html) is 262144 but it doesn't seem to work).
     - Run `sudo sysctl -w vm.overcommit_memory=1`.
@@ -248,10 +249,16 @@ Deployments are initiated via [Jenkins](https://jenkins01.tacc.utexas.edu/view/W
 
 #### Development Workflow
 We use a modifed version of [GitFlow](https://datasift.github.io/gitflow/IntroducingGitFlow.html) as our development workflow. Our [development site](https://dev.cep.tacc.utexas.edu) (accessible behind the TACC Network) is always up-to-date with `main`, while the [production site](https://prod.cep.tacc.utexas.edu) is built to a hashed commit tag.
-- Feature branches contain major updates, bug fixes, and hot fixes with respective branch prefixes:
-    - `task/` for features and updates
-    - `bug/` for bugfixes
-    - `fix/` for hotfixes
+- "Feature branches" contain major and minor updates, bug fixes and hot fixes, and other changes with respective branch prefixes:
+    - `feat/` for features and updates
+    - `fix/` for bugfixes and hotfixes
+    - `refactor/` for large internal changes
+    - `style/` for code style changes (white-space, formatting, etc.)
+    - `chore/` for no-op changes
+    - `docs/` for documentation
+    - `perf/` for performance improvements
+    - `test/` for test case updates
+    - or other "types" from [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/#summary)
 
 #### Testing Core Styles Changes Locally
 
@@ -286,3 +293,4 @@ Sign your commits ([see this link](https://help.github.com/en/github/authenticat
 [Core Styles]: https://github.com/TACC/tup-ui/tree/main/libs/core-styles
 [1]: https://docs.docker.com/get-docker/
 [2]: https://docs.docker.com/compose/install/
+[3]: https://vitejs.dev/

client/jest.config.js

@@ -82,7 +82,7 @@ module.exports = {
     '.*\\.(css|scss|sass)$': 'identity-obj-proxy',
     '^_common(.*)$': '<rootDir>/src/components/_common$1',
     '^utils(.*)$': '<rootDir>/src/utils$1',
-    '^hooks(.*)$': '<rootDir>/src/hooks$1'
+    '^hooks(.*)$': '<rootDir>/src/hooks$1',
   },
 
   // An array of regexp pattern strings, matched against all module paths before considered 'visible' to the module loader
@@ -127,7 +127,7 @@ module.exports = {
   // runner: "jest-runner",
 
   // The paths to modules that run some code to configure or set up the testing environment before each test
-  // setupFiles: [],
+  setupFiles: ['./jest.setup.js'],
 
   // A list of paths to modules that run some code to configure or set up the testing framework before each test
   // setupFilesAfterEnv: [],
@@ -139,7 +139,7 @@ module.exports = {
   // snapshotSerializers: [],
 
   // The test environment that will be used for testing
-  testEnvironment: 'jsdom'
+  testEnvironment: 'jsdom',
 
   // Options that will be passed to the testEnvironment
   // testEnvironmentOptions: {},

client/jest.setup.js

@@ -0,0 +1 @@
+jest.mock('@niivue/niivue', () => {});