Merge upstream

…e differently for EntityListIterator to avoid calling hasNext(); this improves performance and helps with a MySQL JDBC driver issue with useCursorFetch = true where ResultSet.isBeforeFirst() returns true when there are no results, and it should return false in that case

(cherry picked from commit e6a7b8e)

…Actions iterator element did not auto close EntityListIterator instances; thanks to Deepak for reporting this on the forum

(cherry picked from commit d640e8d)

…nclude the catalog from the Connection; this is needed for MySQL which does not restrict meta data queries to the current connected database without specifying a catalog on these methods; this is tested with H2 and should be fine with Postgres as well, may cause issues with other databases and needs more testing

In EntityDbMeta change all calls to getTables() and getColumns() to i…

Upstream sync

…lTemplateRenderer and MNode class (moqui#565)

https://freemarker.apache.org/docs/versions_2_3_32.html

…l4jf, junit

…d testing of service calls with some profiling info, some basics for a first pass

…ail, part of code is more generic addition to ArtifactExecutionInfoImpl

… in addition to entire run stats, prep work for performance charts and such

…hreads * 4 for max load pool size; add permission for SERVICE_LOAD_RUNNER for screens

…ave, also called by cleanAll, was missing in the prior OpenSearch changes

…rsion; the docs are horrible with 3 differences between what actually ended working from a bunch of random experimentation; PIT support is necessary for the Elastic Entity implementation (alternative to a DB cursor); this ended up being needed because more recent OpenSearch that supports PIT also uses more memory than ElasticSearch, enough that the anemic demo.moqui.org server with 2GB RAM can't run Moqui + OpenSearch 2.4

…asticClient is found

… * 3 threads (instead of *2); increasing after tests and production results showing typical low CPU time per thread (much of it is deferred database or search with lots of I/O wait); also improve logging when waiting for worker pool to empty; note that this also improves automated test performance because many of the tests get to hundreds of async services queued up and the ThreadPoolExecutor seems to add threads slowly even if the pool is fully utilized

…r session token

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* fix: consider transitions in screens or parent screens defined with require-authentication in "anonymous-view" or "anonymous-all" as permitted

* fix: consider transitions defined with authenticate in "anonymous-view" or "anonymous-all" as permitted

* fix: consider any of service.@authenticate and screen.@require-authentication to determine permission on transition

* fix: considering verb-based execution actions of single-service when determining transition permission, correctly determining whether view or all allowed by screen

…reen definition instead of NPE

* Upgraded freemarker version to 2.3.32, also updated ftl version in FtlTemplateRenderer and MNode class (moqui#565)

https://freemarker.apache.org/docs/versions_2_3_32.html

* Various library updates: jackson-databind, jetty, joda-time, shiro, sl4jf, junit

* Added missing package name in entity relationship

* Add StatusFlowTransitionFromAndTo view-entity

* In ServiceFacadeImpl add classes for Service LoadRunner, used for load testing of service calls with some profiling info, some basics for a first pass

* In Service LoadRunner add stats broken down by artifact type with detail, part of code is more generic addition to ArtifactExecutionInfoImpl

* Service LoadRunner improved stats gathering with time bin based stats in addition to entire run stats, prep work for performance charts and such

* In Service LoadRunner add small random delay support, use available threads * 4 for max load pool size; add permission for SERVICE_LOAD_RUNNER for screens

* In build.gradle also delete the SaveOpenSearch.zip file in cleanLoadSave, also called by cleanAll, was missing in the prior OpenSearch changes

* Fix PIT testing against ElasticSearch 7.10.2, the last open source version; the docs are horrible with 3 differences between what actually ended working from a bunch of random experimentation; PIT support is necessary for the Elastic Entity implementation (alternative to a DB cursor); this ended up being needed because more recent OpenSearch that supports PIT also uses more memory than ElasticSearch, enough that the anemic demo.moqui.org server with 2GB RAM can't run Moqui + OpenSearch 2.4

* Add new moqui-demo component repo to addons.xml

* Add getScreenPathHasTransition in ScreenRenderImpl

* In ElasticDatasourceFactory add better exceptions than NPE when no ElasticClient is found

* Increase default worker-pool-core to 16 threads, and max to 32 or CPU * 3 threads (instead of *2); increasing after tests and production results showing typical low CPU time per thread (much of it is deferred database or search with lots of I/O wait); also improve logging when waiting for worker pool to empty; note that this also improves automated test performance because many of the tests get to hundreds of async services queued up and the ThreadPoolExecutor seems to add threads slowly even if the pool is fully utilized

* In ScreenRenderImpl remove token created requirement when checking for session token

* Revert "Update to session token condition"

* Fixed a runtime error if Currency is BTC (moqui#555)

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* Anonymous usage of screens with transitions (moqui#541)

* fix: consider transitions in screens or parent screens defined with require-authentication in "anonymous-view" or "anonymous-all" as permitted

* fix: consider transitions defined with authenticate in "anonymous-view" or "anonymous-all" as permitted

* fix: consider any of service.@authenticate and screen.@require-authentication to determine permission on transition

* fix: considering verb-based execution actions of single-service when determining transition permission, correctly determining whether view or all allowed by screen

* Small fix for new authz changes for anonymous-view/-all, handle no screen definition instead of NPE

* Handle usernames with different casing in session data

* Add common java includes to the xml actions ftl file

* Add subTopic field to NotificationMessage

* Refactor getTitle to prioritize title() method call over data

* In MoquiStart add support for webapp_session_cookie_max_age env var to set the cookie expire time based on this age in seconds, if not specified defaults to Session expire and cookie will be dropped by the browser when it quits

* BugFix for DataFeed could not find backward relationship for DataDocument

* Add new WeCreate application component to addons.xml

* Allow for properties to be marked "is-secret" so their values don't get
printed into the log at startup.

* Add weight microgram weight unit of measure.

* Small change to new default-property.@is-secret attribute so that a false value treats it as non-secret even if it contains one of the previously supported substrings (pass, pw, key); add explicit is-secret=true attrs to a few in MoquiDefaultConf.xml

* In ScreenRenderImpl add methods needed for section-include to support pagination; in CollectionUtilities paginateList() and paginateParameters() add support for entity-find with search-form-inputs or other situations where paginate parameters are already in place, don't overwrite them and don't try to subList() a big list

* Add new moqui-image repo, fork of original work be acetousk

* Add Sales app from xolvegroup (third party component)

* Added support for loading data on start with entity_on_start_load_types and entity_on_start_load_components properties/env-vars

---------

Co-authored-by: David E Jones <[email protected]>
Co-authored-by: Acetousk <[email protected]>
Co-authored-by: aabiabdallah <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: Adam Heath <[email protected]>

* Fix regression with partitioned tables in PostgreSQL

PostgreSQL JDBC Driver introduced separating type for partitioned table from  40.2.12 pgjdbc/pgjdbc#1708

* Add subscreensItem.menuInclude to menu data (moqui#600)

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* Add subscreensItem.menuInclude to menu data

* Docker-Image-Pull Feature (moqui#553)

* Improvement: In Moqui-Multi-Instance added a Async-Pull-Image-Feature which pulls the image by Using Docker-Engine-Api from multiple registry like AWS, Azure, Docker-Hub

* Update AUTHORS

* Added: added the generic way to process cmd , by adding an extra field(authTokenPass) inside the entity InstanceImage , now user has separate field for cmd and password so all types of registries is easily configurable

* Update ServerEntities.xml by adding description

* Library updates, including Jetty 10.0.13 to 10.015 which had reported vulnerabilities; there are lots of dependencies updated in this set, see diff for full details

* In ScreenRenderImpl change addFormFieldValue() and related methods to handle first, second, and last rows, for qvt and other client rendered output that needs full data for a form in a map/object

* In root build.gradle change gitStatusAll task to be more tolerant of repos with no master branch

* Add text-area.@autogrow attribute, supported only in qvt for now

* Update various libraries including Groovy to 3.0.19 (which has some minor non-backward compatible changes single 3.0.10 with odd boolean behavior in rare cases, adjusted for in the framework long ago but should be watched for in custom code), Jetty to 10.0.16, H2 database, SLF4J, SnakeYAML, Apache Commons Lang3

* In build.gradle gitStatusAll task also handle upstream remotes with no master branch

* Currency (moqui#614)

* Use moqui.basic.Uom entity to determine currency formatting and rounding details

* Add currency-hide-symbol attribute as a complement to currency-unit-field, displaying the value without the currency symbol

* Update authors file

* Add and Handle Hmac Sha256 with timestamp

* A couple of minor bug fixes in the EntityAutoServiceRunner and ContextJavaUtil (moqui#618)

* In EntityAutoServiceRunner, remove unwanted break statement to ensure support for multiple PK fields with wildcard (*).

* In ContextJavaUtil, add missing future keyword.

* Updated authors file.

* Allow for 10 second threshold in nowTimestamp

* In L10nFacadeImpl.formatCurrency() use disableAuthz() for entity find on Uom; small change to currency formatting test to pass with current OOTB settings

* In addons.xml, added new moqui-sso component.

---------

Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: David E. Jones <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Rohit pawar <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: acetousk <[email protected]>
Co-authored-by: Ayman Abi Abdallah <[email protected]>

* Fix regression with partitioned tables in PostgreSQL

PostgreSQL JDBC Driver introduced separating type for partitioned table from  40.2.12 pgjdbc/pgjdbc#1708

* Add subscreensItem.menuInclude to menu data (moqui#600)

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* Add subscreensItem.menuInclude to menu data

* Docker-Image-Pull Feature (moqui#553)

* Improvement: In Moqui-Multi-Instance added a Async-Pull-Image-Feature which pulls the image by Using Docker-Engine-Api from multiple registry like AWS, Azure, Docker-Hub

* Update AUTHORS

* Added: added the generic way to process cmd , by adding an extra field(authTokenPass) inside the entity InstanceImage , now user has separate field for cmd and password so all types of registries is easily configurable

* Update ServerEntities.xml by adding description

* Library updates, including Jetty 10.0.13 to 10.015 which had reported vulnerabilities; there are lots of dependencies updated in this set, see diff for full details

* In ScreenRenderImpl change addFormFieldValue() and related methods to handle first, second, and last rows, for qvt and other client rendered output that needs full data for a form in a map/object

* In root build.gradle change gitStatusAll task to be more tolerant of repos with no master branch

* Add text-area.@autogrow attribute, supported only in qvt for now

* Update various libraries including Groovy to 3.0.19 (which has some minor non-backward compatible changes single 3.0.10 with odd boolean behavior in rare cases, adjusted for in the framework long ago but should be watched for in custom code), Jetty to 10.0.16, H2 database, SLF4J, SnakeYAML, Apache Commons Lang3

* In build.gradle gitStatusAll task also handle upstream remotes with no master branch

* Currency (moqui#614)

* Use moqui.basic.Uom entity to determine currency formatting and rounding details

* Add currency-hide-symbol attribute as a complement to currency-unit-field, displaying the value without the currency symbol

* Update authors file

* Add and Handle Hmac Sha256 with timestamp

* A couple of minor bug fixes in the EntityAutoServiceRunner and ContextJavaUtil (moqui#618)

* In EntityAutoServiceRunner, remove unwanted break statement to ensure support for multiple PK fields with wildcard (*).

* In ContextJavaUtil, add missing future keyword.

* Updated authors file.

* Allow for 10 second threshold in nowTimestamp

* In L10nFacadeImpl.formatCurrency() use disableAuthz() for entity find on Uom; small change to currency formatting test to pass with current OOTB settings

* In addons.xml, added new moqui-sso component.

* Add AutoCloseable extension to EntityListIterator for use with try with resources, thanks to Deepak Dixit for the suggestion

* BugFix EntityListIterator not closed in NotificationMessageImpl#getNotifyUserIds

---------

Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: David E. Jones <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Rohit pawar <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: acetousk <[email protected]>
Co-authored-by: Ayman Abi Abdallah <[email protected]>

* Fix regression with partitioned tables in PostgreSQL

PostgreSQL JDBC Driver introduced separating type for partitioned table from  40.2.12 pgjdbc/pgjdbc#1708

* Add subscreensItem.menuInclude to menu data (moqui#600)

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* Add subscreensItem.menuInclude to menu data

* Docker-Image-Pull Feature (moqui#553)

* Improvement: In Moqui-Multi-Instance added a Async-Pull-Image-Feature which pulls the image by Using Docker-Engine-Api from multiple registry like AWS, Azure, Docker-Hub

* Update AUTHORS

* Added: added the generic way to process cmd , by adding an extra field(authTokenPass) inside the entity InstanceImage , now user has separate field for cmd and password so all types of registries is easily configurable

* Update ServerEntities.xml by adding description

* Library updates, including Jetty 10.0.13 to 10.015 which had reported vulnerabilities; there are lots of dependencies updated in this set, see diff for full details

* In ScreenRenderImpl change addFormFieldValue() and related methods to handle first, second, and last rows, for qvt and other client rendered output that needs full data for a form in a map/object

* In root build.gradle change gitStatusAll task to be more tolerant of repos with no master branch

* Add text-area.@autogrow attribute, supported only in qvt for now

* Update various libraries including Groovy to 3.0.19 (which has some minor non-backward compatible changes single 3.0.10 with odd boolean behavior in rare cases, adjusted for in the framework long ago but should be watched for in custom code), Jetty to 10.0.16, H2 database, SLF4J, SnakeYAML, Apache Commons Lang3

* In build.gradle gitStatusAll task also handle upstream remotes with no master branch

* Currency (moqui#614)

* Use moqui.basic.Uom entity to determine currency formatting and rounding details

* Add currency-hide-symbol attribute as a complement to currency-unit-field, displaying the value without the currency symbol

* Update authors file

* Add and Handle Hmac Sha256 with timestamp

* A couple of minor bug fixes in the EntityAutoServiceRunner and ContextJavaUtil (moqui#618)

* In EntityAutoServiceRunner, remove unwanted break statement to ensure support for multiple PK fields with wildcard (*).

* In ContextJavaUtil, add missing future keyword.

* Updated authors file.

* Allow for 10 second threshold in nowTimestamp

* In L10nFacadeImpl.formatCurrency() use disableAuthz() for entity find on Uom; small change to currency formatting test to pass with current OOTB settings

* In addons.xml, added new moqui-sso component.

* Add AutoCloseable extension to EntityListIterator for use with try with resources, thanks to Deepak Dixit for the suggestion

* BugFix EntityListIterator not closed in NotificationMessageImpl#getNotifyUserIds

* Implemented withCloseable/try-with-resources where needed in the code… (moqui#625)

* Implemented withCloseable/try-with-resources where needed in the code to ensure proper closure of the entity list iterator resource

* Fixed withCloseable syntax

---------

Co-authored-by: David E. Jones <[email protected]>

* Change EntityDataWriterImpl to use groovy CompileStatic, without this a compile error was missed in the changes for PR moqui#625, fix small issues from this

---------

Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: David E. Jones <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Rohit pawar <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: acetousk <[email protected]>
Co-authored-by: Ayman Abi Abdallah <[email protected]>

* Master (moqui#11) (moqui#12)

* Fix regression with partitioned tables in PostgreSQL

PostgreSQL JDBC Driver introduced separating type for partitioned table from  40.2.12 pgjdbc/pgjdbc#1708

* Add subscreensItem.menuInclude to menu data (moqui#600)

* Fixed the problem that moqui cannot be deployed as non-root webapp in Tomcat

* Fixed a runtime error if Currency is BTC

* Fixed a runtime error if Currency is BTC

* Fixed the retries of Elastic Client

* Add subscreensItem.menuInclude to menu data

* Docker-Image-Pull Feature (moqui#553)

* Improvement: In Moqui-Multi-Instance added a Async-Pull-Image-Feature which pulls the image by Using Docker-Engine-Api from multiple registry like AWS, Azure, Docker-Hub

* Update AUTHORS

* Added: added the generic way to process cmd , by adding an extra field(authTokenPass) inside the entity InstanceImage , now user has separate field for cmd and password so all types of registries is easily configurable

* Update ServerEntities.xml by adding description

* Library updates, including Jetty 10.0.13 to 10.015 which had reported vulnerabilities; there are lots of dependencies updated in this set, see diff for full details

* In ScreenRenderImpl change addFormFieldValue() and related methods to handle first, second, and last rows, for qvt and other client rendered output that needs full data for a form in a map/object

* In root build.gradle change gitStatusAll task to be more tolerant of repos with no master branch

* Add text-area.@autogrow attribute, supported only in qvt for now

* Update various libraries including Groovy to 3.0.19 (which has some minor non-backward compatible changes single 3.0.10 with odd boolean behavior in rare cases, adjusted for in the framework long ago but should be watched for in custom code), Jetty to 10.0.16, H2 database, SLF4J, SnakeYAML, Apache Commons Lang3

* In build.gradle gitStatusAll task also handle upstream remotes with no master branch

* Currency (moqui#614)

* Use moqui.basic.Uom entity to determine currency formatting and rounding details

* Add currency-hide-symbol attribute as a complement to currency-unit-field, displaying the value without the currency symbol

* Update authors file

* Add and Handle Hmac Sha256 with timestamp

* A couple of minor bug fixes in the EntityAutoServiceRunner and ContextJavaUtil (moqui#618)

* In EntityAutoServiceRunner, remove unwanted break statement to ensure support for multiple PK fields with wildcard (*).

* In ContextJavaUtil, add missing future keyword.

* Updated authors file.

* Allow for 10 second threshold in nowTimestamp

* In L10nFacadeImpl.formatCurrency() use disableAuthz() for entity find on Uom; small change to currency formatting test to pass with current OOTB settings

* In addons.xml, added new moqui-sso component.

* Add AutoCloseable extension to EntityListIterator for use with try with resources, thanks to Deepak Dixit for the suggestion

* BugFix EntityListIterator not closed in NotificationMessageImpl#getNotifyUserIds

* Implemented withCloseable/try-with-resources where needed in the code… (moqui#625)

* Implemented withCloseable/try-with-resources where needed in the code to ensure proper closure of the entity list iterator resource

* Fixed withCloseable syntax

---------



* Change EntityDataWriterImpl to use groovy CompileStatic, without this a compile error was missed in the changes for PR moqui#625, fix small issues from this

---------

Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: David E. Jones <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Rohit pawar <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: acetousk <[email protected]>
Co-authored-by: Ayman Abi Abdallah <[email protected]>

* Add sentDate field to WikiBlogCategory entity

---------

Co-authored-by: Yao Chunlin <[email protected]>
Co-authored-by: David E. Jones <[email protected]>
Co-authored-by: Wei Zhang <[email protected]>
Co-authored-by: Rohit pawar <[email protected]>
Co-authored-by: Jens Hardings <[email protected]>
Co-authored-by: acetousk <[email protected]>
Co-authored-by: Ayman Abi Abdallah <[email protected]>

…mmons-lang3, commons-io, commons-logging, commons-validator, jackson-databind to 2.16.0, jsoup, shiro

- Set the require new transaction false whiel generating loginkey, as we are getting lock wait time out.
- Added check to skip the status flow transition check if entities belongs to different group

…ion and better match convention used elsewhere, no functional changes

…n addons.xml

… operations like drop and create FKs have been broken with any of these in place (as they are by default)

…ing when the index name is a comma separated list of index names