2002.0.23

ECE-Tools release notes

2002.1.0

Changelog (49 related PRs)

Infrastructure updates–

• Added separate package for Magento Cloud Docker–Decoupled the Docker package from the ECE-Tools package to maintain code quality and provide independent releases. Updates and fixes related to ECE-Tools are managed from the magento-cloud-docker GitHub repository.
• Updated patching capabilities–Moved the Magento patching functionality from the ECE-Tools package to a separate magento-cloud-patches package. During deployment, ECE-Tools uses the new package to apply patches.
• Updated Composer dependencies–Updated the composer.json file for ECE-Tools with a dependency for the magento-cloud-docker package. Now, ECE-Tools includes dependencies for all components in the Magento Commerce Cloud Suite. These packages are installed and updated automatically when you install or update ECE-Tools.

Support for scenario-based deployments–

• Now you can customize the build, deploy, and post-deploy processes using XML configuration files to override or customize the default configuration.
• Changed the hooks configuration in .magento.app.yaml–We updated the hooks configuration format to support scenario-based deployments. The legacy format from earlier ECE-Tools 2002.0.x release is still supported. However, you must update to the new format to use the scenario-based deployment feature. See Scenario-based deployments.

Services updates–

• Added support for PHP 7.3.
• Added support for RabbitMQ 3.8.
• Added validation to check installed service versions against the EOL date for each service. Now, customers receive a notification if a service version is within three months of the EOL date, and a warning if the EOL date is in the past.
• Fixed an Elasticsearch configuration issue to ensure that the correct Elasticsearch settings are configured in all environments.

Environment variable updates–

• Extended the functionality of the WARM_UP_PAGES environment variable to support cache preloading for specific product pages.
• Added the ERROR_REPORT_DIR_NESTING_LEVEL environment variable to simplify error report data management in the <magento_root>/var/report/ directory. See the variable description in the build variables topic.
• Removed the SCD_EXCLUDE_THEMES, STATIC_CONTENT_THREADS, DO_DEPLOY_STATIC_CONTENT, SKIP_SCD, and STATIC_CONTENT_SYMLINK environment variables.
• Fixed an issue in the Elastic Suite configuration process so that the default configuration is overwritten as expected when you configure the ELASTICSUITE_CONFIGURATION deploy variable without the _merge option.

CLI command updates–

• New cron commandYou can now manually manage cron processing in your Magento Commerce Cloud environment using the cron:disableandcron:enable` commands. Use the disable command to stop all active cron processes and disable all Magento cron jobs. Use the enable command to re-enable cron jobs when you are ready.
• Improved error reporting–Added better logging for Magento CLI command failures that occur during ECE-Tools processing.
• Remove deprecated build commands–Removed the following build commands: m2-ece-build, m2-ece-deploy, m2-ece-scd-dump, and renamed ece-tools docker commands to ece-docker.
• Removed the deprecated build_options.ini file and added validation to fail the build if the file exists. Use the .magento.env.yaml file to configure build options.
• Fixed an issue that caused the build process to fail when the config.php file is empty.

2002.0.22

Changelog (4 related PRs)

The ece-tools 2002.0.22 release changes the structure of the ece-tools package to decouple the release of Magento Commerce Cloud patches from the ece-tools release. Starting with this release, patches and critical fixes will be delivered using the magento/magento-cloud-patches package, which is a new dependency for the ece-tools package. We made these changes to reduce complexity for scheduling release updates and working with community contributions.

Changes to the ece-tools package

• Moved the Magento Commerce patches from the ece-tools package to a new magento/magento-cloud-patches composer package.
• Updated the composer.json file for the ece-tools package to add a dependency for the magento/magento-cloud-patches v1.0.0 package.
• Fixed an issue that caused the ece-tools patching process to break when applying patch sets on top of security-only releases, starting with Magento version 2.3.2-p2 and later. This issue was introduced by the new versioning scheme adopted for security-only patches.

Patches and critical fixes

• Update your Cloud environments with ece-tools version 2002.0.22 to apply the following patches and critical fixes. These patches are included in the magento/magento-cloud-patches v1.0.0 package.
  • Page Builder security patches for 2.3.1.x and 2.3.2.x releases–Fixes an issue in Page Builder preview that allows unauthenticated users to access some templating methods that can be used to trigger arbitrary code execution over the network (RCE) resulting in global information leaks. This issue can occur when using unsupported versions of Page Builder with Magento Commerce versions 2.3.1 and 2.3.2.
  • MSI patches–Fixes issues that caused indexing errors and performance issues when using default inventory settings for managing stock.
  • Backward Compatibility of new Mail Interfaces-Fixes a backward incompatibility issue caused by the Magento\Framework\Mail\EmailMessageInterface PHP interface introduced in Magento Commerce v2.3.3. In the scope of this patch, the new EmailMessageInterface inherits from the old MessageInterface, and Magento Commerce core modules are reverted to depend on MessageInterface.
  • Catalog pagination does not work on Elasticsearch 6.x–Fixes a critical issue with search result pagination that affects customers using Elasticsearch 6.x as the catalog search engine.

2002.0.21

Changelog (21 related PRs)

• Docker updates—

  • New Docker Images—Supported by Magento versions 2.3.3 and later
    • PHP version 7.3.
    • Varnish Cache 6.2.0
  • Added support to apply custom hook configuration specified in .magento.app.yaml in the Docker environment. Previously, the Docker environment supported only the default hook configuration.
  • Docker ENV files are no longer generated during the Docker build, and the docker:config:convert command is deprecated. The corresponding data is now stored in the docker-compose.yml file.
  • Updated PHP image–Added Node.js to the PHP Docker image to support node, npm, and grunt-cli capabilities.

• Environment variable updates–

  • Added the LOCK_PROVIDER deploy variable to configure the lock provider which prevents the launch of duplicate cron jobs and cron groups.
  • Added the CONSUMERS_WAIT_FOR_MAX_MESSAGES environment variable to configure how consumers process messages from the message queue when using the CRON_CONSUMERS_RUNNER environment variable to manage cron jobs.

• Updated the logging information returned by the module:refresh command. Now, you can see a detailed list of enabled modules in the cloud.log file.

• Improved version compatibility validation and warning notifications for compatibility issues between Magento version and installed services, such as Elasticsearch, RabbitMq, Redis, and DB.

• Updated interactive validations for service compatibility to reflect supported versions for the new Magento 2.3.3 and 2.2.10 releases.

• Improved the log message returned when the cron job management process in the deploy phase tries to stop a cron job that has already finished to clarify that this issue is not an error. Changed the log level from INFO to DEBUG.

• Fixed an issue when running the setup:upgrade command that did not interrupt the deployment process when a failure occurred during the app:config:import task.

• Fixed an issue affecting WARM_UP_PAGES functionality that uses a default store URL. Now, if the config:show:default-url command cannot fetch a base URL, then the URL from the MAGENTO_CLOUD_ROUTES variable is used.

• Changed the default log level for the file handler to debug to reduce the amount of detail in the log displayed in the Project Web Interface, while still providing detailed information for debugging.

• Fixed an issue that can cause database deadlock errors when the consumers_runner cron job starts multiple instances of the same consumer on different nodes. Now, if you have enabled the CRON_CONSUMERS_RUNNER deploy variable in your environment, the consumers_runner job uses the single-thread option to start one instance of each consumer on only one node.

• Fixed an issue that caused an error with static content deployment during build. After a Magento installation and config dump, an error occurred if there was no locale specified for the admin user in the config.php file. Now, there is a default locale for the admin user in the config.php file.

• Fixed an Undefined index error that occurs when a Magento Cloud CLI command fails in an environment that is not configured with a secure URL (https). Now, the ece-tools package uses the base URL (http) if the secure URL is not available.

2002.0.20

Changelog (30 related PRs)

• Docker Updates—
  • You can now perform functional testing using the ECE-Tools package in the Docker environment.
  • Added support for configuring PHP modules using the .magento.app.yaml file. Any PHP Extensions specified in the .magento.app.yaml file become available in the Docker PHP containers.
  • There are new commands available to improve the Docker command line experience.
  • Added the ability to use Mutagen.io to synchronize files during development between the local host and Docker.
  • Corrected the default path when using the Docker environment. Now, when you use SSH to log in to the Docker container, you are at the Magento root in the /app directory, as expected.
  • Updated the Sodium library from version 1.0.11 to version 1.0.18, and updated the Sodium PHP extension.
  • Added the analysis-icu and the analysis-phonetic Elasticsearch plugins to all Docker images.
  • Improved validations: When using options for the docker:build command, you must provide a value when using an option. Also, added validation for the Node version when using the docker:build run command.
• Environment variable updates—
  • Added support for database table prefixes using the DATABASE_CONFIGURATION environment variable.
  • Added the FORCE_UPDATE_URLS deploy variable to update Magento base URLs when deploying to Pro and Starter production and staging environments.
  • Added the TTFB_TESTED_PAGES post-deploy variable to configure Time to First Byte page tests to check Magento Commerce application performance on sites deployed to Cloud infrastructure.
  • Fixed an issue with multi-threaded SCD, which caused random failures in static content deployment. The workaround involved setting the SCD_THREADS variable to 1. You can now increase the count as needed.
  • You can configure the WARM_UP_PAGES environment variable to cache single pages, multiple domains, and multiple pages.
• Added the pub/static/.htaccess file to the exclude list.
• Fixed an error when all validation messages were showing as Critical if at least one critical level validator returned an error.
• Fixed an issue that caused a deployment failure if the Magento base URL did not exist in the database.
• Added a new env:config:show command to the ECE-Tools package that displays environment services, routes, or variables.
• Fixed an issue that caused a critical error when attempting to install Magento 2.2.6 or earlier with latest ECE-Tools after shell refactoring.
• Fixed an issue that caused Magento 2.1.x and 2.2.x installations to fail with a warning about using a deprecated version of nesbot/carbon.
• Decreased the cloud.log log level for shell output from info to debug.
• Added the --remove-definers (-d) option to the ece-tools db-dump command to remove definers from the dump file.

2002.0.19

Changelog (1 related PR)

• Fixed an issue that overwrites the env.php file during a deploy, resulting in a loss of custom configurations. This update ensures that ECE-Tools updates the env.php file with every deployment, while preserving custom configurations.

2002.0.18

Changelog (24 related PRs)

• Docker Updates—
  • Now, the Docker environment supports the cron configuration defined in the crons property of the .magento.app.yaml file.
  • New Docker Container—Added a TLS termination proxy container to facilitate the Varnish SSL termination over HTTPS.
  • New Docker Image—Added a Node.js image to support Gulp and other capabilities, such as Jasmine JS Unit Testing.
  • Docker build modes—Now you can choose to launch the Docker environment in Production mode or Developer mode. Developer mode supports active development with full, writable filesystem permissions. This improves the developer experience by providing an easier way to develop and sync extensions, for example.
  • Fixed an issue that caused Docker deploy to fail with a Name or service not known error if the cache is configured for a service that is not available. Now, you can remove a service from the .magento/services.yaml file. The Docker configuration generator updates the service in the docker/config.php.dist file automatically.
  • Added interactive validations for service compatibility. Now, if a requested service is incompatible with the Magento version or other services, the interactive mode prompts the user with a message and a choice to continue. See the Service versions available for Docker. Use the -n option to skip the interactivity for CICD purposes.
  • Fixed an issue with the Docker compose db-dump command that erased existing dumps.
• Fixed an issue that assigned Redis session, default, and page_cache cache storage to the same database ID.
• Environment variable updates—
  • The new ELASTICSUITE_CONFIGURATION environment variable retains your customized service settings between deployments.
  • Added the SCD_MAX_EXECUTION_TIMEOUT environment variable so you can increase the time to complete the static content deployment from the .magento.env.yaml file.
  • Added the MAGENTO_CLOUD_LOCKS_DIR environment variable to configure the path to the mount point for the lock provider on the cloud infrastructure. The lock provider prevents the launch of duplicate cron jobs and cron groups. This variable is supported on Magento version 2.2.5 and later and automatically configured.
  • Changed the SCD_THREAD environment variable default values to automatically determine the optimal value based on the detected CPU thread count.
• Fixed an issue with a patch for DB Isolation Mechanism that caused an error when upgrading to ECE-Tools versions 2002.0.16 and 2002.0.17
• Added a patch that replaces Google Image Charts with Image-Charts. See the DevBlog article Google Image Charts deprecation and update for M1.
• Added validation for the SEARCH_CONFIGURATION variable. Deploy fails when the 'engine' option is not set and _merge is not required.
• Fixed an issue that exposed sensitive data after an exception occurs. Now the sensitive information is masked appropriately.
• Improved the fault-tolerant settings of the ECE-Tools package. In the case when Magento 2 cannot read data from the Redis slave instance, a reading is made from the Redis master instance.

2002.0.17

Changelog (8 related PRs)

The ECE-Tools version 2002.0.17 includes an important security patch. See Tech Resources: Magento Open Source Patches.

• Service updates —Supported by Magento versions 2.3.1 and later 2.3.x versions or 2.2.8 and later 2.2.x versions.

  • Added support for Elasticsearch version 6.x.
  • Added support for Redis version 5.0.

• New Docker images—Added the following services to the Docker build:

  • Elasticsearch 6.5
  • Redis 5.0

• New environment variable —Previously, there was a hard-coded timeout for SCD compression. Now you can configure the SCD compression timeout using the SCD_COMPRESSION_TIMEOUT environment variable.

• Added the --use-rewrites option to the Magento install command so that it uses web server rewrites for generated links in the storefront and Admin access to improve security and customer experience.

• Added timestamps to the var/log/install_upgrade.log file so that it shows dates for the Magento installation and upgrade events.

2002.0.16

Changelog (30 related PRs)

• Docker updates—

  • Now, the default service configuration generated in the Docker environment is the same as the default configuration in the Cloud template.

  • You can send mail from your Docker environment using the sendmail service

  • Added the ability to configure Xdebug to debug in the Cloud Docker environment.

  • Fixed an issue with web service permissions when generating the docker-compose.yml file.

• Upgrade improvement—Added validation to confirm that the autoload property in the composer.json file contains required configuration changes before upgrading to Magentgo v2.3.

• The compression process in deploying static content now includes all assets—naively generated or customized—and occurs during the build phase at the beginning of the build:transfer section. Previously, the compression process occurred before applying custom minification and bundling of static assets.

• Fixed a database connection error that occurred during deployment immediately after configuring an additional database and service relationship. Also, this fix addresses an issue that occurred during the configuration process of MBI for Starter. For Starter, this upgrade is a "must have" for using MBI.

• Fixed a validation issue with the database configuration that caused the deploy process to fail.

• Updated the constraint with the appropriate version of the symfony/yaml package to use with PHP constants. Constant parsing does not work when using a symfony/yaml package version earlier than 3.2.

• Environment configuration check—Added validation to check the PHP version and warn users if they are not using the latest recommended version.

• Fixed an issue with processing malformed JSON variables. Now, if a JSON variable causes a syntax error, a warning appears in the cloud.log file and deployment continues using the default variable.

• Fixed a connection error that occurred during deployment immediately after disabling the Redis service.

• Logging changes—Updated the log level from Info to Notice for the following build and deploy process events:

  • Begin and end of the process for reconciling installed modules in composer.json with shared configuration settings in the app/etc/config.php file
  • Begin and end of the configuration validation process
  • Begin and end of the setup:di:compile process for generating classes

• New environment variables—

  • RESOURCE_CONFIGURATION deploy variable—Use this variable to map a resource name to a database connection.
  • X_FRAME_CONFIGURATION global variable—Use this variable to change the X-Frame-Options header configuration for rendering a Magento page in a <frame>, <iframe>, or <object>.

• Environment variable updates—Changed the following environment variables:

  • WARM_UP_PAGES—Added the capability to preload the cache for specified pages on all domains defined for a Magento store. Previously, if your site was configured with multiple domains, the post-deploy process failed to preload the cache for the specified pages on non-default domains and returned the following error in the post-deploy log: ERROR: Warming up failed: <uri>
  • SCD_COMPRESSION_LEVEL—Updated the documentation and the sample .magento.env.yaml file with the correct default values for SCD compression level. See the definitions in the build variables and the deploy variables content.
  • SCD_EXCLUDE_THEMES—This environment variable is deprecated. Use the SCD_MATRIX variable to control theme configuration. See the definitions in the build variables and the deploy variables content.
  • SCD_MATRIX—Fixed the validation process to prevent a problem that occurred when the SCD_MATRIX ignored a theme value that contained different character cases. See the definitions in the build variables and the deploy variables content.
  • ADMIN variables—
    • Improved security when managing credentials for the Magento Admin user using environment variables. You can no longer use the ADMIN_EMAIL, ADMIN_USERNAME, and ADMIN_PASSWORD environment variables to override admin credentials during upgrades. If you cannot access the Admin panel, use the Forgot password feature or the Magento CLI admin:user:create command to create a new admin user.
    • ADMIN_EMAIL is no longer required when upgrading or applying patches.

2002.0.15

Changelog (21 related PRs)

• Docker updates—
  • Now the Docker generator uses the services specified in the .magento.app.yaml and .magento/services.yaml configuration files when building your Docker environment. You can choose a different service version using build parameters.
  • Added PHP 7.2 image—Added support for PHP 7.2 in Cloud Docker; updated the Launch Docker configuration to include the docker:build --php option to specify the version of PHP compatible with your Magento Commerce version.
  • Added a Cron container based on the PHP-CLI image.
  • Added the following services to the Docker build:
    • RabbitMQ 3.5 and 3.7
    • ElasticSearch 1.7, 2.4, and 5.2
    • Redis 3.2 and 4.0
  • Now you have the DB dump capability when using the Cloud Docker database container. Also, you can share files between a host machine and a container using the docker/mnt directory.
• Configure with PHP constants—Added support for PHP constants in the .magento.env.yaml configuration file.
• New environment variable—By default, only the Production environment has Google Analytics enabled. You can enable Google Analytics on the Staging and Integration environments using the ENABLE_GOOGLE_ANALYTICS environment variable.
• Fixed an issue that removed customized cron configurations from the env.php file after a redeployment. Now, custom cron configurations safely remain in the env.php file.
• Fixed inconsistencies in the messages and log levels for build, deploy, and post-deploy phases. Increased beginning and ending log message levels from info to notice for all phases and sub-phases. Added beginning and ending log messages, where appropriate.
• Fixed an issue involving cron processes that prevented the start of the post-deploy phase, when configured. Now, if you have the post-deploy hook enabled, the cron processes are enabled again at the beginning of the post-deploy phase.
• Resolved an issue that prevented a successful installation of Magento when specifying a custom database configuration. Previously, the installation process used the database configuration from the MAGENTO_CLOUD_RELATIONSHIP variable even if you designated customized connection information in the DATABASE_CONFIGURATION environment variable.
• Corrected the config:dump command so that it includes each website locale in the system section of the config.php file.
• Fixed an issue that resulted in warm-up errors during the post-deploy phase by correcting the source base URL reference.
• Fixed an issue that generated files improperly during the setup:di:compile process, which affected the Amazon Pay module.