release-0.24.0

BACKWARD INCOMPATIBLE CHANGES

• ClickHouseKeeper support has been completely re-written. It is now fully compatible with CHI in syntax and supported functionality that closes multiple issues discovered by early adopters. The downside is that CHK resources created by operator 0.23.x will loose the storage and change service name due to different naming conventions.
  See CHK upgrade instructions from 0.23 to 0.24 for detail.
  This closes #1504, #1497, #1469, #1465, #1459, #1408, #1362, #1296

Added

• Added 'pdbMaxUnavailable' attribute to the cluster to manage PDB behavior. See 01-simple-layout-03-multiple-clusters-pdb-spec.yaml
• Added 'all-clusters' cluster into auto-generated remote_servers.xml that references multiple clusters in CHI with respect to shards and replicas. by @tanner-bruce in #1409

Changed

• Operator will now create zookeeper root path if specified in CHI configuration. Previously it assumed root path already exists.
• Operator now waits for the last host to be included in the cluster before marking CHI as completed
• Update helm chart to allow add pod labels by @bruno-s-coelho in #1359. Closes #1356
• Support install operator from local template by @orloffv in #1401
• Added prometheus zeros init for reconciliation metrics by @nmiculinic in #1398
• Update helm chart to allow adding topologySpreadConstraints by @naturalett in #1462
• Use ConfigMap instead of Secret for grafana dashboards by @cwrau in #1457
• Altinity_ClickHouse_Operator_dashboard.json: dashboard has been updated by @nmiakushina in #1512 and #1515

Fixed

• Fixed hostAliases in podTemplate by @chengjoey in #1453. Closes #1445
• Fix install zookeeper operator by @realyota in #1437
• Fix custom hostAliases do not take effect in sts by @chengjoey in #1453
• Fixed ClickHouseKeeper dashboard templates/queries by @ja14000 in #1481

New Contributors

• @vitaliyf made their first contribution in #1365
• @kam1kaze made their first contribution in #1366
• @orloffv made their first contribution in #1401
• @nmiculinic made their first contribution in #1398
• @tanner-bruce made their first contribution in #1409
• @holycheater made their first contribution in #1430
• @chengjoey made their first contribution in #1453
• @cwrau made their first contribution in #1457
• @AlexandrKhabarov made their first contribution in #1460
• @naturalett made their first contribution in #1462
• @ja14000 made their first contribution in #1481
• @jirislav made their first contribution in #1470
• @p-wall made their first contribution in #1511
• @nmiakushina made their first contribution in #1512

Full Changelog: release-0.23.7...release-0.24.0

release-0.23.7

This is a maintenance release while 0.24.0 is delayed.

Fixed

• Fixed metrics exporter compatibility for ClickHouse 24.3 and above. Closes #1411
• Fixed deletion of S3 data when dropping replicas or deleting clusters. Closes #1388
• Fixed a bug in CHK reconciler that might result in looped updates of CHK status by @holycheater in #1430. Closes #1413

Full Changelog: release-0.23.6...release-0.23.7

release-0.23.6

Fixed

• Fixed a bug when schema was not fully restored if volume from the first node in the cluster has been re-provisioned
• Updated golang to 1.21 to address CVEs in dependent modules

Other

• Moved baseline version for tests to 24.3

Full Changelog: release-0.23.5...release-0.23.6

release-0.23.5

Changed

• 'chi' label is currently added to 'clickhouse_operator_*' metrics
• CHI annotations are now added as labels to 'chi_*' metrics
• clickhouse-operator Helm charts are moved from 'deploy/helm' to 'deploy/helm/clickhouse-operator'

Fixed

• Fixed a bug when nodes were not correctly excluded from remote_servers on replicated clusters with 4 or more shards, that might result in failed distributed queries and unnecessary reconciliation delays.

Full Changelog: release-0.23.4...release-0.23.5

release-0.23.4

Changed

• Allow adding pod labels in Helm chart by @bruno-s-coelho in #1369. Closes #1356
• The default service type has been changed from LoadBalancer to ClusterIP
• Cluster restart rules were fixed for some structured settings like 'logger/*'. Now changing of those would not cause a restart of pods.
• Operator does not fail if ClickHouserKeeperInstallation resource type is missing in k8s
• Added 'clickhouse_operator_chi_reconciles_aborted' operator metric

NOTE: There was a lot of internal refactoring related to Keeper code, but no functional changes yet. The Keeper configuration functionality will be improved in the next major release.

Full Changelog: release-0.23.3...release-0.23.4

release-0.23.3

Changed

• Enabled parameterized databases propagation to shards and replicas in ClickHouse 22.12+. That includes support for Replicated and MySQL database engines. Closes #1076
• Removed object storage disks from DiskTotal/Free metrics since those do not make any sense
• Added ability to import packages with operator APIs by @dmvolod in #1229
• Introduced number of unchanged hosts in CHI status

Fixed

• Fixed an issue with cluster stop could be taking long time. Closes #1346
• Fixed a bug when inconsistent cluster definition might result in a crash. Closes #1319
• Fixed a bug when hosts-completed could be incorrectly reported in status when reconcile is re-started in the middle

Full Changelog: release-0.23.2...release-0.23.3

release-0.23.2

What's Changed

• Fix environment variables generation for secrets that might be off in some cases. Closes #1344
• Golang is upgraded to 1.20. Closes CVEs in dependent libraries.

Full Changelog: release-0.23.1...release-0.23.2

release-0.23.1

Fixed

• Fixed generation of users that could be broken in some cases. Closes #1324 and #1332
• Fixed metrics-exporter that might fail to export metrics in some cases. Closes #1336
• Fixed Keeper examples
• Include installation of ClickhouseKeeperInstallations CRD in Helm chart readme by @echozio in #1330

Improved

• Updated grants example by @lesandie in #1333
• Upgrade ClickHouse version to 23.8-lts by @orginux in #1338

Full Changelog: release-0.23.0...release-0.23.1

release-0.23.0

Added

• Kubernetes secrets are currently supported with the standard syntax for user passwords, configuration settings, and configuration files, for example:

    users:
      user1/password:
        valueFrom:
          secretKeyRef:
            name: clickhouse_secret
            key: pwduser1
    settings:
      s3/my_bucket/access_key:
        valueFrom:
          secretKeyRef:
            name: s3-credentials
            key: AWS_ACCESS_KEY_ID
    files:
      server.key:
        valueFrom:
          secretKeyRef:
            name: clickhouse-certs
            key: server.key

See updated Security Hardening Guide for more detail.

• Added experimental support for ClickHouse Keeper by @frankwg in #1218

kind: ClickHouseKeeperInstallation

See examples in there: https://github.com/Altinity/clickhouse-operator/tree/0.23.0/docs/chk-examples
The implementation is not final, following things yet needs to be done:

1. dynamic reconfiguration, that is required in order to support adding and removing Keeper replicas
2. integration with ClickHouseInstallation, so Keeper could be referenced by a reference, instead by a service

• CHI labels are now added to exported Prometheus metrics

Changed

• Services are now re-created if ServiceType is changed in order to workaround Kubernetes issue. Closes #1302
• Operator now waits for ClickHouse service endpoints to respond when checking node is up.
• CHI templates are now automatically reloaded by operator. Before, templates were only reloaded during startup. In order to apply changes, CHI update needs to be triggered.
• Operator will now crash if operator configuration is broken or can not be parsed. That prevents the fallback to the defaults in case of errors.

Fixed

• Fixed schema propagation on new replicas for ClickHouse 23.11 and above
• Fixed data recovery when PVC is deleted by a user. Closes #1310

Improved

• Improve helm, update values.yaml to properly generate helm/README.md by @Slach in #1278
• Improve clickhouse-keeper manifests by @Slach in #1234
• chore: remove refs to deprecated io/ioutil by @testwill in #1273
• Update URL for accepted logging levels by @madrisan in #1270
• Add a chi example for sync users by @ccsxs in #1304
• Bump zookepper operator version to 0.2.15 by @GrahamCampbell in #1303
• Optional values.rbac to deploy rbac resources by @Salec in #1316
• update helm chart generator to treat config.yaml as yaml in values by @echozio in #1317

Full Changelog: release-0.22.2...release-0.23.0

release-0.22.2

What's Changed

• Fixed a bug when operator did not restart ClickHouse pods if 'files' section was changed without 'config.d' destination, e.g. files/settings.xml.
• Fix ServiceMonitor endpoints #1276 by @MiguelNdeCarvalho, and #1290 by @muicoder. Closes #1287
• Disabled prefer_localhost_replica in default profile

Full Changelog: release-0.22.1...release-0.22.2