4.1.0rc2

[sadpandajoe]

Hello Superset Community,

This is a call for the vote to release Apache Superset version 4.1.0.

The release candidate:
https://dist.apache.org/repos/dist/dev/superset/4.1.0rc2/

The Git tag for the release:
https://github.com/apache/superset/tree/4.1.0rc2

The CHANGELOG for the release:
https://github.com/apache/superset/blob/4.1.0rc2/CHANGELOG/4.1.0.md

The instructions for updating to the release:
https://github.com/apache/superset/blob/4.1.0rc1/UPDATING.md

Public keys are available at:
https://www.apache.org/dist/superset/KEYS

[michael-s-molina]

#29997

[sadpandajoe]

Looks like this could be a blocker for people trying to upgrade, will cherry for rc3

[sadpandajoe]

🍒 'd

[michael-s-molina]

#29884

[sadpandajoe]

Not a blocker but will cherry as we'll need a rc3

[sadpandajoe]

🍒 'd

[sadpandajoe]

#30017

if the sl_ tables don't exist, current migrations will fail

[sadpandajoe]

Could be a blocker to upgrade for users so will cherry it in once merged.

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30019

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30025

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30026

[sadpandajoe]

This creates logs but doesn't sound like it breaks functionality. Nice to get it in but doesn't seem like a blocker for 4.1.

[sadpandajoe]

fixed in: #30074

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30035

[sadpandajoe]

This seems like a blocker as the functionality is unusable if you're trying to drill on a lower dashboard.

[sadpandajoe]

fixed in: #30073

[sadpandajoe]

🍒 'd

[hao-zhuventures]

#30043

[sadpandajoe]

Here is a fix associated with the issue: #30042

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30054

[sadpandajoe]

doesn't feel like a blocker for 4.1, but let me see if I can find a root cause for this.

[sadpandajoe]

Looks like there's a fix here: #30077

[sadpandajoe]

🍒 'd

[padbk]

Probably not a blocker, but I'm seeing this issue #28833 when SLACK_ENABLE_AVATARS and websockets are enabled on 4.1.0rc2.

[sadpandajoe]

Agreed not a blocker for 4.1. Thanks @padbk for finding this

[sadpandajoe]

#30029

[sadpandajoe]

🍒 'd

[sadpandajoe]

#30020

[sadpandajoe]

🍒 'd

[sadpandajoe]

#29997

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30056

[sadpandajoe]

blocker for 4.1

[michael-s-molina]

@kgabryje @geido Can any of you help with this issue? I don't have much context about cross-filtering.

[sadpandajoe]

fix is here: #30176

[sadpandajoe]

🍒 'd

[sfirke]

Not sure if this is worth an issue, but for both rc1 and rc2 my docker-compose deployment fails due to lack of psycopg2:

superset_app          | 2024-08-29 17:22:19,865:ERROR:superset.app:Failed to create app
superset_app          | Traceback (most recent call last):
superset_app          |   File "/app/superset/app.py", line 40, in create_app
superset_app          |     app_initializer.init_app()
superset_app          |   File "/app/superset/initialization/__init__.py", line 474, in init_app
superset_app          |     self.setup_db()
superset_app          |   File "/app/superset/initialization/__init__.py", line 637, in setup_db
superset_app          |     pessimistic_connection_handling(db.engine)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 998, in engine
superset_app          |     return self.get_engine()
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1017, in get_engine
superset_app          |     return connector.get_engine()
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 594, in get_engine
superset_app          |     self._engine = rv = self._sa.create_engine(sa_url, options)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/flask_sqlalchemy/__init__.py", line 1027, in create_engine
superset_app          |     return sqlalchemy.create_engine(sa_url, **engine_opts)
superset_app          |   File "<string>", line 2, in create_engine
superset_app          |   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned
superset_app          |     return fn(*args, **kwargs)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 544, in create_engine
superset_app          |     dbapi = dialect_cls.dbapi(**dbapi_args)
superset_app          |   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 811, in dbapi
superset_app          |     import psycopg2
superset_app          | ModuleNotFoundError: No module named 'psycopg2'

I can start Superset if I add psycopg2-binary==2.9.6 to my requirements-local.txt file.

My guess is that some part of the docker compose deployment process changed and I need to sync a newer version of files in the repo. I'll look into that and I don't think this is a bug, but wanted to post here in case others were experiencing it.

[sadpandajoe]

we can just leave it open for anyone else running into issues but this isn't a blocker for 4.1

[sfirke]

We could add a note to the release notes and/or updating doc, pointing to the docker builds docs where @mistercrunch explained this and noting that as of 4.1.0 there are many drivers/packages that moved into the -dev image and are not present in lean?

(One more such package: I tried testing Excel uploads with the lean image and it failed because openpyxl was not installed. Works great with -dev)

[mistercrunch]

Not a blocker AFAIC, people should not use docker-compose to deploy in production. As far as ASF goes people should be using official tarballs. Adding a note in UPDATING.md about lean docker images being actually lean would be a nice thing to do though. I feel it's been the case for multiple releases though, no (?)

[sfirke]

Pretty sure it's new for 4.1.0, I'm running 4.0.2 and it came with psycopg2-binary etc. I created a PR to add a note to the release notes about it: #30243

[sadpandajoe]

🍒 'd the note in the release notes

[michael-s-molina]

#30338

[michael-s-molina]

fixed in #30339

[sadpandajoe]

🍒 'd

[sadpandajoe]

#30347

[sadpandajoe]

fixed with #30348

[sadpandajoe]

🍒 'd

[benvdh-incentro]

@sadpandajoe can you provide any estimate on when RC3 is going to be released? At our company we're eagerly awaiting the 4.1.0 release due to some features/bugfixes being present in that release. I've been following this thread for a while now (and the one on the dev list too), but so far I saw no one voting on the RC2 release on the mailing list (also no -1's), and only bugs + fixes being mentioned in this discussion that subsequently got cherry-picked into the release branch.

Do not get me wrong, I appreciate all the hard work going into a release, and understand it can take some time to get a release out, both in terms of the size of this project, and most OSS projects relying on volunteers (even though I am aware preset also plays a role here). I was mostly curious...

[sadpandajoe]

@benvdh-incentro as you noted that there haven't been any voting done on RC2 but the number of bugs that have been reported is currently not going to pass. I'm trying to minimize the number of RCs we're going to have to create so we're waiting for at least #30205 to be done before creating RC3 unless if any blockers are showing up in RC2.

[borismod]

Our company is also waiting for SuperSet release and it's a blocker. Snyk identified several vulnerabilities in SuperSet dependencies. There is a discussion to abandon SuperSet at the company level. May I suggest having a minor release that will contain only security fixes?

[sadpandajoe]

@borismod creating a superset release with just vulnerability fixes would still require a vote for it to go through so would take time to get an official release for that too. My suggestion is that you can either use the current RC2 as is knowing that these are the known bugs and they'll be fixed during the official release, or the other option is to branch off of 4.0.2 and cherry-pick they vulnerability fixes into that branch and use that until 4.1 is ready. I know it's taking awhile to get 4.1 out but as you can see from this discussion, there have been a lot of regressions that needed to be fixed.

[sfirke]

If anyone in the community wants to lead a 4.0.3 release, that could be a possibility. They could pick security/bugfix cherries onto 4.0.2. That's a matter of someone stepping forward who is willing and able to do the work.

[borismod]

According to the enterprise policy, we're not allowed to use RC versions. It actually puts us in a position that have to start looking for SuperSet alternatives.

[michael-s-molina]

#30389

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30393

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30405

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30416

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30061

[sadpandajoe]

#30424

[sadpandajoe]

fix here: #30425

[sadpandajoe]

#30258 and #30417

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30502

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30551

[sadpandajoe]

🍒 'd

[michael-s-molina]

#30563

[sadpandajoe]

🍒 'd

[sadpandajoe]

Since RC2 failed, going to close this discussion until RC3 is created. To follow along blockers or to add new issues, feel free to follow the new project board here: https://github.com/orgs/apache/projects/418

[iercan]

This error still exist on our side.
#29729