Let Studio Home REST API determine if libraries v1 and/or v2 are enabled [FC-0062]

[pomegranited]

Description

Removes the LIBRARY_MODE config variable in favor of flags returned by the Studio Home REST API. See openedx/edx-platform#35576 for the new waffle flags added there.

Also adds a "Beta" badge and explanatory text to the Libraries v2 home page.

This change impacts Course/Library Authors.

Screenshots

• Both flags disabled (default, v1 + v2 libraries enabled):
  

• Only contentstore.new_studio_mfe.disable_new_libraries flag enabled (v2 libraries disabled):
  

• Only contentstore.new_studio_mfe.disable_legacy_libraries enabled (v1 libraries disabled):
  

• Both flags enabled (both v1 + v2 libraries disabled):
  

Supporting information

Relates to:

• Add "beta" label and text to new Libraries tab on Studio Home #1336
• Enable Libraries v2 (beta) in production #1324

Blocked by:

• [DEPR]: Legacy ("V1") Content Libraries edx-platform#32457

Private-ref: FAL-3858

Testing instructions

1. Run branch from [DEPR]: Legacy ("V1") Content Libraries edx-platform#32457,
   and ensure the Meilisearch plugin is enabled: https://github.com/open-craft/tutor-contrib-meilisearch
2. View Authoring Home page: should show "Libraries (beta)" tab linked to /libraries and "Legacy Libraries" tab linked to /libraries-v1
3. In the Django Admin, add a Waffle Flag named contentstore.new_studio_mfe.disable_new_libraries and enable it for everyone. Refresh Authoring home page
   The "Libraries (beta)" tab should be gone now, and "Legacy Libraries" should be titled "Libraries", still linking to /libraries-v1
4. In the Django Admin, add a Waffle Flag named contentstore.new_studio_mfe.disable_legacy_libraries and enable it for everyone. Refresh Authoring home page
   No "Libraries" tabs should be shown.
5. In Django Admin, disable contentstore.new_studio_mfe.disable_new_libraries for everyone.
   The "Libraries (beta)" tab should be the only tab showing, linked to /libraries.
6. In Django Admin, disable both libraries flags.
   Should be back to the default view with both "Libraries (beta)" and "Legacy Libraries" showing.

Other information

1. The link in the explanatory text will be updated once we have a tutorial available.

[openedx-webhooks]

Thanks for the pull request, @pomegranited!

What's next?

Please work through the following steps to get your changes ready for engineering review:

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

🔘 Let us know that your PR is ready for review:

Who will review my changes?

This repository is currently maintained by @openedx/2u-tnl. Tag them in a comment and let them know that your changes are ready for review.

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.12%. Comparing base (4facf1c) to head (d4d7671).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1329      +/-   ##
==========================================
+ Coverage   93.09%   93.12%   +0.02%     
==========================================
  Files        1047     1047              
  Lines       20138    20132       -6     
  Branches     4201     4267      +66     
==========================================
- Hits        18748    18747       -1     
+ Misses       1328     1322       -6     
- Partials       62       63       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bradenmacdonald]

@jristau1984 CC @KristinAoki We are planning to merge this PR soon to turn on the new content libraries v2 features by default, for the upcoming Sumac release. But I imagine that 2U may wish to hold off on launching this feature (?). I'd like to check if that's the case. If so, could you please override the LIBRARY_MODE Authoring MFE setting for your environments to set it to LIBRARY_MODE="v1 only"? Then we can merge this PR without affecting you, and you can launch it later at your preferred time. (Or launch it now to start getting early feedback; it doesn't replace any current functionality, just adds a new area of Studio that's totally separate from the v1 libraries.)

LIBRARY_MODE="v1 only" will look exactly the same as it does today

LIBRARY_MODE="mixed", the new default, would look something like this:

[jristau1984]

@bradenmacdonald - Will there be a waffleflag for this experience, so that we can release it in a controlled way for testing/migration?

[bradenmacdonald]

Tagging @kdmccormick and @ormsbee. I realized this will actually be more complicated and we might be jumping the gun a bit.

First, 2U can't turn this on anytime soon because it requires Meilisearch. For those deploying with Tutor, it's easy to get that set up, but obviously it's a bigger lift for 2U. Plus the current reindex_studio command would take too long to run on a huge instance, so needs to be refactored to allow incremental index building for such a large instance.

Second, we can think about a waffle flag to turn it on only for certain groups of users. Since that's a backend thing it might also make sense to detect if Meilisearch is configured, and default to OFF (v1 only) in that case or otherwise ON (mixed). Then we achieve our goal of enabling it by default for the release while not prematurely launching it on edx.org nor for MIT.

Third, we haven't implemented permissions yet so this is mostly useful for global staff - but we'll be trying to add that ASAP.

[pomegranited]

@bradenmacdonald

I realized this will actually be more complicated and we might be jumping the gun a bit.

Noted -- I've moved this to Draft to avoid merging too soon.

@jristau1984

Will there be a waffleflag for this experience, so that we can release it in a controlled way for testing/migration?

There isn't yet, but there can be.. I can refactor out this LIBRARY_MODE config and replace it with two waffle flags, something like:

• contentstore.new_studio_mfe.legacy_libraries: to toggle libraries v1 on and off, default ON
• contentstore.new_studio_mfe.libraries_v2: to toggle libraries v2 on and off, default OFF

First, 2U can't turn this on anytime soon because it requires Meilisearch.

Product have asked us to turn v2 libraries on by default if Meilisearch is enabled. But I think it would be better for operators and more predictable to implement if we have the v2 libraries waffle flag off by default, and spend time documenting everything operators need to do before they can enable this feature, and let them turn it on manually.

Plus the current reindex_studio command would take too long to run on a huge instance, so needs to be refactored to allow incremental index building for such a large instance.

Ach good point -- created openedx/modular-learning#235.

Second, we can think about a waffle flag to turn it on only for certain groups of users.

That functionality is baked into waffle flags, and so the dynamic MFE config hit should take care of that for us?

Third, we haven't implemented permissions yet so this is mostly useful for global staff - but we'll be trying to add that ASAP.

Working on this in the next 2 weeks: #1342

[kdmccormick]

@pomegranited I think replaced LIBRARY_MODE with two Waffle flags is a great idea.

Regarding the toggles:

contentstore.new_studio_mfe.legacy_libraries: to toggle libraries v1 on and off, default ON
contentstore.new_studio_mfe.libraries_v2: to toggle libraries v2 on and off, default OFF

In vanilla Open edX, it's impossible to default a waffle flag to ON. The way we default-ON a bunch of waffle flags is, unfortunately, with a big ol tutor-mfe init script.

Could I suggest instead having to opt-out toggles? If we tell 2U and MIT this now, then they can stick contentstore.new_studio_mfe.disable_new_libraries into their databases right now, which will clear us to merge the toggles into master in the next week or so.

• contentstore.new_studio_mfe.disable_legacy_libraries
• contentstore.new_studio_mfe.disable_new_libraries

[kdmccormick]

But I think it would be better for operators and more predictable to implement if we have the v2 libraries waffle flag off by default, and spend time documenting everything operators need to do before they can enable this feature, and let them turn it on manually.

Ah, on second thought, that is a good point @pomegranited . Maybe the New Libraries flag needs to remain opt-in. Regardless, +1 on the two-waffles idea.

[bradenmacdonald]

@kdmccormick @pomegranited I disagree, and I want to have it so that (1) Tutor installs and configures Meilisearch by default (the plugin code is moved into the core, just like for Elasticsearch), and (2) v2 libraries beta is on by default and admins must opt out. The reason is that we saw with Redwood, almost nobody has tried out any of the new taxonomy features or course search features if they require even the slightest effort to set up.

[kdmccormick]

@bradenmacdonald I am on board with that. Two opt-out waffles, then?

[kdmccormick]

Just confirming for all parties involved: There will be two new Waffles:

1. contentstore.new_studio_mfe.disable_legacy_libraries
2. contentstore.new_studio_mfe.disable_new_libraries

In order to keep the current (legacy-only) experience, operators will need to force-on waffle (2).

[pomegranited]

👍 I'll work on adding those two "disable" waffle flags.

@bradenmacdonald

I want to have it so that (1) Tutor installs and configures Meilisearch by default (the plugin code is moved into the core, just like for Elasticsearch)

I see you've already added Meilisearch to tutor contrib plugins -- I will submit a PR to move it to core and make whatever updates are needed to enable it by default.

Do we need to get https://github.com/open-craft/tutor-contrib-meilisearch moved to openedx, or is it ok to stay under open-craft?

Also, has anyone raised this with BTR? I can start that conversation if needed.

[pomegranited]

@bradenmacdonald @kdmccormick

How should these new flags interact with the libraries_enabled / FEATURES['ENABLE_CONTENT_LIBRARIES'] flag? Would be nice to remove that feature flag if we can, to ease confusion.

[kdmccormick]

@jmakowski1123 , right now there is a feature flag called ENABLE_CONTENT_LIBRARIES. It defaults to True (legacy libraries on) but some site operators override it to False (legacy libraries off).

For those site operators who have set it to False, what should happen with New Libraries in Sumac? Should they see the just the New Libraries experience, or should all Libraries features remain disabled?

[bradenmacdonald]

👍 I'll work on adding those two "disable" waffle flags.

@bradenmacdonald

I want to have it so that (1) Tutor installs and configures Meilisearch by default (the plugin code is moved into the core, just like for Elasticsearch)

I see you've already added Meilisearch to tutor contrib plugins -- I will submit a PR to move it to core and make whatever updates are needed to enable it by default.

Do we need to get https://github.com/open-craft/tutor-contrib-meilisearch moved to openedx, or is it ok to stay under open-craft?

Also, has anyone raised this with BTR? I can start that conversation if needed.

@pomegranited Before you open any PR, please discuss with Regis and BTR to ask what they'd like. I'm open to handling this however Regis prefers - keep the plugin separate, change plugin ownership, or integrate the plugin into tutor core. As I've mentioned, elasticsearch isn't a plugin so by analogy meilisearch perhaps "shouldn't" be a plugin either. But it is nicely contained as a plugin.

[bradenmacdonald]

Even if I set contentstore.new_studio_mfe.disable_new_libraries, I can still access the new libraries UI by going to the URL like http://apps.local.openedx.io:2001/course-authoring/library/lib:OpenCraftX:ALPHA/

Is it possible to disable the route when the waffle flag is disabled? Or have the <LibraryProvider> block any children in that case?

Otherwise, looks great!

[pomegranited]

Is it possible to disable the route when the waffle flag is disabled? Or have the block any children in that case?

It's surely possible, but it requires dealing with redux, see useStudioHome. So will leave this as a bugfix for later.