-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Discuss] The future of the classic notebook interface #6210
Comments
I'll do my best to document some of the things discussed in our weekly Notebook meetings. |
A major issue is that this repository still receives a steady stream of issues and PRs, even though the project is technically in maintenance mode, meaning we'll only review bug/security fixes. |
The challenge is that most of the people currently "maintaining" this repository are not frontend developers. When issues are raises in the Javascript APIs, the team lacks the experience to review/respond to these issues. Further, the Javascript is a bit outdated, so even frontend develops don't want to engage these issues. |
As a result, there is a heavy push from the developer community to move this project into an archived state and encourage users to switch to JupyterLab or other Jupyter frontends (e.g. nteract, retrolab, etc.) |
An alternative proposal was to offer this project as an opportunity for another team/company/group to maintain this project moving forward. There might be some team with vested interest in making sure this project stays alive indefinitely. There was some concern raised with this idea. Handing off the project to another (possibly not Jupyter-affiliated) team puts the Jupyter brand (?, just echoing what I've heard said here) at risk. |
The urgency of this issue (i.e. what to do with classic notebook) comes from a growing fear/possibility that notebook will break permanently because of decaying dependencies.
Moving the classic notebook project off of these dependencies is a massive amount of work. |
No matter what we decide, this issue impacts a large number of users. We'll want to carefully consider how to proceed, seek out consensus amount developers and users, and provide the necessary messaging to help everyone have a clear understanding of the future. I hope this issue helps promote this discussion. |
This issue has been mentioned on Jupyter Community Forum. There might be relevant details there: https://discourse.jupyter.org/t/the-future-of-classic-notebook-interface/11328/1 |
Could a jupyter notebook interface be developed that would approach the philosophy/aesthetics of a UI similar to the SourceHut ecosystem/suite of apps? That style of app would be more maintainable for a classic notebook in the long term with minimal dependency on fast moving js libraries. That said, I imagine this would be a ton of work to implement :) |
Apologies in advance for a long post... This repeats a lot of things posted elsewhere, but I think I still haven't found a way to express the growing gulf I see between the experience and expectations of highly skilled developers, and people who are very naive end-users of Jupyter tools doing very, very simple tasks that appear to be little more than (and often are very little more than) the following to a seasoned developer:
At the same time, this is not to say that we shouldn't be providing such novices with environments that are, underneath, very powerful, and scalable in terms of what you can do with them. What I try to do below is explore what notebooks made possible, who they empowered, and what we are at risk of losing. As I've commented on this issue many times, I'll argue the position of:
Skill level of users: JupyterLab IDE is overkill. Classic notebook UI is at the limit of UI complexity; notebook extension toolbar buttons bewilder a significant a percentage of users and make the UI unusable. Duty cycle for upgrading software: 5 years +. (In my institution, modules run for 5-6 years then possibly get extended for another 3+ years. They take two years to produce. The software freeze (except for minor updates once a year) is typically during the first year of production, so things then need to stay largely in place for the next 7-8 years. In general, educational materials get reused year on year on year. Computers in K12 schools may be 5 years+ old. Note that education needs are global and extend to developing nations not just Ivy League feeder High Schools and inner-city deprived ghetto schools with low student expectations and harassed teaching staff. The original notebooks allowed people who could write basic HTML pages to write extensions; the My skill level excludes me from updating my original classic notebook extensions. There is no internal developer support available eg for porting extensions to JupyterLab. I do not have the skill or the time to learn either js frameworks or js dev tools, let alone make sense of JupyterLab API docs and how-to example fragments that are buried in code tests. My instructional material relies on classic Jupyter notebooks with their current extensions. My institution has large numbers of developers working in institutional IT projects and growing numbers of courses using Jupyter notebooks, but with zero central IT or developer support provided to support the development of educational end-use of Jupyter notebooks. At the moment, pretty much all "development" is carried out by academics who are not developers (why should a statistician using LaTex and R know how to develop front-end web apps?) and who are supposed to be spending their time developing subject matter material and teaching students. For them, the Jupyter notebook environment is a means to an end - a way of maximising the chance that a novice with very poor computing skills can run 3 lines of code and see the result. The notebook model means the code can be placed in instructional text and made into an interactive textbook. In many cases, Jupyter Book + Thebe (and even better, thebe w/ a pyodide kernel) would be appropriate for supporting this sort of interactive text book use case. But it lacks the ability to persist code changes or add and persist new cells against a user account. In other cases, the more general authoring environment that classic notebook allows users to populate their own notebook (a digital version of a kid's paper school exercise book that they write their homework in), with text, media assets, maybe some code and code outputs (with a total single lines of code count across the whole notebook in the low tens at most). A single notebook might take up a complete K12 school lesson (35 minutes) and be the equivalent of a page or two pages of A4 paper. My fear is that this large body of legacy users, who you are unlikely to see on Github, are going to be left in limbo (why would a K12 maths teacher in a rural school who has a dozen or so notebooks used over a whole school term to run some simple ipyturtle activities, need a Github account?). I think we saw some signs of this when a recent classic notebook update clobbered inline HTML styles in markdown cells. Classic notebooks provided folk who don't have access to webservers, and don't need the grief of installing or supporting even basic, default Python environments, with a read-write-render-execute medium for HTML, rich media (embedded images, videos, audio), and REPL based code execution and code output previewing. Teachers of 12 year olds could get instructional material they'd written in front of kids, and the kids could run the code, write their own, write their own notes, and save a copy of it. That's what drove my adoption. Classic notebook was an on-ramp. If I came to Jupyter now, and was exposed to JupyterLab as the first experience, I'd probably walk away because I'd stand zero chance of persuading anyone else it was something we'd be able to use with very novice distance education students. (It's been hard enough selling classic notebooks.) Another fear is that a company like Pearson will pick up the maintenance of classic notebook and then lock users in. A regret is that institutions such as my own (which contributes to Moodle core development, for example) do not appear to value the benefit of putting support into classic notebook, not in terms of improving accessibility, nor providing maintenance effort, nor helping turn educator contributed have-a-go extensions into production stable extensions that can be of general effort. The Jupyter project is a phenomenal undertaking, but it's evolution is starting to mirror that of the web. A couple of decades a go, you could see something interesting on a web page, view source, and stand a good chance of being able to appropriate the HTML+CSS code you saw and do something interesting. Classic notebook provided a UI that supported that level of engagement. But it's not what developers who want to push things forward are interested in. We need to find a way of resourcing some developers who want to push things back to as simple an environment as possible, that still affords the power of working with other Jupyter components. IMVHO!;-) |
Bower is a problem, but it seems like it should not be too hard to port that over to webpack or similar. Tornado, however, has had frequent releases (at least until October 2020) and can handle the load, so I'd like to understand that concern better. |
I agree with @psychemedia and would like to add some more to their arguments. In my opinion before we give up on the classic notebook a number of things need to happen:
The problem I see with JupyterLab is that the vast majority of people working on it are developers/programmers. So they are developing a tool to do what they do. Many of us need tools to do other things and the classic notebook is better at those other things. The very best software I have used over my 40+ years was either written by the people who use it or a team consisting of the people who use the software and collaborating expert programmers. Here's a list of some of the very best pieces of software I have used: IgorPro (commercial, by Wavemetrics, lab scientists writing tools for lab scientists); Gamess (opensource quantum chemistry); VMS (Vax operating system, still available as openVMS); Linux/Unix; Python (I hate the syntax, but its ease of use and facile sharing of modules makes it a winner), Jupyter notebook, Jmol (opensource), Plone (opensource). JupyterLab does not make this list. If people are interested in some of the projects I am working on that benefit from the classic notebook, see Jupyter Physical Science Lab. |
I completely agree with @psychemedia. In the early IPython notebook days, I saw how the simple, intuitive classic notebook helped make coding more accessible for young coders, people who were returning to the workforce, and people who had had horrible first coding experiences (i.e. women in their 20s and 30s who were told by arrogant teachers that they couldn't program because they weren't good in math and science). Much like the early Macs and iPod, the classic Notebook is an easy to learn, approachable, and intuitive tool (after teaching folks Shift+Enter). It's also the user interface that drove the success of JupyterHub and Binder. While JupyterLab serves advanced users and Jeremy's notebook based on JupyterLab is a nice project, the classic notebook appeals to a large group of users who value a computational notebook that is a communication tool with the ability to do computation and make concepts easier to understand with the use of prose, visualizations, and multimedia. While I do think the classic notebook needs a more modern JS foundation, I think there are many users and use cases where the classic notebook is preferred. As @stefanv mentions, Tornado is still viable and moving to asyncio or another async framework is certainly possible. |
I've been struggling to try to articulate my issues with JupyterLab for several years, so value the additional takes provided by @gutow and @willingc. I don't want to knock the huge contributions made by the core developers but I think part of the issue is that for the not-developer community it's hard to know where to raise concerns. Github is a development environment and issues (and even more, PRs) are often not the place, or not felt to be a place, where non-developers feel comfortable in raising what we recognise as technical issues. Part of this is because we don't necessarily have the vocabulary or technical understanding to be able to communicate our concerns in what is essentially a specialist technical community workplace. And part of this is that our concerns, as users of medium, don't resonate with the developers who don't see why we want to do particular things in a notebook (example answer: because we are using it as an authoring and worksheet medium to print out classical Latin declensions, draw electrical circuits, work with very simple musical scores, author chemical compound lookup sheets, or render flashcards, not as a development environment or a scripting environment). One way I think of Jupyter notebooks is a read/write/execute environment for communicating scholarly content. I don't care about the plumbing. The notebook UI surfaces a simple development environment for me where I can write HTML and simple javascript, or more commonly, generate rich outputs using third party subject-matter-specialist packages, or perhaps author some simple magics, at a level that it very close to the end user but still hidden from them. Some very simple patterns were evident in the original Jupyter notebook extensions that were enabling of a wide range of customisations. For example, an extension that iterated over cells, tested cell type, and then applied simple styling to HTML of the cell is very powerful and usefully reusable. I still can't fathom how to do this in a JupyterLab extension (in the following I deliberately abuse all sorts of notation to make a frankenstein flavour of pseudo-code):
Another really powerful pattern is a toolbar button:
As soon as that gets buried in a 150 lines of boilerplate, plumbing, variously nested flavours of bracket, awaits, development environments, node, gulp, test, and poetry, I really don't know where to start. I think another gulf is that a lot of "educational hacks" are bits of code to get a job done once and once only for a particular end user task. The code only has to work in a very limited context (known to the author), potentially only have to work once, and even then, potentially under supervision, where you can say "don't try this because it won't work" and then move on. This is not the same as production code... |
As I read the comments made so far, I notice that the theme seems to be that the classic Jupyter notebook is being abandoned by the original authors because they are moving on to something they can envision as a better tool for the things they do (generate "production" code). I also think they believe JupyterLab will be easier to maintain. So the questions is how do we convince a core of developers to maintain or update the Classic Notebook in a way that is useful for the large number of users who do not need an IDE in a browser, and actually find it a hindrance to their work? My personal opinion is that there are already other very good IDEs out there (PyCharm and the venerable Eclipse to name two). Thus, those working on JupyterLab are spending their time rewriting an existing tool (reinventing the wheel) in a new front end (web browser) rather than developing something truly new and exciting. The only part of JupyterLab I can interpret as a genuine improvement over existing options is its potential for real-time collaboration on the same piece of code (I believe PyCharm already supports something similar). I've been using computers long enough to be confident that we will cycle back to more localized computing from cloud computing again. When I started using computers most computing was sent to computing centers (the equivalent of today's "cloud" computing). Then, except for really big jobs, the computing was distributed to PCs. Recently, the "cloud" has been popular again. However, you can already see the return to more localized computing starting up with the discussion of "edge computing for IOT". I think there will be a mix, so JupyterLab is not going to win out over other IDEs that run more locally, just because it is inherently a server-client application. A second reason pure cloud computing is doomed is that anything in the cloud is accessible by anybody who is determined enough. This means there will always be a need for localized computing resources and tools. A third reason pure cloud computing is doomed is that there are many instances where being on a network is not practical. So what do we need to do to get help maintaining and updating this really useful tool? |
Some ideas for a path forward but migrating to JupyterLab:
|
Many thanks for opening this issue @Zsailer . One important thing to note is that many data scientists (including me) choose to use classic notebook, despite having the technical proficiency to use Lab, since it meets our needs better. Classic Notebook is not just something that people use because they find Lab too complex. Classic Notebook has had years of fine-tuning such that for people that use it every day, it's exceptionally productive. For instance the rapid ability to jump around sections with keyboard shortcuts provided by the collapsible headings extension, or the direct access to editing keyboard shortcuts from any notebook menu. In the most recent Kaggle developer survey, more than twice as many people use classic notebook than use lab. People really like it, and for good reason! I'm extremely grateful that the classic notebook project hasn't been archived, and that some kind folks are continuing to maintain it. |
Trying to do extensions in lab is still very much a moving target. I haven't been able to replicate all of our previous extensions in JupyterLab yet, and some I haven't even been able to truly map because of the complexity of the new APIs for extensions and the sparseness of examples compared to classic. Some features I needed to complete the extensions didn't even exist till Lab 3. I have even been able to actually devote a small portion of my time to writing lab extensions, as opposed to any organizations that don't have the resources to do so. We had several, more casual, developers who contributed to the classic extensions. Now, I am the only one writing our Lab extensions as the casual devs either couldn't wrap their heads around how Lab extensions work, or just didn't have the time to try and figure things out from what examples are out there. While I love the Lab interface, writing extensions for it (which we absolutely need for our environment) has been a bear and so we still lean on classic for some things and probably will for a while. |
Yeah tornado is a problem in the Jupyter stack general due to lack of maintenance - and the fact that it's async model is essentially a compatibility wrapper on top of asyncio. But this specific issue is accessory to the main discussion here. |
I support putting the classic notebook in an archived state, which would reflect the level of support and maintenance that the project is currently receiving. If we don't communicate this clearly, we will be failing our users who expect more. The "simple mode" of lab and "retrolab" both offer a more maintained implementation of the "full-page single document" model of the classic notebook, that benefit from all the work of the JupyterLab team on all other aspects (collaborative editing, accessibility, visual regression testing, internationalisation). |
@SylvainCorlay - the general theme on this thread seems to be "please, please don't archive this project, I need it" - but you are saying you think it should be archived anyway. Do you mean to say that the various arguments that people have made here, that the classic notebook continues to be a better match for their needs - are based on a misunderstanding? Or that the needs of the project developers to move people onto the newer platform must override the needs of the users? And if you're saying neither of those things, surely it would be better to consider the options for continued support the classic notebook? |
On Sun, Oct 24, 2021 at 8:49 PM Jonathan Gutow ***@***.***> wrote:
As I read the comments made so far, I notice that the theme seems to be
that the classic Jupyter notebook is being abandoned by the original
authors because they are moving on to something they can envision as a
better tool for the things they do (generate "production" code).
I realize this is somewhat off-topic - but that is a related question for
the JupyterLab developers - do you, in fact, use JupyterLab for writing
production code? In the past, when I've asked Jupyter developers, they
have said they do not use Jupyter very much, for their own work, largely
because writing code for libraries, like Jupyter, really needs a proper
code editor like Vim, Emacs, PyCharm or VSCode.
|
@jph00 for one thing, I think the JLab version is only mouse-based. I'm not familiar with the details on how this works keyboard-wise in Classic, but perhaps we can open an issue to track this specific point. I'm a huge fan of good hierarchical document management -- the LyX navigator/outliner is IMHO still better than many others, as it also allows for safe structure-based reordering of the document. So improvements to this that help all users would be fantastic (on Retro/Classic but also on Lab!). |
Yes that's right, lab doesn't have the keyboard shortcuts, which is the key thing - since with the keyboard shortcuts one can jump to start and end of sections so easily, collapsing them actually becomes much less important. Note also that collapsible headings integrates with the TOC2 extension to allow them to optionally sync collapsed sections.
Here's what the docs say about the keyboard shortcuts:
* Command-mode keyboard shortcuts, (enabled by default, and set to left and right arrow keys to collapse/expand sections, or go to the previous/next heading, plus shift-right to select a heading cell's section, shift-a/b to insert a heading above/below the current cell, ctrl-shift-left and ctrl-shift-right to collapse/uncollapse all headings). Bindings are also configurable from the config page
Details: https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/collapsible_headings/readme.html
|
I opened an issue in the JupyterLab repo about collapsible heading shortcuts to track this. |
Re: @Zsailer
We use
The first set of extensions should be easily portable to JupyterLab or any other Notebook interface for that matter and in my personal opinion doesn't require a specific Jupyter Notebook version or setup, in so long as it's possible to develop extensions that can communicate with the file system and that can edit ipynb metadata (file-wise and cell-wise). We are currently porting the client extensions to use the JupyterLab setup but most of our users ("our" is IllumiDesk, we are an EdTech company that leverages nbgrader and Jupyter extensively) prefer the Classic interface so are developing the extensions to maintain compatibility with the RetroLab setup. We decided on embarking on a complete rewrite of item 2 above ( |
Can jupytext be added to the list of extensions that aren't yet fully compatible with JupyterLab? I think having text-based formats (rather than |
Duly noted @astrojuanlu, thx! See the PR above where the actual JEP is starting to take shape. |
Adding to the comment I was tagged on above by @jgwerner — I use the I have dozens if not a hundred student assignments written over several years (plus re-write or tweak exams every semester). These are all auto-graded using |
Hey all - just a note that we're nearing final language around the Notebook v7 JEP, please provide comments and thoughts soon before we head to a vote. As there has been a lot of discussion and refinement already, it'd be most-helpful if comments are focused around specific and actionable things to improve the proposal! jupyter/enhancement-proposals#79 |
@choldgraf Just an observation that if you wanted to broaden comment outside the dev community, expecting folk to know how / where to comment on elements of the proposal via GitHub PRs makes huge assumptions about the skill base of the folk whose comments you want to solicit... |
Wow, that is a lot of work in a short time, @choldgraf! I'm thankful to all the authors who put in work on the concrete proposal! |
@choldgraf I'm not sure if this the right channel or not for this offer, but we would be more than willing to sponsor the development of the We have the cc/ @labarba |
Hey all - I'm going to close this issue, as the JEP in jupyter/enhancement-proposals#79 was approved, which means that our immediate term plan will be to base Notebook v7 on JupyterLab components, with the goal of making the package more maintainable and also bringing in new features that have been created in the JupyterLab ecosystem. I've included links in that PR so that we keep this conversation here for reference, because there were a lot of excellent perspectives, ideas, and opinions shared. Thanks everybody for being so thoughtful and open about your experiences |
The purpose of this issue is to discuss the future of the jupyter/notebook repository.
Background: The classic Jupyter notebook interface is currently maintained by a skeleton crew of developers. On the other hand, many people still use the classic interface today.
Problem: We have a widely-used piece of software with very little developer support.
Let's have an open discussion about what to do with this project moving forward.
The text was updated successfully, but these errors were encountered: