Experiment with integrating with (possibly shipping with) nteract

[embray]

It seems nteract might provide everything I've been wanting in terms of better desktop integration experience. I had heard the name before but didn't really realize what it did. At the high level it has the key features I've been wanting for better UX of Sage in Jupyter, particularly:

• A native UI experience--it still uses a web rendering backend, but in a simplified window that hides away as much of the "browser" as possible
• Similarly hides away the terminal running the notebook server, and provides a UI (presumably, I haven't tested it yet) for managing the notebook server process.

Remains to be seen how easily I can integrate it with Sage's Window installer, but it should be possible in theory.

[embray]

Based on my investigation so far, there are a few issues (though not necessarily showstoppers). It seems the Windows version of nteract is using the Windows packaging facilities provided by Electron (the cross-platform framework on top of which nteract is built). So a lot of how the current Windows package is built depends on Electron itself it seems.

Out of the box it comes with just one kernel, a Node.js kernel (since Electron is Node-based). It can work with any other Jupyter kernels of course, at least in principle, but it's not entirely clear yet how to configure that. The Windows version anyways leaves something to be desired in this regard. There are no actual configuration options I could find or instructions on how to get it to pick up other kernels.

If we want a more "Sage-branded" experience it might actually make more sense to fork nteract and add our own branding, as well as the requisite configuration to get it to find Sage's Jupyter kernel and load the Sage kernel by default (though it would still be nice to include other kernels).

I'll get in touch with the nteract team and see what their thoughts are.

[rgbkrk]

Hi @embray! I'm one of the leads on nteract and would be super happy for you all to develop on nteract, even if it ends up being a white labeled version specifically for sage. Speaking for me personally, I haven't had cycles for the desktop app lately so I could use others helping with it and encouraging this kind of usage. Much of the electron and cross platform builds work was a lot of pain, which became much easier the further integrated we got with electron builder tools. That's where you'll end up seeing a lot of the tradeoffs we've had to make so we have a nice to build and ship desktop app.

Kind of off the cuff, here's my current thinking:

Fork the nteract app by using what's in applications/desktop. As you end up finding shared packages that you'd like to co-maintain with us, we'd love to keep them in packages/ within the monorepo (more on this in the "Designing for maintenance" section of our design docs). My only fear would be a loss of synchronized changes on the UI (which is where I spend most of my work on the project now).

Another option would be to build what you need inside the nteract repo, possibly as a new module in applications. It could be a more configurable / easily white-labeled version. I don't know what the difficulty of doing that would be.

As for hooking up the kernel, the way the built-in node kernel works is that we add it in to our kernelspecs on startup. The ijavascript script file is bundled as part of the app. I suppose if the sage kernel is exposed in a similar way -- built in kernelspec, binaries included as part of the shipped app, then you would have sage included as a default. Folks have actually looked into doing something similar with a built in conda environment via snakestagram.

That's all I've got for now, excited to see this as an experiment!

[embray]

Hey @rgbkrk thanks for your thoughts on this and sorry for the delay. This came up just before I left for several weeks of travel, and it's taken me the better part of the last month to get caught up this far :(

When you say "cycles for the desktop app" I'm not sure exactly what you mean by that, because my understanding was that the entire point of nteract is that it's a desktop app, but maybe I'm missing something. I need to spend more time studying electron, since without understanding its basics it's going to be hard for me to dig into nteract.

So most of what you wrote here went over my head right now. I will spend some time studying electron, and nteract's development docs that you linked to.

One thing I'm currently baffled by is simply where the Python interpreter lives. Or is there even a Python interpreter?? Did you guys re-implement the Jupyter core in Node?

[rgbkrk]

Hey @rgbkrk thanks for your thoughts on this and sorry for the delay.

No problem. :)

When you say "cycles for the desktop app" I'm not sure exactly what you mean by that

That meant "spare time" -- I haven't been able to do as much on the desktop app as I would like, compared to other work (the nteract app for the jupyter server).

I need to spend more time studying electron, since without understanding its basics it's going to be hard for me to dig into nteract.

Electron quick start is largely how I got started with electron which at least shows the basics of the main and renderer process.

Most of the tricky parts to understand are node, subprocesses, and the jupyter specifications (notebook format, kernel specs, messaging protocol).

One thing I'm currently baffled by is simply where the Python interpreter lives. Or is there even a Python interpreter?? Did you guys re-implement the Jupyter core in Node?

The Python interpreter is invoked as a child process of the electron app, provided the user has ipython installed. The electron app communicates directly over a TCP Socket (ZeroMQ based), speaking the Jupyter protocol, directly to/from an interpreter (the kernel). No jupyter server is launched and jupyter doesn't even have to be installed. We look into the folders where kernels are supposed to be installed to, and use those to launch the kernels (as well as generate the menu of available kernels).

When it comes down to it, electron provides two primary things we use:

• APIs for native functionality like menus, file dialogs, etc.
• Access to any node.js builtin

The second one, using node, is the real underpinning to nteract. Electron makes it convenient to launch the kernels since we can require('child_process') directly, or use a library that does it for us. We wrote this one called spawnteract that launches a kernel given a kernel name:

const spawnteract = require('spawnteract')

spawnteract.launch('python3').then(kernel => {
  // Returns
  // kernel.spawn <-- The running process, from child_process.spawn(...)
  // kernel.connectionFile <-- Connection file path
  // kernel.config <-- Connection information from the file

  // Print the ip address and port for the shell channel
   console.log(kernel.config.ip + ':' + kernel.config.shell_port);
})

I wrote up a little hokey utility that tries to elaborate on the fundamentals in https://github.com/rgbkrk/grumble. Not sure if it's readily consumable, as I wrote it specifically for someone new to nteract yet not new to RxJS (an underlying library we use for managing asynchronous events).

---

We're holding some sprints for nteract in a few weeks, you should join us!. Not sure what your schedule is like, though a bunch of us will be working on various nteract, jupyter, and binder things that week. My current plan is more documentation, which should likely benefit you soon. 😄

[slel]

Related Stack Overflow question:

• Stack Overflow question 62830376: Have SageMath notebooks open on double click in Windows