Neptune Notebook

The Neptune notebook is a minimal web-based notebook environment for interactive computing.

intro.mp4

Heavily inspired by https://jupyter.org/ and https://plutojl.org/.

Note

Goal was to create a minimal, lean alternative to Jupyter notebooks. This is NOT and was NEVER meant to be a replacement for Jupyter Notebooks. Have that in mind. This is a simple, minimal alternative (single binary) that is easy to run.

If you need a more advanced notebook, stick with Jupyter Notebooks.

Currently, only GNU/Linux and macOS are supported. If there will be any interest in this native Windows support can be added. Until then, use it with WSL2 under Windows.

Supported environments

Environment	Keyword	Debugger	Note
Python 3	python3	Yes	Supports virtual environments
Bash	bash	Yes
Shell	sh	Yes
Ruby	ruby	WIP	With the use of IRB
Z shell	zsh	WIP
PHP	php	Yes
Node.js	nodejs	No
Lua	lua	WIP
R	r	No	R console
SQLite 3	sqlite3	No
Tcl (tickle)	tcl	No

You can also check which environments are available on your system with neptune -check which will test and list all available ones.

Take Neptune for a spin using Docker

If you have Docker installed, you can quickly give it a test run by using the following container. This container has all the supported environments already preinstalled.

# Find out more about the options.
docker run -p 8484:8484 -it mitjafelicijan/neptune:latest neptune -help

# Starts in python3 environment.
docker run -p 8484:8484 -it mitjafelicijan/neptune:latest neptune -host=0.0.0.0

# Starts in nodejs environment.
docker run -p 8484:8484 -it mitjafelicijan/neptune:latest neptune -host=0.0.0.0 -environment=nodejs

Docker image is built with Alpine Linux and has a couple of packages like requests, pandas already preinstalled. You can check more at https://hub.docker.com/r/mitjafelicijan/neptune. Image is relatively small (~90MB) for everything.

Keyboard shortcuts cheatsheet

Operation	Win/Linux	macOS	Description
Insert cell	Ctrl+B	⌘+B	Inserts a new cell at the end.
Execute	Ctrl+Enter	⌘+Enter	Executes the current cell.
Clear	Ctrl+M	⌘+M	Clears all the cells from the current notebook.
Save	Ctrl+S	⌘+S	Saves currently opened notebook into browser's local storage.
Export	Ctrl+E	⌘+E	Exports currently opened notebook as a source file.

Why another notebook environment?

Why indeed? I like building small tools. One of my favorite things. Let's get that out of the way first.

I like using Jupyter Notebooks, and often I find myself wanting to have the same experience using other languages in the same manner. But, I wanted a simple way of starting a notebook without tinkering and installing libraries and configuring stuff and getting lost in all the information out there. A single executable binary that doesn't require any special environment to run.

This comes with drawbacks. Simple tools usually solve simple problems. One of them is that outputs that you get into the browser are not formatted really well. They are a dump of the interpreter spit out.

Jupyter has numerous rich widgets to display tables etc. This can still be added to Neptune, but it would require work, and it would need to be language-specific. Until then, raw outputs are good enough to have the context of what is going on.

Installation and basic usage

One of the possible ways of running the notebook is through Docker, described above. But normally, you want to run on your actual system.

Under Releases you will find pre-build binaries for GNU/Linux and macOS systems.

Download the binary for your system and architecture and follow the next steps. The example below is for x86-64 GNU/Linux, and the procedure is very similar on macOS.

# Extract the tarball.
tar -xvf neptune-linux-amd64.tar.gz

# Move executable to $PATH.
mv neptune ~/.local/bin

Neptune is now accessible from everywhere and you can start using it.

# Shows all the argument options.
neptune -h

# Checks which environments are available on system.
neptune -check

# Start a server with different environments.
neptune -environment=python3
neptune -environment=nodejs
neptune -environment=bash

# Exposes the server over the network.
neptune -host=0.0.0.0 -environment=python3

A few key points about using the notebooks

• Security wise, this is NOT a secure system. You are essentially exposing interpreters over the network. However, I did add a token-based verification/authentication, which is needed to execute code. This is very similar to what Jupyter Notebooks do, even though more simplistic. Furthermore, by default, the server starts in a mode in which only localhost connections are accepted. If you need to expose this over the network (for everybody), use a flag -host=0.0.0.0 when starting a server.
• If you run a notebook with python3 environment and have virtual environment activated the environment will take that into account and use python3 binary from the virtual environment.
• Each time you refresh a browser page, a new instance of the environment gets spawned. This is important because if you refresh the page the state of the previous notebook (such as variables etc.) gets reset. Be mindful of that.
• When you save a notebook, the contents get saved to browse's Local Storage. These files do not exist on your filesystem.
• Clearing a notebook removes all the cells from the current notebook without restarting the environment.
• Restarting via user interface will stop the current environment process and create a new connection. This will also restart the state.

Debugging

Not all environments allow debugging. Some of them either don't support getting global state or with something like SQLite3 it doesn't make sense.

Debuggers as they are in the current state are limited and will improve in the future if there is expressed interest.

TODO: Add image of debuggers in use.

Local development

If you are using Nix you can avoid installing Golang and other environments for testing and use the shell provided in the repository with nix-shell shell.nix and then continue to the cloning of repo etc.

You will need Golang v1.20 or more installed. Other than that nothing else is needed. Frontend does not use any build system. It's just good old-fashioned JavaScript. All the frontend stuff is located in resources folder.

git clone [email protected]:mitjafelicijan/neptune.git
cd neptune
make dev

This is enough to get you started. This will start a local server on port 8484 and provide you a link to the page. make dev does not open a browser by default because during development this can get annoying quickly.

If we inspect Makefile target dev closely, we can see that two environmental variables are passed to it NO_BROWSER=1 and USE_LOCAL_RESOURCES=1. NO_BROWSER one disables opening system's default browser on server start and USE_LOCAL_RESOURCES uses local filesystem for accessing templates and all the files in the resources directory. Without it, resources directory gets embedded into the binary and accessed through embed.FS. When developing locally, this allows us to refresh the page without restarting the server when we change files in the resources directory.

Variable	Description
NO_BROWSER	Does not open browser on server start.
USE_LOCAL_RESOURCES	Uses local filesystem instead of embedded one.

NO_BROWSER=1 go run .
USE_LOCAL_RESOURCES=1 NO_BROWSER=1 go run .

You can combine this with something like https://github.com/cosmtrek/air but I prefer to manually restart things when needed.

Payload structure

Each cell that gets executed has a bunch of additional things appended to. Because the reading of the standard output of cells is done in an asynchronous way, we need to tag those outputs so we know which cell the response belongs to.

This is how a normal payload to interpreter looks like. DEBUGGER_INFO is not used in all the environments due to some of them not supporting getting the state of the environment. Read more about what is supported in section Supported environments.

#=NEPTUNE::START_CELL::{cell_id}

  code gets executed here

#=NEPTUNE::DEBUGGER_INFO::{symbols}

#=NEPTUNE::END_CELL::{cell_id}

On that note, if you were to print out this string #=NEPTUNE::{string}::{string} in your code intentionally, this would confuse or even break the application. Bear that in mind.

Potential feature idea: This could be used to our advantage in the future to introduce another panel to the UI where you could store custom things, similar to the watch option in debuggers.

Future plans / Whishlist

• Add support for macro similar to !pip but with !sh that could run arbitrary commands instead. That is potentially very dangerous for the host machine but then again, these notebooks rely on people knowing what they are doing.
• Add support for macro !image which would local image and display it in output cell. This could be useful when you are creating charts with something like Matplotlib in python but instead of doing .show() you would save the image and then add !image macro at the end of the code block with image path and that would also display that image.
• Add support for debugging panel that would show all currently used variables and their values in the current scope to help with the development.

Special thanks and acknowledgment

• Re-sizable split panels - https://split.js.org/
• Reordering of code and output blocks - https://github.com/SortableJS/Sortable
• Code editor and syntax highlighting - https://github.com/codemirror/dev/
• Icons - https://devicon.dev, https://fonts.google.com/icons

License

neptune was written by Mitja Felicijan and is released under the BSD two-clause license, see the LICENSE file for more information.