Followup from twitter

[Carreau]

Proof of concept for dynamically rendering IPython Notebooks using Flask

Yep, I used flask at the beginning of nbviewer, but we definitively need something async
to handle many requests in parallel hence we moved to tornado.

Unlike nbviewer each code cell in a notebook is executed on request.

Cool, using ipython/ipython#5639 and ipython/ipython#5720 nbconvert should get the ability to run the notebook on the fly which will then only be a config option on nbviewer.
your input on it would be great. It won't be enable by default for obvious performance reasons, and also because most of the time we won't get the library installed.

Running app.py starts the flask site which renders the markdown and updated code output cells.

Yes, work great locally for me.

Notebooks are rendered within the existing flask site using the normal templating approach.

I'm not sure I get what you mean here. Can you develop ? Do you think we can do better in nbviewer itself ?

Pages can be developed by starting the IPython Notebook server in the notebooks directory.

I do not get that either, do you mean that your app is serving files from the local filesystem ?

[iiSeymour]

So the usecase here is the ability to render notebooks seamlessly within an existing site/framework, abstracting the fact a page has been developed as a notebook. Kind of like a scientific CMS where the IPython Notebook server is the WYSIWYG editor. So if a notebook is doing some analysis that draws data from a source that is continually updating i.e. stock data, then the rendered view should show the upto date analysis.

Cool, using ipython/ipython#5639 and ipython/ipython#5720 nbconvert should get the ability to run the notebook on the fly which will then only be a config option on nbviewer.

This would be excellent, especially if combined with the ability to only display markdown and output cells.

I'm not sure I get what you mean here. Can you develop ? Do you think we can do better in nbviewer itself?

Not a case of doing better. The nbviewer is a great solution for sharing notebooks.

Once the preprocessor for executing notebooks is merged and a simple API for filtering notebooks (markdown, code, output) exists then it should be pretty easy to embed (up to date views of) notebooks as pages. Which is what this POC tries to achieve:

• Pick up a notebook JSON file
• Execute it (in memory, no need to write back to disk)
• Filter it for only the cells we want to display
• Convert to HTML
• We now have a blob of HTML to render however we like i.e. within our existing framework

The notebook function in app.py pretty much achieves this:

@app.route("/notebook/<notebook>")
def notebook(notebook):
    try:
        notebook = nbf.read(open('notebooks/%s' % notebook, 'r'), 'ipynb')
    except IOError:
        abort(418)
    html_notebook= convert_nb_html(notebook)
    return render_template('notebook.html', content=html_notebook, notebooks=g.get('nbs'))

What I mean by normal templating approach is the output HTML for each notebook is passed into the template notebook.html by the content variable.

{% extends "base.html" %}
{% block content %}
{{ content | safe }}
{% endblock %} 

The notebook.html extends the base template base.html which just contains the stuff relevant to all pages in the site, in this case just the site sidebar, but usually the site header, footer ect.

I do not get that either, do you mean that your app is serving files from the local filesystem?

I simply meant that to edit pages in the flask site is as easy as going to the notebooks directory and starting the IPython Notebook server via ipython notebook and visiting localhost:8888. Any changes saved to the notebook after developing with the IPython Notebook Web Interface would be reflected in the flask site (i.e. ipython notebook is the CMS style WYSIWYG editor).

If this approach works with Widgets this becomes very powerful. Although I don't see this being so straight forward.

[Carreau]

Oh, ok I better understand. If you use nbviewer you have a --localfile debug option that allow to serve files from disk. see python -m nbviewer --help, though this might be a security risk, so we don't advertise it.

What I mean by normal templating approach is the output HTML for each notebook is passed into the template notebook.html by the content variable.

There might be a lot of indirection but in final that whats nbviewer does, even if it does not look like it :-)

If this approach works with Widgets this becomes very powerful. Although I don't see this being so straight forward.

Widget will be complicated to render on nbviewer, but we are working on it.

Once the preprocessor for executing notebooks is merged and a simple API for filtering notebooks (markdown, code, output) exists then it should be pretty easy to embed (up to date views of) notebooks as pages. Which is what this POC tries to achieve:

You could write your own preprocessor for nbconvert, publish it on PyPi, and a simple configuration line can allow to insert it into nbconvert pipeline. You don't have to wait for IPython to integrate all imaginable preprocessors, and also we won't include all imaginable preprocessors.

[iiSeymour]

Widget will be complicated to render on nbviewer, but we are working on it.

Awesome! I've been thinking lately about how you could pass in parameter values before running a notebook. For example a notebook that does some weather analysis for a given year, you would want to pass in the year of choice before the notebook code gets executed. I just pushed an example of one way this can be achieved, it's not safe in the slightest and a little brittle but allows some parameter to be injected.

[Carreau]

You should have a look at runipy that allow similar functionality and already merged with some other projects.