(Thanks to Glyph Lefkowitz for some of the inspiration for this port, and to Mahmoud Hashemi for helpful comments and suggestions. All mistakes and issues that remain are mine alone.)

The Python REPL has always been touted as one of Python's greatest strengths. With Jupyter, Jupyter Lab in its latest incarnation, the REPL has been lifted into the 21st century. It has become the IDE of the future: interactive, great history and logging -- and best of all, you can use it right from your browser, regardless of your platform!

However, we still have 20th century practices for developing web applications. Indeed, the only development is that instead of "CTRL-c, up-arrow, return", we now have "development servers" which are not "production ready" support auto-reloading -- the equivalent of a robot doing "CTRL-c, up-arrow, return".

Using the REPL to develop web applications is pure bliss. Just like using it to develop more linear code: we write a function, test it ad-hocly, see the results, and tweak.

When we are sufficiently pleased, we can then edit the resulting notebook file into a Python module -- which we import from the next version of the notebook, in order to continue the incremental development. Is such a thing possible?

Let's start by initializing Twisted, since this has to happen early on.

from tornado.platform.twisted import install
reactor = install()

Whew! Can't forget that! Now with this out of the way, let's do the most boring part of every Python program: the imports.

from twisted.web import server
from twisted.internet import endpoints, defer
import klein
import treq

Now, let's start Klein app. There are several steps involved here.

root = klein.app.resource()

We take the Klein resource object...

site = server.Site(root)

...make a wrapping site...

ep = endpoints.serverFromString(reactor, "tcp:8000")

...create an endpoint...

ep.listen(site)
<Deferred at 0x7f54c5702080 current result: <<class 'twisted.internet.tcp.Port'> of <class 'twisted.web.server.Site'> on 8000>>

and like "Harry met Sally", eventually bring the two together for Klein to respond on port 8000. We have not written any application code, so Klein is currently "empty".

What does that mean?

async def test_slash():
    response = await treq.get('http://localhost:8000')
    content = await response.content()
    return content

This function uses Python 3's async/await features, to use treq (Twisted Requests) and return the result. We can use it as our ad-hoc debugger (but we could also use a web browser -- this is naturally hard to show in a post, though).

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c5532630>
b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">n<title>404 Not Found</title>n<h1>Not Found</h1>n<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>n'

Ah, yeah. Even / gives a 404 error -- we have literally defined nothing. OK, this is easy to fix:

@klein.route("/")
def something_useful(request):
    return 'Hello world'

Wait, did this work?

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c53d8d30>
b'Hello world'

Yep. But it's not a proper sentence...woops.

@klein.route("/")
def something_useful(request):
    return 'Hello, world!'

Nice. Punctuation. Force. Determination. Other nouns.

Did it change anything?

defer.ensureDeferred(test_slash()).addBoth(print)
<Deferred at 0x7f54c4b9e240>
b'Hello, world!'

Yes.

Incremental web application development. Without an "auto-loading" "not production grade" server.

We took advantage of several incidental issues. The Jupyter kernel is Tornado based. Twisted has both a production-grade web development framework, and the ability to run on top of the tornado loop. The combination is powerful.


Running Modules

Mon 19 March 2018 by Moshe Zadka

(Thanks to Paul Ganssle for his suggestions and improvements. All mistakes that remain are mine.)

When exposing a Python program as a command-line application, there are several ways to get the Python code to run. The oldest way, and the one people usually learn in tutorials, is to run python …

read more

Random Bites of Pi(e)

Wed 14 March 2018 by Moshe Zadka

In today's edition of Pi day post, we will imagine we have a pie. (If you lack imagination, go out and get a pie.) (Even if you do not lack imagination, go out and get a pie.)

As is traditional, we got a round pie. Since pies are important, we …

read more

The Python Toolbox

Thu 01 February 2018 by Moshe Zadka

I have written before about Python tooling. However, as all software, things have changed -- and I wanted to write a new post, with my current understanding of best practices.

Testing

As of now, pytest has achieved official victory. Unless there are overwhelming reasons to use something else, strongly consider using …

read more

Jupyter for SRE

Sat 30 December 2017 by Moshe Zadka

Jupyter is a tool that came out of the data science community. In science, being able to replicate experiments is of the utmost importance -- so a tool where you can "show your work" is helpful. However, being able to show your work -- have colleagues validate what you have done, repeat …

read more

Write Python like an expert

Sun 17 December 2017 by Moshe Zadka

Ten tricks to level up your Python.

Trick 0 -- KISS

Experts know about the weird dark corners of Python -- but do not use them in production code. The first tip is remembering that while Python has some interesting corners, they are best avoided in production code.

Make your code as …

read more

Interesting text encodings (and the people who love them)

Wed 13 December 2017 by Moshe Zadka

(Thanks to Tom Prince and Nelson Elhage for suggestions for improvement.)

Nowadays, almost all text will be encoded in UTF-8 -- for good reasons, it is a well thought out encoding. Some of it will be in Latin 1, AKA ISO-8859-1, which is popular in the western world. Less of it …

read more

Exploration Driven Development

Sun 10 December 2017 by Moshe Zadka

"It's ok to mess up your own room."

Sometime there is a problem where the design is obvious -- at least to you. Maybe it's simple. Maybe you've solved one like that many times. In those cases, just go ahead -- use Test-Driven-Development, lint your code as you're writing, and push a …

read more

Abstraction Cascade

Tue 14 November 2017 by Moshe Zadka

(This is an adaptation of part of the talk Kurt Rose and I gave at PyBay 2017)

An abstraction cascade is a common anti-pattern in legacy system. It is useful to understand how to recognize it, how it tends to come about, how to fix it -- and most importantly, what …

read more

Gather

Mon 13 November 2017 by Moshe Zadka

Gather is a plugin framework -- and it now has its own blog.

Use it! If you like it, tell us about it, and if there is a problem, tell us about that.

read more