What's New in PyScript 2023.12.1

The PyScript team just cannot be stopped - three releases out so far between Halloween and Christmas. And having remade the world just a few weeks back, the runway is free and clear to add some really neat features. In today's 2023.12.1 release release, there's three big ones:

• <py-editor> (the supercharged version of the old py-repl)
• 'Managed' import of JS modules in Python
• document is automatically patched in workers

Let's hit those one by one, and close out with some Odds and Ends, plus some recent featured projects from the community.

<py-editor>

Allow users to run their own Python code on your page with a built-in, rich editor by adding a <script type="py-editor"> tag anywhere on your page. It looks like this:

The essential parts are

• A fully featured editor powered by CodeMirror
• A run button in the lower right (this is hidden except on mouseover by default, but can be made permanently visible by setting .py-editor-run-button{ opacity: 1;} in your css)
• A small environment notation in the upper right - more on this below.

Following the rest of PyScript's naming conventions, a <script type="mpy"> tag to your page instead adds an editor that runs Micropython:

You can tell which interpreter is running by the small text the way to the right of an Editor. But that text is displaying more than just the interpreter-type: it shows which 'environment' is responsible for the editor...

Shared Environments

By default, each editor is connected to its own, independent interpreter. But there are many situations where having a group of editor elements that share a single interpreter could be handy. To that end, any editors of the same type (py or mpy) with the same env attribute will share an interpreter. For instance:

1
2
3
4
5
6
7
8

<script type="py-editor" env="first">
    x = 1
    print(x)
</script>
<script type="py-editor" env="first">
    print(x)
</script>
<script type="py-editor" env="second">

9

    print(x) # Will error, as 'second' is a different interpreter than 'first'

10

</script>

Notice that the environment is displayed in small text to the upper-right of the code editor, to make it clear to users that editors are running in different environments. To hide these labels, set .py-editor-box::before { display: none } in your css.

'Environments' (as specified by the env attribute) are not shared between different types of interpreters, so <script type="py-editor" env="foo"> and <script type="mpy-editor" env="foo"> will not share objects nor scope.

Editors == Workers

Editors are a worker only feature, meaning that while <script type="py"> tags default to running in the main thread, editor tags run their code inside of Web Workers so as not to block the main (UI) thread.

As with any of PyScript's worker need, this means you will need to have appropriate headers set for using SharedArrayBuffers. Of course, if you're comfortable setting your own headers, you can set Cross-Origin-Opener-Policy: 'same origin' and Cross-Origin-Embedder-Policy: ' require-corp'.

But if you're not as comfortable/familiar with setting response headers, or if you're making use of a simple static hosting service like GitHub Pages, an S3 Static Site or readthedocs.io, you can make use of mini-coi, a single JS file you copy to your hosting which takes care of the headers for you. In fact, that's what's powering the demos on this page! Simply copy the the contents of that file to a new local file in the same directory/path as your HTML file called mini-coi.js, then add <script src="mini-coi.js"> to your HTML, and your workers/editors should just work.

Just to be explicit what about that setup might look like, here's some snippets from the site you're looking at now:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

/*! coi-serviceworker v0.1.7 - Guido Zuidhof and contributors, licensed under MIT */
/*! mini-coi - Andrea Giammarchi and contributors, licensed under MIT */
(({ document: d, navigator: { serviceWorker: s } }) => {
    if (d) {
        const { currentScript: c } = d;
        s.register(c.src, { scope: c.getAttribute('scope') || '.' }).then(r => {
        r.addEventListener('updatefound', () => location.reload());
        if (r.active && !s.controller) location.reload();
        });
    }
    else {
        addEventListener('install', () => skipWaiting());
        addEventListener('activate', e => e.waitUntil(clients.claim()));
        addEventListener('fetch', e => {
        const { request: r } = e;
        if (r.cache === 'only-if-cached' && r.mode !== 'same-origin') return;
        e.respondWith(fetch(r).then(r => {
            const { body, status, statusText } = r;
            if (!status || status > 399) return r;
            const h = new Headers(r.headers);
            h.set('Cross-Origin-Opener-Policy', 'same-origin');
            h.set('Cross-Origin-Embedder-Policy', 'require-corp');
            h.set('Cross-Origin-Resource-Policy', 'cross-origin');
            return new Response(body, { status, statusText, headers: h });
        }));
        });
    }
    })(self);

1
2
3

<script src="mini-coi.js"></script>
<script type="module" src="https://pyscript.net/releases/2023.12.1/core.js"></script>
<script type="py-editor" env="first">

4
5

    for i in range(5):
        print(i)

6

</script>

While Editors may seem familiar to PyScript users who were familiar with the former <py-repl> tag, they're another full rewrite for this release. So if there are features, APIs, or behaviors you think should be different or changed, please come let us know!

Pythonic Import of JS Modules

As we've seen more and more users taking advantage of the ability to interact with any existing JavaScript module while writing only Python, it seemed only natural to add a feature that allows for easily importing JavaScript modules. That is, it would be nice to write python like from js_modules import some_cool_JS_module and have it just work.

To that end, there are two new kfeys in <py-config>: js_modules.main and js_modules.worker. The main and worker parts specific where the JavaScript module itself is loaded, but main thread modules are accesbile from Python in both the main thread and in workers.

Each key takes a list of js_module_url: py_module_name pairs - that is, it maps URLs that JavaScript modules will be loaded from to the Python name they can be imported as. For example:

1
2
3
4
5
6
7
8

<py-config>
    # URL of esm module = "python module name"
    [js_modules.main]
    "https://cdn.jsdelivr.net/npm/leaflet@1.9.4/dist/leaflet-src.esm.js" = "leaflet"
</py-config>

<!-- JS Module is available to Python in the Main Thread-->
<script type="py">

 8
 9
10

  from js_modules import leaflet as L

  print(dir(L))

12
13
14
15

</script>

<!-- And in the worker thread -->
<script type="pyodide" worker>

16
17
18

  # JS Module available as a proxy of the module in the main thread
  from js_modules import leaflet as L
  print(dir(L))

19

</script>

Notice again that modules included via js_modules.main are still available to Python running in Workers - the JS library still lives on the main thread, and the workers can interact with it through a proxy. js_modules.worker is for JavaScript modules that are best installed in worker threads themselves, like libraries that might have a significant computational load. In that case, the imports look the same:

1
2
3
4
5

<py-config>
    [js_modules.worker]
    "https://cdn.jsdelivr.net/npm/html-escaper" = "html_escaper"
</py-config>
<script type="py" worker>

6

  from js_modules import html_escaper

7

</script>

Automatic Patching of document in Workers

This is one of those very short PRs that still has a big, meaningful impact. When running Python in a worker, the js.document object is now a proxy for the main thread's document, instead of just not existing.

Let's back up a step - as of version 2023.11.1, PyScript has the ability to run code inside of separate worker threads. This allows you to run code without ever blocking up the browser's UI. But a Worker is a different place than the main thread, in that code running in a worker doesn't have access to the DOM (page), nor to events on it.

PyScript provides a little "magic" to ease the pain of writing different code for the main thread and worker threads. If you run from pyscript import window, you always get a reference to the main window's global scope, no matter if your code runs in the main thread or a worker. Similarly, from pyscript import document is a always reference to the main thread's document.

So here's the change - as a way of improving usability of existing Pyodide packages that assume js.document always exists, PyScript always now sets js.document to the same value as pyscript.document - that is, from within PyScript, js.document will now also point to the main thread's document, even in Workers where it shouldn't normally exist. This allows existing packages written for Pyodide that expect js.document to always exist to work out of the box - matplotlib being (with its Pyodide backend) being the primary example so far.

If you need js.document to not exist for some reason - maybe your package is doing feature detection in Python to detect whether you're in a worker or now, you can always do del js.document before importing your package, or similar.

This is a relatively niche change, that I've now spent five paragraphs on. But hopefully it eases compatibility of PyScript and workers with existing Pyodide packages, which is always a nice plus.

Odds and Ends

There have been a couple more UX and quality of life interfaces made for this release. To rip through them quickly:

Multiple Configs -> Error

PyScript now throws an error if there are multiple conflicting <py-config> or <mpy-config> tags or configurations on a page. Now that we can have both inline and as-a-tag py-configs, the error messages attempt to clear up when they're in conflict. (#1885)

Examples have been moved/removed

Since the alpha release, PyScript has hosted a collection of examples in the main repo, but this caused issues both in terms of maintenance burden, consistency, and updating timeline. As of this release, the best places to see examples are the pyscript/examples repo or pyscript.com/@examples. The PyScript.Com examples are easy to run in-place/clone-and-tinker-with etc - even if PyScript.com isn't how you ultimately want to deploy your site, I think it's a great home for these examples. (#1884)

Featured Projects

Let's close out by looking

Chris Laffa has been building out a declarative UI project in PyScript called LTK with a frankly incredible demo page - go check out what's possible in PyScript with UI components, a custom pub/sub model, and more.

Piers Storey and the MeArm Controller folks are building a robot controller with MQTT and PyScript that's in development, but already looking quite neat.

And finally on a personal note, I'm still chugging along building solutions to Advent of Code in PyScript - if you're interested in coding challenges or building solutions in PyScript, those may be of interest to you!