What's New in PyScript Next (2023.11.1)

Today marks the release of PyScript 2023.11.1, a ground-up total-rewrite of PyScript that adds a wide swath of new functionality, smaller file sizes, faster loading, and so much more. This post (and really, this release) is a doozy, so get your brains in gear.

You read that right - all of PyScript has been re-written from the ground up. While as many features and behaviors as possible have been retained from 'PyScript Classic' (our internal shorthand for all releases from Alpha/2022.05.1 through 2023.05.1), there are some things in 'PyScript Next' that have been tweaked, changed, and added. There are also features that have either been removed from scope or have been temporarily removed as lower-priority in the name of getting this release out. As such, any articles and tutorials about PyScript prior to today should be taken as potentially out-of-date. For those migrating existing PyScript applications, see the Migration Section of this post.

That said, the underpinnings of PyScript are still the same. It's still built on top of the Pyodide Runtime (plus an additional runtime... keep reading!), it still allows users to run Python code directly in the browser by writing it into their HTML, etc.. What's changed is how PyScript delivers that experience, in a way that's faster, lighter, and more web friendly.

As this is a different kind of release from any of the previous ones, this Release Post also looks quite different. Rather than meticulously cateloging and demoing every single change and new feature, this post will:

• Explore the major features and attributes
• Discuss migration and major changes
• Expand on the performance and size differences
• Tease some exciting PyScript related projects

In the coming days and weeks, we'll take a closer look at PyScript Next's new features. (The official PyScript documentation also goes into deeper detail.) In the meantime, here's the roaring headlines:

CPython and Micropython

Before we look at specific tags and attributes, it's important to note: PyScript now offers you a choice of Python runtimes - you can use either CPython (via Pyodide) or Micropython to execute your code. For those unfamiliar with the latter, Micropython is a very lean Python interpreter originally written for use on microcontrollers, which has since been used in space, in the lab, and in innumerable hobbyists' hands. It's a reimplementation of (almost) all of Python with keen eyes toward minimizing memory usage and startup time. This makes it a very attractive tool on the web where milliseconds matter.

Note that while writing Micropython feels almost exactly like writing Python, they are not the same language and don't have the same underlying object model. This means that packages written for one generally don't work in the other - no numpy, matplotlib, or scikit in Micropython, for instance. For a list of the differences between the two languages, see Micropython Differences from CPython from their documentation.

That said, if what you're interested in is writing Python code to manipulate data on the web and you don't need to dip too deeply into the standard library or PYPI's resources, I'd encourage you to give Micropython a shot - I was personally stunned at how fast the startup time is, and how small the download for its core is. See the performance and size section below for more details.

See the Scripts and Tags - Micropython section to learn how to run Micropython on your page.

Running Code with PyScript

Just like in PyScript Classic, you can use PyScript on your page by adding a <script> tag that points to a specific url:

1

<script type="module" src="https://pyscript.net/releases/2023.11.1/core.js"></script>

Note that previous PyScript releases required the defer attribute on this tag - that's no longer necessary, but specifying a type of module is. If you forget to add the module type, you'll see an error like: Uncaught SyntaxError: await is only valid in async functions and the top level bodies of modules:

Note too that the name of the file has changed from pyscript.js to core.js.

One optimization: the Python runtime used (Pyodide or Micropython) is not bootstrapped until the first <py-script> or <script type="py"> tag is encountered on the page. That is, if you're injecting PyScript tags dynamically, the Pyodide and Micropoython only actually are downloaded/loaded in the browser when the first tag appears on the page.

Script Tags and Attributes

Script Tags

To run Python code in the browser, wrap it in a <py-script> or <script type="py"> tag:

1
2
3

<py-script>
    print("Hello, world!")
</py-script>

1
2
3
4

<script type="py">
    for i in range(5):
        print(i)    
</script>

This will execute code in the browser with no backend using the Pyodide runtime. The two types of tags (<script type="py"> and <py-script> are mostly* equivalent from PyScript's point of view, and I will use <script type="py"> for the rest of this post. The script tag has the advantage of not pre-parsing its context as HTML prior to being interpreted as code, which can cause parsing errors for code that looks like HTML tags inside a py-script tag. In general, using <script type="py"> is preferred.

Micropython

To run code using Micropython, use either <mpy-script> or <script type="mpy">. In general, these tags function exactly like their Pyodide counterparts. For brevity, I'll only illustrate the examples below with Pyodide tags; swap type="py" for type="mpy" in any of the code samples below to use Micropython.

1

<script type="mpy">

2
3
4
5
6
7
8

    from pyscript import display
    display("Wow that was fast!")

    # The micropython stdlib is also availble
    # https://docs.micropython.org/en/latest/library/index.html
    import hashlib
    display(str(hashlib.sha256("hello world!").digest()))

9

</script>

Tag Attributes

src

The src attribute is a URL pointing to an external Python file to be used as the source code to execute. If included, any source written inside the actual <py-script> or <script type="py"> tag is ignored.

1
2
3

# /some/url/my_module.py

print("Hello, world!")

1
2

<!-- index.html -->
<script type="py" src="/some/url/my_module.py"></script>

config

The config attribute defines the configuration to be used with a particular script tag. This can either be a URL (relative or fully qualified), or a string of JSON specifying the configuration directly. More info is given in the config section below.

1

<script type="py" config="my_config.json"></script>

async

Adding the async tag will run the Python code with top level await enabled. This allows users to use the await, async for, and async with statements at the top level of a module (i.e. not inside a coroutine/async def block). This allows users to, in essence, write coroutines which are automatically scheduled into an event loop without using asyncio.ensure_future or similar.

1

<script type="py" async>

2
3
4
5
6

    import asyncio
    
    for i in range(5):
        await asyncio.sleep(1) # Normally top-level 'await' is forbidden, but allowed with 'async' tag attribute
        print(i)

7

</script>

target

By default, calls to display() output to the same location on the page as the PyScript tag itself. If you wish the output produced by display() to appear somewhere else on the page, you can specify that location using the target attribute, which takes a CSS Selector:

1
2
3
4

<div id="foo"></div>
<p>------</p>
<div id="bar"></div>
<script type="py" target="#foo">

5
6
7

    from pyscript import display, current_target
    display("This appears up at the top, in the div with id 'foo'")
    display("This appears in the div with id 'bar'", target="bar")

8

</script>

worker

Adding the worker attribute to a script tag causes the Python code to be executed in a JS Web Worker. The code can either be included in-line, or in an external file specified by the src attribute. Note that each tag with the work attribute runs in its own isolated worker thread - in the example below, the second worker tag does not have access to the x variable defined in the first tag, as they are running in separate interpreters/threads.

Code running in a worker also accepts the async and config attributes. Note that each worker can have its own config. If no config is specified, the worker will use a copy of the config being used by the main thread script tags.

1

<script type="py">

2
3

    from pyscript import display
    display(f"hello, world! {1+2=}")

4
5
6

</script>

<script type="py" worker>

 7
 8
 9
10

    from pyscript import display
    display("first worker")
    x = 1
    display(f"{x=}")

11
12
13

</script>

<script type="py" worker>

14
15
16

    from pyscript import display
    display("second worker")
    display(f"{x=}") # Error - each 'worker' tag is a separate worker

17

</script>

The use of worker threads via PolyScript's XWorker Utility is an advanced but hugely powerful topic that deserves its own series of posts. For those looking to dive in on the deep end, I'd start with:

• The PolyScript XWorker Documentation
• The official PyScript docs page on Workers
• The PyScript documentation's worker-specific builtins

But note that your server must either provide the appropriate headers/permissions (COEP and CORP at least), or you can use a shim like mini-coi to provide them for you in a service worker.

The worker attribute gains some additional superpowers when used with the terminal attribute...

terminal

Adding the terminal attribute to a script tag causes an xtermjs terminal to be loaded on the page in the same location as that script tag.

If the script tag in question is running in on the main thread, the terminal is a "passive" output system, in the sense that it doesn't have any built-in capacity to send input back to the Python interpreter. It's worth noting that, with either main-thread interpreters or workers, only one terminal is permitted per-page:

Here's a main-thread terminal:

1
2
3
4

<script type="py" terminal>
    print("Hello, world!")
    print("You can even \x1B[1;3;31mprint in color!\x1B[0m")
</script>

Where the terminal becomes powerful is when it's combined with worker scripts. Not only can a terminal serve as a rich graphic output for your Python code, but it supports running an interactive console session as well! With your CPython code, simply add import code; code.interact() to start a fully interactive Python REPL session:

1
2
3
4

<py-config>
    packages = ['rich']
</py-config>
<script type="py" terminal worker>

 5
 6
 7
 8
 9
10

    import code
    code.interact()
    # Try copy/pasting the following into your REPL session:
    #
    # from rich import print
    # print("Hello, [bold magenta]World[/bold magenta]!", ":vampire:")

11

</script>

For an upcoming release, we're looking at adding an additional attribute to the <script type="py"> tags which means 'please drop me into an interactive session when this block finishes executing', but the semantics of that turn out to be surprisingly complicated. (And also, naming things is hard!) If you have opinions about what that attribute should be called and how it should work, please come tell us about it!

Configuration

A number of runtime options can be configured by including them inside a <py-config> tag, or as the config attribute to one of the script tags on the page. These options are largely the same as those from the previous release. Here's an example using JSON:

1

<py-config type="json">

 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

    {
        "packages": ["numpy"],
        "files": {
            "htts://example.net/file1.py": "file1.py"
        },
        "fetch": [
            {
                "files": [
                    "file2.py"
                ],
                "from": "www.example.net"
                "to_folder": "internal"
            }
        ],
        "interpreter": "0.23.2"
    }

18

</py-config>

Note that only one <py-config> tag can be used per page - if multiple are present, all but the first will be ignored. Similarly, the first main-thread (i.e. nonworker) script tag with config attribute will be used, and all others will be ignored. The <py-config> tag takes precedence over any config attributes.

The exception to the only-one-configuration rule is that Pyodide and Micropython receive separate configurations. If you're using Micropython, use the mpy-config tag to specify a separate configuration for your Micropython environment.

A config in a <py-config> tag can be written in JSON or TOML. Specify the type attribute for which language you're using. The config can also be sourced from an external file using the src attribute; in this case, PyScript will attempt to infer the language from the ending of the file name (.json or .toml).

1

# testconfig.json

2
3
4
5
6

{
    "packages": [
      "cryptography"
    ]
  }

1
2
3
4
5
6

<py-config src="testconfig.json"></py-config>
<script type="py">
    from cryptography.fernet import Fernet
    from pyscript import display
    display(Fernet.generate_key())        
</script>

The config supplied with the config attribute of a script tag can be either a URL referencing an external JSON or TOML file, or an inline config written in JSON within the attribute itself.

1
2
3
4
5

<script type="py" config='{ "packages": [ "cryptography" ] }'>
    from cryptography.fernet import Fernet
    from pyscript import display
    display(Fernet.generate_key())        
</script>

The configuration supports the following keys:

packages

A list of Python packes to install from PYPI or the the list of packages pre-built for Pyodide. Additionally, the list may include URLs of .whl files to load and install.

1
2
3
4

<py-config>
    packages=['numpy']
</py-config>
<script type="py">

5
6
7
8

    from pyscript import display
    import numpy as np
    a = np.arange(15).reshape(3, 5)
    display(a)

9

</script>

fetch

Loads external resources at various URLs into the virtual filesystem where Pyodide can interact with them. I previously detailed how [[fetch]] configurations work in my release notes from version 2022.12.1; their usage has not changed:

1

<py-config>

2
3
4
5

    [[fetch]]
    files = ['__init__.py', 'helloworld/greetings.py', 'helloworld/__init__.py']
    from = '../packages/my_package/'
    to_folder = './my_package'

6

</py-config>

files

Another, simpler way to load external files in Loads external resources at various URLs into the virtual file system. This added feature in this release has a much simpler interface than the (quite verbose) [[fetch]] syntax. In short, files is just a list of key:value pairs, where the keys are URLs and the values are locations in the virtual filesystem that Python interacts with:

1

<py-config>

2
3
4
5

    [files]
    "dummy.html" = "dummy.txt"
    "https://example.com/database_file" = "./database.csv"
    "relative/urls/work/too" = "/fully/qualified/filepaths/not/recommended.txt"

6

</py-config>

What makes this feature really powerful is the (simple) built-in templating. Specifying a key name that's surrounded by {CURLY_BRACKETS} are re-usable placeholders, allowing partial URLS or filepaths to be re-used:

1

<py-config>

2
3
4
5
6
7

    [files]
    "{BASE}" = "https://example.com"
    "{LOCAL_FOLDER}" = "data/files"

    "{BASE}/resource.txt" = "{LOCAL_FOLDER}/local_resource.txt"
    "{BASE}/data/file2.csv" = "{LOCAL_FOLDER}/data/file2.csv"

8

</py-config>

See the official documentation for more details on how templating works, and further examples.

plugins

A list of plugins, used to specify which built-in plugins should not be included using an exclamation point. Currently, there are only a couple of built-in plugins - error, which causes Python and other errors to be displayed on the page; and terminal, discussed above. To disable this, use plugins = ['!error'] in your configuration. As more plugins get added to the core, you can investigate the full listing in the core/src/plugins folder of the source code itself.

interpreter

The release version of the interpreter (Pyodide or Micropython) to use; or, a URL pointing to a version of that runtime, which allows you to link to your own custom or local builds.

Note that each release of PyScript is pinned to, and only officially supports, one release of Pyodide and one release of Micropython-in-WASM. The interpreters key doesn't imply additional compatibility - rather, it's meant for use by experimenters testing custom builds of Pyodide, or bleeding-edge builds, or those with specific optional flags enabled etc.

1

<py-config>

2

    interpreter = "0.23.2" # Use Pyodide 0.23.2

3

</py-config>

1

<py-config>

2

    interpreter = "https://example.com/built_my_own_pyodide/pyodide.js" # Use a custom build, local or online

3

</py-config>

Event Handling

The syntax for event handling has shifted in this release. Any HTML element on the page can be assigned an attribute of py-[event] (or mpy-[event]), where [event] is any of the browser events or a user-defined Custom Event. The handler will be passed the Event Object representing that event.

1
2
3

<button id="one" py-click="foo">One</button>
<button id="two" py-click="foo">Two</button>
<script type="py">

4
5

    def foo(event):
        print(f"I got called by the element with id: {event.target.id}")

6

</script>

Migration and Major Differences

So you have an existing PyScript application that you've built on top of the previous build, and your wondering how to adapt and change. While not a formal migration guide, here's a few informal pointers.

First, if you're not making use of any of the new major features (Micropython or Workers specifically), you're unlikely to notice major breaking changes in how your Python code runs. The core tags (<py-script> and <script type="py">) have been rebuilt to almost entirely mimic their previous behavior.

<py-terminal> tag is retired

The <py-terminal> tag as a concept has been replaced by the terminal attribute of a <script type="py"> tag. See that section for details and (using workers) interactivity.

<py-repl> is on Haitus

This version of PyScript is releasing without the <py-repl> tag, which is busy getting a makeover in the background. The name py-repl, while catchy, didn't really capture what the component was, which was much closer to the cell of a Jupyter notebook. With the py-terminal getting overhauled in a way that would let it run an actual Python REPL, it was time to retire the former <py-repl> component. Currently, the plan is for that element to rise again as the <py-cell> (or maybe <script type="py-cell">) tag (perhaps contained inside a <py-notebook> tag as well), but that will have to wait for a future release.

Fabio Pliger and I recently authored a proposal over on GitHub with a sketch of what this feature might look like - if you're interested in this feature, please check it out and leave feedback!

Events are Different

As discussed in the events section of this post, inline event handlers work differently than how they used to. Previously, inline event attributes were strings of Python code, which were exec()'d when the handler was invoked. Now, they're the name of a function in the Python global namespace to be called, and passed an Event object.

The one usage pattern that this doesn't accommodate (yet) is something like the following, since there's no way to directly pass arguments.

1
2
3
4

<button py-click="select_color('red')">Red</button>
<button py-click="select_color('green')">Green</button>
<button py-click="select_color('blue')">Blue</button>
<script type="py">

5
6

    def select_color(color_name):
        ... # ???

7

</script>

That said, there are some web-ish ways to continue to pass element specific information to your functions, like embedding that information in a data-attribute:

1
2
3
4

<button data-color='red' py-click="select_color">Red</button>
<button data-color='green' py-click="select_color">Green</button>
<button data-color='blue' py-click="select_color">Blue</button>
<script type="py">

5
6

    def select_color(event):
        print(f"Setting color to: {event.target.getAttribute('data-color')}")

7

</script>

Plugins

The plugins system has been overhauled from the ground up, with an entirely different set of named hooks and actions. If you've written any plugins for PyScript Classic - and to be honest, because of the interim and relatively undocumented nature of that API, I'd be shocked if there were that many in the wild - you'll need to re-write re-adapt them going forward.

That said, if you have written a plugin that you found usfeul and would like to keep using, the PyScript team would love to hear from you and what you'd find useful in the API as it evolves!

For now, the core of the PyScript Next plugin system is the Polyscript Hooks system - see that documentation for more info. This is another area where I hope to share more examples and info in the coming weeks.

Import Recommendations

There's no getting around the fact that the browser's main thread and worker threads are fundamentally different. Even with coincident under the hood making passing proxies back and forth relatively painless, and Polyscript's Pythonic XWorker wrapper around it, there are things which are only permitted in one environment or the other. This could lead to pain points, where every PyScript code sample from now on could need to specify whether it's meant to run in the main thread, a worker, or either.

import js is one such sticking point. This statement uses some Pyodide (and now Micropython) magic to import JavaScript objects from the current global scope and proxy them as Python objects. But the main thread local scope gets access to lots of useful goodies - specifically the DOM itself and DOM events - that aren't available in worker threads. So something simple like from js import document works in the main thread, but will break in a worker thread which has no access to the document.

To help smooth out this particular difficulty, a couple more (slightly) magically imports have been added - from pyscript import window and from pyscript import document, both of which refer to the main thread's global scope and document, respectively, regardless of whether your code is running in the main thread or a worker. For this reason, in PyScript specifically, I recommend using from pyscript import window instead of import js most of the time - whenever you're working with DOM manipulation or events for sure. Of course, if you need access to the worker thread's global JavaScript scope, you can use import js as usual.

To illustrate - the first script below works identically with or without the worker attribute; the second script only works on the main thread:

1

<script type="py">

2
3
4
5

    from pyscript import document
    p  = document.createElement("p")
    p.innerText = "Hello, world"
    document.body.appendChild(p)

6

</script>

1

<script type="py">

2
3
4
5

    from js import document
    p  = document.createElement("p")
    p.innerText = "Hello, world"
    document.body.appendChild(p)

6

</script>

No output or stderr attributes

The output and stderr attributes of <py-script> (or <script type="py"> tags are not implemented in this release. We had included them in previous releases to accommodate use cases of using desktop-style code (i.e. that relies on print for output) to still output to somewhere on the page. I (personally) expect we'll re-implement them sometime soon, but there's some additional underlying architecture to be investigated first.

No More Magic Imports

In an effort to keep things cleaner, more predictable, and better-behaved with IDEs and linters, PyScript no longer imports any names into the Python namespace for you. This differs from the previous release, where the names js, pyscript, Element, display, and HTML were treated a bit like builtins, and imported prior to any user-written code.

Those objects still exist, you'll just have to import them for yourself.

Performance and Size

Startup Time

Here's a short PyScript page which measures the time between (roughly) the start of paging loading and when Python code starts being executed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

<!DOCTYPE html>
<html lang="en">
<head>
    <script>
        var startTime=Date.now()
    </script>
    <!-- Uncomment only one of the following two lines to compare performance -->
    <!-- <script type="module" src="https://pyscript.net/releases/2023.11.1/core.js"></script> -->
    <script defer src="https://pyscript.net/releases/2023.05.1/pyscript.js"></script>
</head>
<body>

<script type="py">

14
15

    import js
    js.console.log("Elapsed Time:", js.Date.now() - js.startTime)

16
17
18

</script>
</body>
</html>

Of course, the specifics of the timing will vary widely from machine-to-machine and connection-to-connection. But for this informal test (on my particular laptop, on my particular Wifi, at this particular spot on my couch, on a Sunday afternoon), I found that the previous 2023.05.1 release averaged about 4.9 seconds, where the new 2023.11.1 release averaged around 3.4 seconds. That's roughly a 30% decrease in initialization time just for upgrading to the new version. Not bad!

Transfer Size

There isn't an enormous difference in the total file size transferred to the browser between this release and the previous one: 5.4 MB previously vs. 5.2MB now. This makes some sense though - PyScript is still transferring the entire CPython interpreter (compiled to Web Assembly) to the browser; compared to that, the entire PyScript codebase is tiny in either case. But where both of the above metrics get blown out of the water is with...

Micropython

Take the above example, and replace <script type="py"> with <script type="mpy"> to switch to using Micropython. Now the transfer size drops to just over 250Kb and the average startup time plummets to ~200ms. That's 200 milliseconds from page-load to start-of-script-execution. In relative terms (again, in my informal experiment), that's a transfer that's ~4% of the one needed for CPython/Pyodide, with roughly 6% of the loading time. Like I mentioned earlier - if you don't need additional CPython packages or deep needs from the standard library, it's worth giving Micropython a try.

Two More Announcements - A Teaser

On a personal note - in the next two weeks, I'm excited that I'll be able to share a couple of PyScript-based projects I've been working on for some time! The first is aimed at helping PyScript users get acclimated to its features and uses as quickly as possible. The second integrates PyScript on top of another existing platform, to bring the power of Python on the Web to even more users.

But it wouldn't be much of teaser if I just told you what they were, would it? In the meantime, I hope you get your hands on the new PyScript release! Send us those new issues, join us on Discord, and show us the things you make!