What's New in Pyscript 2023.03.1

The PyScript team is absolutely steamrolling ahead in the past few months, working toward a new version of the PyScript open source library and some other developments that will become visible in the near future. (I hate to be a tease, but this isn't my piñata to pop). What follows is a writeup of the new improvements, features, and deprecations in PyScript 2023.03.1.

If the PyScript section looks a little shorter than the last release, well, the last release set a very high bar! But there are a couple other reasons why there's less user-facing changes to talk about this time. It's partly because the team wanted to do a release to pin some key features before pushing some really significant PyScript changes that are coming soon - skip down to "What's Next?" for those.

But additionally, a ton of work has been happening under the hood, especially in the past few weeks. Better linting, testing, deployment; unvendoring some necessary packages; refining and clarifying our approach to changes and issues. The kind of things that don't fill out a blog post, but make a big difference in the long run.

What's not listed here are bugfixes, and they have been several nice ones since the last release. For that kind of granular information, see the newly-added changelog document.

As always, for help, discussion, and bleeding-edge development on PyScript, come join us on The Discord Server.

PyScript

<py-script> output="..."

The output attribute of the <py-script> tag has been restored. (#1063) This allows PyScript users to route Python's output to stdout to a specific place in the dom, like so:

1

<py-script output="some-div">

2
3

    print("Hello world!")
    print("This output should go somewhere")

4

</py-script>

Users who are writing code specifically for PyScript can use the display() function to route their output (whether text or rich MIME types) to a specific place on the DOM. The output attribute is meant to allow the use of libraries which output directly to stdout, like Rich or Pygments. Or so, you know, print("Hello World") doesn't have to print in the same location as the <py-script> tag.

"runtime" is now "interpreter"

The attribute of the PyScript object which represents the internal Python interpreter has been renamed from runtime to interpreter. (#1082) This is largely an internal PyScript naming change, but it does have ramifications for some users who were making use of this key access attributes of the runtime, as in:

1

<script>

 2
 3
 4
 5
 6
 7
 8
 9
10

    //Previous naming using 'pyscript.runtime'
    function showX_2022_12_1(){
        console.log(`In Python right now, x = ${pyscript.runtime.globals.get('x')}`)
    }

    //Updated attribute name in PyScript 2023.03.1 and later
    function showX_2023_03_1(){
        console.log(`In Python right now, x = ${pyscript.interpreter.globals.get('x')}`)
    }

11

</script>

Hiding the Splashscreen

Several users have requested the ability to hide the default splashscreen that's displayed while PyScript is loading. And we heard you! The <py-config> tag now accepts a splashscreen.enabled property (defaults to True). If set to False, the default loading screen will not be shown. (#1138)

Auto-IDs for py-[event]

A small but very handy update to the py-[event] behavior: users no longer need to specify an ID when adding this attribute to an HTML element. Under the hood, an ID is stilll necessary, but if the user doesn't provide one, PyScript now adds an auto-generated UUID as the ID. (#1122)

1
2
3
4
5

<!-- The 'id' attribute was required in previous versions-->
<button py-click="someFunction()" id="old">Click me!</button>"

<!-- In version<h1 class="text-4xl text-center text-red-800">This is a draft this post hosted on a development server; not for release.</h1> 2023.03.1 and later, an ID will be auto-generated for you -->
<button py-click="someFunction()">Click me!</button>"

So Long, <py-box>, <py-title>, <py-inputbox>, and <py-button>

As previously promised, these elements (which were deprecated in version 2022.12.1) have been removed in version 2023.02.1. (#1084). If you were still making use of these custom elements, check out the 2022.12.1 release post for suggested plain HTML elements to use instead.

Plugins

What Are Plugins?

Plugins are code objects, either in Python or JavaScript, whose methods are called at specific points in the PyScript lifecycle (e.g. as PyScript installs itself, fetches the interpreter, related resources, executes <py-script> tags, etc). Internally, PyScript uses the plugin concept to orchestrate some behaviors like the Splashscreen and the <py-terminal>, but the idea is that these methods are available for users to write their own plugins to hook into.

This is a super powerful functionality! Users can (for the most part) rewrite the rules of PyScript and its execution by simply pointing part of the <py-config> at a URL with their plugin resource. You could emit events corresponding to certain actions, pre-scan and parse the Python code and act upon it before the code executes, add additional custom tags that extend PyScript's behavior... the sky's the limit.

So if they're so powerful, why isn't there more documentation on Plugins? The honest answer is that the API is rapidly changing, both in naming conventions and scope, and there's some understandable reticence at putting out a significant amount of functionality that users might rely on, only for the names and conventions to entirely change in the next release. Currently, there are two major outstanding discussions:

• Rename the phases of the page lifecycle and lifecycle methods (#1238)
• Use a metadata file for plugin specification, instead of linking directly to a code file (#1228) (#1229)

So with both the method names and keys/format likely to change, it's daunting to write documentation that may already be out-of-date by the time it's published. That said, here's a peek at what's changed in the Plugins API since the last release:

Plugins Can Now be Fetched from URLs

Where in version 2022.12.1 plugin files could only be referenced from specific .py files, a plugin can now be fetch'd from any URL. (#1065). What's more, plugins can be written either in Python or in JavaScript.

PyScript Tag Lifecycle Hooks

In addition to the hooks which happen at specific points in PyScript's loading process, we've added a couple of hooks which are called immediately before and after any <py-script> tags on the page, allowing plugins to check, for example, whether the source code adheres to certain guidelines, or whether the result was of a desired type. (#1063)

Plugin Method are Now Optional

Previously, any and all plugins had to be implemented for every plugin, or an error would be thrown. Now, plugins can implement any subset of the plugin methods (or none of them, although then what would be the point?). (#1134)

No Duplicate Plugin Calls

I know I said I wasn't going to delve into bugfixes here, but this is one that was plaguing a couple of users with specific issues. In PyScript 2022.12.1, any Python plugins were being added to the list of managed plugins twice, meaning each of their methods was called twice. This was causing some specific tricky issues where a plugin method (which should only run once) would run once, succeed, then appear to fail... tricksy indeed. That's no longer happening.(#1064)

Documentation

Changelog.md

As mentioned at the top, PyScript now has an incremental Changelog! If you're sick of wading through a couple thousand of my (questionably spelled) words every time there's a release, the Changelog has the short-and-sweet version (#1066).

Admittedly, the PyScript team is still getting used to updating the changelog as part of our workflow, so it's possible a few small things were missed. That changelog is meant to be primarily user-facing, and doesn't necessarily capture all the changes to PyScript's internals.

Since we have this additional central document I've opted to focus this post more on changes in features and utility, rather than minor-but-important changes like bugfixes. If you're interested in seeing what changed in a more specific way, and what previous bugs you can now safely ignore, I'd recommend checking out the changelog.

Event Listeners Documentation

PyScript has a very handy but under-documented way of adding event listeners directly to HTML elements using the py-[event] syntax. At least, it was under-documented until Mariana went and wrote some!

Fair warning to those making use of this feature, though - the syntax is likely to change in an upcoming version. There's active discussion around the new syntax and a PR in the works, so keep your eyes peeled for what the next iteration of that API looks like.

requests package / pyodide-http tutorial

Of all the popular Python packages that users wish they could use in the browser, probably the most asked about is requests, the ubiquitous package for making HTTP requests. Unfortunately, that package doesn't work natively within the browser, as the user doesn't have access to the same kind of low-level networking capabilities that Python running natively on a computer does.

But one person's problem is another person's call to action. Koen Vosson has created the pyodide-http package, which shims both the requests and urllib packages (if desired), allowing code previously written for "desktop flavored" python to just work in the browser. And to get users started smoothly, PyScript now includes a tutorial on how to integrate pyodide-http into your PyScript project. (#1164)

Tutorials Overhaul

The tutorials index page at docs.pyscript.net/tutorials has gotten a facelift, for a better onboarding process for new users (#1090).

The PyScript core team is always interested in having more tutorials and guides. Have you figured out how to do something with PyScript that you felt could use better documentation? We'd love to see a Pull Request!

Examples

GitHub User romankehr contributed a new example to the PyScript repository for uploading a CSV file into PyScript and loading it into a Pandas dataframe (#1067). For those looking to data-sciency things with Python in the browser, this is a great place to start.

Pyodide

Pyodide, the CPython-interpreter-in-WASM project that is the primary runtime for PyScript at the moment, has had a couple of releases in recent months; This release brings PyScript up-to-date with Pyodide 0.22.1, which brings a host of new and nifty features.

Pyodide's own release notes for version 0.22.0 provide a great overview and insight into these changes, but they're so exciting that I can't help but feature them here as well:

JS Module Typeshed

Many of PyScript's most powerful features rely on Pyodide's ability to import ... from js to get objects from the JavaScript global namespace. But it does get a little tiring to stare at a squiggy red line underneath every instance of from js import console or js.document.getElementById. The Pyodide team have added a stub (.pyi) file to make things a little better! Simply download a copy of the most recent js.pyi file and place in your IDE or project's location for stub files (VS Code, PyCharm) or simply adjacent to your .py file for simply projects. And like magic, intellisense will start filling in common attributes from the JS module! (#3298)

New Packages

A litany of new packages have been added to Pyodide, including:

pycryptodome (#2965), coverage-py (#3053), bcrypt (#3125), lightgbm (#3138), pyheif, pillow_heif, libheif, libde265 (#3161), wordcloud (#3173), gdal, fiona, geopandas (#3213), the standard library _hashlib module (#3206), pyinstrument (#3258), gensim (#3326), smart_open (#3326), pyodide-http (#3355)

Improved Python Collections APIS

The process by which JavaScript objects are transmogrified (proxied) into Python continues to get more sophisticated - JS objects that feel like they should behave like the corresponding Python collections now generally do. For instance, JavaScript arrays now implement reverse, __reversed__, count, index, append, and pop, so that they implement the MutableSequence API (#2970). This allows us to treat JavaScript arrays much more like a Python list (or other mutable sequence), eliminating the need to manually convert from one type to another. For instance, this is now possible:

1

<script>

2

    var myarray = ["PyScript", "and", "Pyodide", "and", "JavaScript", "Are", "Awesome"]

3

</script>

4

<py-script>

 5
 6
 7
 8
 9
10

    from js import myarray
    item = myarray.pop()
    print(item)
    myarray.append("Super!")
    print(" ".join(myarray))
    print(myarray.count("and"))

11

</py-script>

Awesome
PyScript and Pyodide and JavaScript Are Super!
2

Similarly, Map-like JS objects now implement MutableMapping (#3275),

1

<script>

2

    var myarray = ["PyScript", "and", "Pyodide", "and", "JavaScript", "Are", "Awesome"]

3

</script>

4

<py-script>

 5
 6
 7
 8
 9
10

    from js import myarray
    item = myarray.pop()
    print(item)
    myarray.append("Super!")
    print(" ".join(myarray))
    print(myarray.count("and"))

11

</py-script>

Awesome
PyScript and Pyodide and JavaScript Are Super!
2

Destructuring JS Objects with python match

Here's a neat one, combining the features of Python >3.10's match statement with JavaScripts (relatively simple) object structure. (Pyodide #3273)

If you have some JavaScript object that you've imported into Python, it will (unless it's a very simple object) be a JsProxy object that behaves like a Pythonic "interpretation" of the JavaScript object, with a few additional attributes and methods related to the proxy-ing behavior itself. One of these additional methods is the as_object_map() function, which, as the Pyodide docs say: returns a new JsProxy that treats the object as a map. This can be useful in several circumstances, but one in particular is using it with the match statement, as follows:

1

<script>

2
3
4
5
6
7
8

    var actor = {
        name: "Keanu",
        role: "Neo",
        action: () => {
            console.log("I know kung foo")
        }
    }

9

</script>

10

<py-script>

11
12
13
14
15
16
17
18
19
20

    import js
    pyActor = js.actor.as_object_map()
    for key, value in pyActor.items():
        print(f"{key}: {value}")

    match pyActor:
        case {"name": name, "role": role}:
            print(f"This actor is named {name} in the role {role}")
        case _:
            print("No match")

21

</py-script>

name: Keanu
role: Neo
action: () => {
            console.log("I know kung foo")
        }
This actor is named Keanu in the role Neo

JS Promises are thenable in Python

For users coming from the JavaScript world, it's perfectly natural to create a chain of thenables - that is, a sequence of objects that have a then() method, each calling its next one when its promise resolves. This makes it easy to write out a succession of functions, each one returning a promise that should be awaited, in a reasonable way.

Now, it's possible to do the same kind of then-ing directly in Python: (Pyodide #2997)

1

<py-script>

 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

    #Example borrowed from the Pyodide tests
    import asyncio

    async def fetch_demo():
        from js import fetch

        name = (
            await fetch("https://pypi.org/pypi/pytest/json")
            .then(lambda x: x.json())
            .then(lambda x: x.info.name)
        )
        print(name)
        
    asyncio.ensure_future(fetch_demo())

16

</py-script>

pytest

JS Proxy Descriptors (Using JS Functions as Python Methods)

At a recent PyScript team gathering, I was musing with Pyodide core dev Hood about the possibility of subclassing a JavaScript object in Python, so that one could write the "JavaScripty" behaviors of one's class in JavaScript and subclass it into Python to handle the "Pythony" bits. Hood kindly let me know that that way probably lies madness, but that it is now possible to use JavaScript functions as Python methods, which accomplished much of the same thing. (#3130). And if the function is defined within the Python class statement, the this object references the current Python object (like self):

1

<script>

2
3
4

    var area = (a, b) => {
        return .5 * a * b
    }

5

</script>

6

<py-script>

 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

    import js
    from pyodide.code import run_js

    class Triangle():
        def __init__(self, a, b):
            self.a = a
            self.b = b

        area = js.area
        hypo = run_js("function h() {return Math.hypot(this.a, this.b);} h")
        
    c = Triangle(a=3, b=4)
    print(f"Area is: {c.area(c.a, c.b)}")
    print(f"The hypotenuse is {c.hypo()}")

21

</py-script>

Area is: 6
The hypotenuse is 5

Mounting the Native Filesystem

By default, when Python in PyScript/Pyodide interacts with the filesystem (when writing something like with open(...) as ...), it references a "virtual", in-memory filesystem that lives in the browser window's memory for as long as the page exists. But Emscripten, the c-program-to-Web-Assembly compiler that Pyodide uses to build CPython for the web, offers additional filesystem options, one of them being Chrome's interface for mounting directories directly. (Pyodide #2987)

One thing to note: mounting a local folder into the browser - like some other potentially-invasive browser actions - can only be triggered when handling a user interaction. This is so you can't, say, open Reddit and immediately be asked to mount a folder on your computer into the browser. You can imagine the kind of chaos that would cause.

This functionality currently only works in Chrome/Chromium, though it does seem that other browsers are picking it up as well.

This is a neat-enough functionality that I want offer a live demo here. If you are using Chrome/Chromium, you can choose to mount a folder on your filesystem here, and PyScript will print the listing of its contents.

But Beware! When you click the button below, you will be asked for a folder on your computer that the PyScript/JavaScript code that runs will have access to. You can inspect the source on this page and see for yourself what I'm doing, and I do guarantee that it's the code you see on the page here, but I want you to be aware - by mounting this folder, you are implicitly trusting me, Jeff Glass, with the contents of whatever's inside that folder.

1

<py-script>

 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

    from js import showDirectoryPicker, Object
    from pyodide.ffi import to_js
    import pyodide_js
    import os

    async def requestAndPrintFolder():
        modeObject = to_js({ "mode": "readwrite" }, dict_converter=Object.fromEntries)
        dirHandle = await showDirectoryPicker()
        if await dirHandle.queryPermission(modeObject) != "granted":
            if await dirHandle.requestPermission(modeObject) != "granted":
                raise Exception("Unable to read and write directory")
        nativefs = await pyodide_js.mountNativeFS("/mount_dir", dirHandle)
        
        print(os.listdir('/mount_dir'))

16
17

</py-script>
<button py-click="requestAndPrintFolder()">Click to request folder</button>

Package Loading Improvements

Pyodide v0.22 brings a number of changes and improvements to the package loading process, most of which won't be immediately visible to casual users of PyScript, but which are useful to know. The biggest of which is that micropip, the pip-like software that handles installing packages from both PyPI and the Pyodide packages, has been moved to it's own repository so it can be maintained separately from Pyodide itself. It also allows users to install different versions or copies of micropip, as opposed to being locked to one that's bundled with Pyodide.

Additionally, the error messages that Pyodide provides when a package fails to load have been beefed up quite a bit (#3137) (#3263)

For more details, see the Package Loading section of the Pyodide changelog.

Build System Improvements

If you're interested in building packages for Pyodide, or working within the Pyodide build system, version 0.22 brings another swath of improvements. There are some new commands in the pyodide CLI which allow for finer control of the build process for specific packages, or from which sources to build. Also, the meta.yml files that specify the build process for particular packages have been expanded. For more details, see the Build System section of the Pyodide changelog.

Beyond that, Pyodide is now using the most recent Emscripten version (3.1.27, from 3.1.14), which I gather is quite nice, but honestly a little deeper in the stack than your humble author is familiar with. For details on that, check the Emscripten Changelog.

What's Next?

Web Workers

This was a topic we touched on briefly in the last release post, but a huge amount of progress has been made in this area since then... just not quite in a user-facing way.

The gist of using Web Workers, you'll recall, is to offload the actual Python execution to a separate thread so it doesn't block the main browser thread while it's executing "in the background." This means that all calls in the main thread to "run some Python" become asynchronous, which comes with its own pitfalls. The gist of the process so far is:

• Make calls to runPython into async calls. Don't do anything else different, just make sure the lifecycle is still consistent, everything that needs to be awaited is awaited, etc. (#1212)
• Split the Interpreter class (which is PyScripts abstraction around Pyodide and other future interpreters) into two classes, InterpreterClient and RemoteInterpreter. One calls the other, but other live in the main thread. (#1218)
• Move the RemoteInterpreter to a Web Worker (this is the easy part, but then...)
• Work out all the message passing/proxying to maintain communication between the main thread and remote thread. This is (maybe?) the most complicated part - Antonio, Madhur, and Hood's combined efforts have yielded a hacked together demo which does indeed run all the Python code in a web worker. It uses the synclink library to make the synchronous actions in Python block correctly while waiting for a response from the main thread, as well as handling message passing.

But my understanding here is there's still quite a bit of work to be done before this effort is ready to merge. But that by no means should take away from the tremendous effort already put in! Really cool things are coming in this area.

Events Overhaul

Another issue still circling since the previous release, but after a recent PyScript core team gathering, I think we have a way forward. The new py-[event] syntax (which to emphasize is not in this release) will be:

• For any browser event [event], the attribute py-[event]="someCallable" can be added to any HTML element. When the specified event is triggered on that HTML element, the Callable will be called.
• If the callable takes no arguments, it will be called with no arguments. If the callable takes a single argument, the event object generated by the browser event will be passed to it. If it takes two or more arguments, an Exception is raised.
• For any browser event [event], the attribute py-[event]-code="someExpression()" can be added to any HTML element. When the specified event is triggered on that HTML element, the expressed is eval()'d in the global namespace

The following examples illustrate the difference between the two scenarios: py-[event] is for registering event handlers, whereas py-[event]-code is for running snippets of code:

1

<py-script>

2
3

    def printEventTarget(event):
        print(event.target)

4
5

</py-script>
<button py-click="printEventTarget">Click me to print a reference to this button</button> 

1

<py-script>

2
3

    def addition(a, b):
        return a + b

4
5
6

</py-script>
<button py-click="print(f'{addition(2, 3)= }')">What is two plus three?</button> 
<button py-click="print('Hello, world!')">Click me to print Hell Worldo</button> 

Just to say it one more time - this syntax is coming soon and is not a part of this release. We're getting close though.

Pyxel in Pyodide

Pyodide core dev Gyeongjae Choi has a development branch in the works that I'm personally very excited about - it compiled Pyodide with Emscripten support for SDL (Simple DirectMedia Layer), a cross-platform graphics library. It powers lots of desktop-oriented graphics software, like Pygame and Pyxel. The branch is specifically working on integrating Pyxel, since it's written mostly in Python and Rust, which Pyodide has been increasingly included as a supported language to build against.

Now, Pyxel already has a build-to-web option which compiles the requisite components into Web Assembly and spits out an html file that wraps it. But with Pyodide integration, we get all the nifty browser and JS interoperability features that we've come to love in PyScript. For example, we can not only run games in the browser, but we can use LocalStorage to save user data, add HTML control or display elements outside of the game, respond to window events, or even allow game objects to be directly scripted by Python, in realtime, in the browser.. How cool is that?>

So cool I had to build a little demo, that's how cool.

As an example of what's possible with Pyxel in the browser, try putting focus on another browser window - you'll see that the game detects the window event and automatically pauses. Additionally, this demo uses the browser's local storage to keep track of your best time.

Load Demo

And more...

As I teased about 3500 words ago, there are some very cool things coming soon for PyScript; if you want to be the first to hear about them, come join us on The Discord Server