Whats New in Pyscript 2022.09.1
Published September 30, 2022
PyScript Version 2022.09.1 was just released, and just as tech lead Fabio Pliger said in proposing the versioning scheme:
"...An important aspect to keep in mind is that PyScript is still in its very early stages. So, we should highlight that the expectations should be that think can often break until we reach a level of maturity and stability."
And wow, are there a lot of new things in this version of PyScript. What's more, the default Pyodide runtime has been upgraded to the recently-released version 21.2, which itself provides many new features and improved functionality to PyScript.
I want to specificically highlight new features, breaking changes, and neat behind-the-scenes work. The full details of what's changed are captured in the PyScript Release Changelog
<py-env> Will Be Going Away
<py-env> tag was where one would specify additional libraries to download from PyPI, as well as URL's to load into the local filesystem. Now, those options are being folded into
<py-config>, alongside other options like plugins and runtimes and metadata like the pages name and version number. The use of
<py-env> is deprecated and will be removed in a future release.
<py-config> can now accept configurations in JSON in addition to TOML. Creators using build systems that strip out whitespace (which isn't very kind to TOML) may find this especially useful.
The alpha and 2022.06.1 releases supported a couple of special attributes on HTML tags -
pys-onKeyDown - that PyScript hooked into to allow the running of Python code in response to a couple of common browser interactions.
Release 2022.09.1 radically expands this capability with many, many more browser events supported.
event syntax. Previously, you supplied a
Callable which was called with no arguments. Now you write a line of code (optionally broken up with
; symbols) which is run when the event triggers. The correct usage is now:
py-* attribute can be any valid Python code, not just a single function call.
Better Input/Output Escaping
Embedding something that looks like HTML inside of Python inside of HTML is... well, even just saying it is a mouthful, and it comes with its own pitfalls. Previously, PyScript tags like the following would fail in a couple of ways:
First, the Browser needs to be prevented from interpretting the
<b> tag as internal HTML, and second, the output needs to recognize that the
< > symbols are not an HTML tag. These issues have been solved by a pair of changes.
Logging to the Developer Console that PyScript does is now much cleaner, and annotated by what file the log line is generated in. This makes it easier to see what's logged by the user's program and what's being logging by the PyScript mechanisms themselves.
Framework for Multiple Runtimes
The use of a specific version of Pyodide is no longer hardcoded into a PyScript release - users may now opt to supply a URL and name for a 'runtime' in the
<py-config> tag. If one is not supplied, the default is still to load the version of Pyodide that PyScript has been most recently tested against, which should be the right option for most users. But this does open the door to future improvements like:
py-scriptblocks in different versions of Pyodide
py-scriptblocks in runtimes that are not Pyodide (Micropython??)
py-scriptblocks in a self-built/custom build of Pyodide for experimentation or demonstrating new features
A new Try PyScript section now leads the main ReadMe on the PyScript GitHub, to more quickly get new users up to speed on how to try out PyScript in their browser.
The CONTRIBUTING guide has been fleshed out with more guidance on developing submitting useful issues, forking the repository for local building and setting up the a development environment, and more. Both Mariana Meireles and Fabio Rosado have contributed excellent information on how to build PyScript and how to create and submit a Pull Request - every open source project should be so lucky!
Two new How-To guides were added to the documentation. The first covers how to make HTTP requests in pure Python by using pyodide's
The Getting Started guide got a huge update to reflect the new
<py-config> changes (see above).
It's no secret that the beating heart of the PyScript project is the Pyodide project, which makes it possible to run Python in the browser by compiling the CPython runtime to Web Assembly. (This is now nicely highlighted at the top of the PyScript readme.) Which means that improvements to Pyodide are big boons for PyScript!
While PyScript's Alpha and 2022.06.1 were designed around Pyodide 20, PyScript 2022.09.1 fully embraces Pyodide 21.2 and the many changes and improvements it brings. We'll only hit the highlights here; for more details, see the Pyodide 21 Release Post and Change Log
This is probably the most visible change for the casual PyScriptian - the functionality of the Pyodide Python API has been divied up into a number of individual packages for clarity and namespace control. So now, rather than using
from pyodide import create_proxy, one would use
from pyodide.ffi import create_proxy, and so on.
The old locations of the functions are still present but deprecated in version 21, so this change alone won't break code written for Pyodide 20. But you will see a deprecation warning, and any new code should obey the new namespacing as the deprecation
Python Wrappers for
setTimeout, and more
As part of the reorganization mentioned above, the Pyodide API added a bunch of Python utility functions that handle common JS actions. Specifically, within the a[href="https://pyodide.org/en/stable/usage/api/python-api/ffi.html">
pyodide.ffi.wrappers namespace, we now have functions for
clear_interval. This avoids the need import those JS functions directly from
js.document, and since the Python functions automatically wrap passed functions with
create_proxy, that can be left out as well.
Yet another API addition is
A whole load of new packages have now been bundled with Pyodide, including
python-magic, and many more. See the full list to see if your favorite package is now included.
Improved Build Process for Binary Wheels
For those looking to integrate their own Python wheels into apps built with Pyodide, the process for building binary wheels for Pyodide has been significantly improved. See the Pyodide team's blog post on Binary Wheels for more information.
There's been some additional work and corrections to this process, and I'd imagine we'll continue to see it evolve and refine.
Correct Handling of Objects with Null Constructor
Just as PyScript uses Pyodide as its primary runtime to run Python in the browser, so Pyodide relies on Emscripten to compile CPython for the browser. Pyodide 21 now moves from using Emscripten version 2.0.27 to version 3.1.14
To be honest, I'm not well enough versed in EmScripten to be able to parse the changelog details enough to highlight them. If you're more familiar with that program and its capabilities, let me know!
The last two categories of changes really shouldn't impact end-users of PyScript much, but they're already making a huge difference to the PyScript devs and maintainers. Prior to this release, there wasn't much of a testing regimin. Now there's multiple different means of testing the Python and TypeScript code that make up PyScript, as well as integration tests that test them both, making it easier and faster to tell when something's going to break. The testing methods are:
- Integration Tests with Playwright - loads HTML pages in the browser and checks that PyScript works as intended.
- TypeScript Testing with Jest - Testing whether Py-Script elements load correctly via TS/jS
- Python Tests with PyTest - Test functionality in PyScript.py, Python things like Element, PyScript, ContextManager, etc.
Finally, there's the bounty of little improvements that make the codebase stronger and the dev process more repeatable.
The CI/CD pipeline continues to get refined and grow more resiliant - there have been some improvements to the CD process to ensure PyScript is rebuilt with every commit and pushed to Unstable.
PyScript is being developed in TypeScript, which has the nice property of allowing quick prototyping with loose typing and gradually refining the typing to make the Linter/compiler happier. Several users, especially contributor Woxtu have been hard at work makign sure types line up, Promises are resolved, and type signatures are accurate.
Though end-users don't see the results directly, having thorough and consistant type signatures makes it easy to spot smelly code when adding new features. Does this function really need to return two different types of thing, or should we be rethinking the code structure? Why is this
So what's coming down the pipe next for PyScript? Frankly, a ton, and that work is largely visible in the open PR's and issues on the PyScript GitHub.
New Output and Rendering Design
One of the largest overhauls coming to a near-future version of PyScript is a total rethink of how PyScript renders to the browser window.
print() is the right output method for a terminal, but it doesn't quite make sense in the context of a browser window, where the world of UI is much much larger.
To that end, there's a large project in the works that, among other things:
- Introduces a new display() function, which is the preferred way of outputting to the browser window
- Routes stdout to the developer console by default
- Improves escaping of HTML-like text included inside PyScript source
The exact syntax and methodology of
display() is still being hashed out, but work is proceding a breakneck pace, and it'll be exciting to see where it ends up.
PyScript Lifecycle Changes
Antonio Cuni et. al. have laid the groundwork for a sweeping rethink of how PyScript manages the lifecylce of initializing, loading Pyodide, processing tags into custom elements, and more. It includes provisions for user-created plugins (to extend functionality) and widgets (essentially custom tags) on the page, in more-or-less a plug-and-play fashion.
This isn't the kind of dish that can be cooked up in 20 minutes - it'll touch almost every part of the PyScript codebase. But it's exciting to see the code moving in a direction that's more flexible and understandable, which will only make it more expansible.
This one is near and dear to my heart. Exactly how asynchronous code should function in PyScript has been a hot topic for some of the maintainers, considering we broke it and had to fix it again.
Pyodide has a curious feature that allows you to run code with Top Level Await, since there's always an event loop running (the browser event loop). This is both handy and confusing, since it doesn't correspond to the experience of running Python in a terminal at all. So what's to be done? Perhaps we need to further specify the execution order of async tags? Or preclude top-level-await entirely? Do we even entirely understand how Pyodide is interacting with the browser event loop? What if an async task never terminates? Lots to be done here.
There's plenty more swirling around in the PyScript ecosystem - web workers, arrow functions, further documentation, a file API...
Of course, not all ideas become plans, and not all plans become reality, but there's no lack of great ideas to keep pushing PyScript forward.