Uploading and Manipulating Images in Pyscript

A curious dev on the PyScript Discord (which you should really come check out) asked:

I am taking file input in HTML where I am selecting image, how to show image when submit button is hit in PyScript?

Actually, I need to use that file in PyScript to process. How can I do that?

Well, there's an interesting question. How do we deal with uploaded files in Javascript/Pyscript?

For those looking to skip to the punchline - here's a working demo. We'll show off both the ability to upload and display images, as well as manipulating them with the Pillow image manipulation library:

If all has gone to plan, images uploaded in the first dialog should just appear onscreen full-size; images uploaded in the second dialog should appear below the upload dialog, having been (1) "embossed", (2) rotated 45 degrees, (3) had any empty space filled with a dark green background, and (4) been rescaled to 300x300 pixels

Simple Image Upload and Display

The HTML portion of this project is very straightforward - an input with type=file and an ID to refer to it by, as well as an empty div for us to shove output in later:

my-page.html

1
2
3
4

<label for="Upload a PNG image"></label><input type="file" id="file-upload">
<div id="output_upload"></div>
<py-script src="./image_upload.py"></py-script>
    

The Pyscript portion of this example is only slightly more involved. We use addEventListener() to trigger a function when the selected file in the input field changes. Then we get the file targetted by that input, and create a temporary URL for it using window.URL.createObjectURL(). Finally, we create a new <img> tag and stick it inside our output div.

If desired, this functionality could be trigged by submitting a form, clicking a separate "Process Image" button, or any other event. This demo just slaps the image up as soon as its chosen, for brevity of example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

from js import document, console, window
from pyodide import create_proxy
import asyncio

def _upload_file_and_show(e):
    console.log("Attempted file upload: " + e.target.value)
    file_list = e.target.files
    first_item = file_list.item(0)

    new_image = document.createElement('img')
    new_image.src = window.URL.createObjectURL(first_item)
    document.getElementById("output_upload").appendChild(new_image)

upload_file = create_proxy(_upload_file_and_show)

document.getElementById("file-upload").addEventListener("change", upload_file)

Image Processing with PILLOW

Things get slightly more involved if we want to use the Python Image Library (or its kinder wrapper, PILLOW) to work with the images. The HTML looks almost identical, but we do need to add Pillow to a new <py-env> tag so that micropip will install Pillow into our environment for us.

my-page.html

1
2
3
4
5
6

<py-env>
    - Pillow
</py-env>
<label for="Upload a PNG image"></label><input type="file" id="file-upload-pillow">
<div id="output_upload_pillow"></div>
<py-script src="./image_upload_pillow.py"></py-script>

However, the Pyscript in this case is somewhat more involved, getting the bytes back and forth from Pyscript and the browser in formats they like. Full caveat - through testing, I think all of these castings and conversions are necessary for this to work, but if anyone finds a shorter way, please let me know!

That said, to load the image data into Pillow, we:

• Get the raw bytes of data from the image using await item.arrayBuffer()
• Cast that data into a bytearray and then as an io.BytesIO object, which is an in-memory object that behaves as a file-like object for IO purposes.
• Load that BytesIO object into Pillow using Image.open().

Once we have the image loaded, we can do all of our usual Pillow-based adjustments to it - in this case, I'm having it filter, rotate, fill, and resize the image using a succession of operations.

Finally, to retrieve the data in a format that we can use in the DOM, we:

• Create another BytesIO file-link object, and use Image.save() to write our image out to it.
• Create a new File object containing the bytes of our image, with a placeholder name and a MIME type of image/png
• Create an URL we can use for this File using indow.URL.createObjectURL()
• Use that URL as the src of a new img tag (made with document.createElement()) and append that as a child of our div.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

from js import document, console, Uint8Array, window, File
from pyodide import create_proxy
import asyncio
import io

from PIL import Image, ImageFilter

async def _upload_change_and_show(e):
    #Get the first file from upload
    file_list = e.target.files
    first_item = file_list.item(0)

    #Get the data from the files arrayBuffer as an array of unsigned bytes
    array_buf = Uint8Array.new(await first_item.arrayBuffer())

    #BytesIO wants a bytes-like object, so convert to bytearray first
    bytes_list = bytearray(array_buf)
    my_bytes = io.BytesIO(bytes_list) 

    #Create PIL image from np array
    my_image = Image.open(my_bytes)

    #Log some of the image data for testing
    console.log(f"{my_image.format= } {my_image.width= } {my_image.height= }")

    # Now that we have the image loaded with PIL, we can use all the tools it makes available. 
    # "Emboss" the image, rotate 45 degrees, fill with dark green
    my_image = my_image.filter(ImageFilter.EMBOSS).rotate(45, expand=True, fillcolor=(0,100,50)).resize((300,300))

    #Convert Pillow object array back into File type that createObjectURL will take
    my_stream = io.BytesIO()
    my_image.save(my_stream, format="PNG")

    #Create a JS File object with our data and the proper mime type
    image_file = File.new([Uint8Array.new(my_stream.getvalue())], "new_image_file.png", {type: "image/png"})

    #Create new tag and insert into page
    new_image = document.createElement('img')
    new_image.src = window.URL.createObjectURL(image_file)
    document.getElementById("output_upload_pillow").appendChild(new_image)

# Run image processing code above whenever file is uploaded    
upload_file = create_proxy(_upload_change_and_show)
document.getElementById("file-upload-pillow").addEventListener("change", upload_file)

Scroll to see complete code