Hey guys im getting an error using PyMuPDF python lib
Each time that i run this on windmill i get:
ImportError: cannot import name 'Sequence' from 'collections' (/usr/local/lib/python3.11/collections/init.py)
pymupdf==1.25.1
But when i run this locally with uv i dont have any issue
52 Replies
@Pyra that looks like importlib not knowing about the python path or similar
Looking into that
Also found another edge case error on "Custom HTTP routes":
When i removed the user that deployed a custom route and then try to call that route i receive:
"Internal: Could not insert into queue 0194469a-4767-6d58-9f5b-76bec3cf6b08 with tag python3, schedul"
@rubenf FYI
@Osmar Could you share your imports? I can't reproduce by just Importing pymupdf
Maybe something with the version @Pyra ? v1.435.2-12-g45d4fc2de should i try to update?
import pymupdf # PyMuPDF
def main(
start_time: datetime = None, end_time: datetime = None, as_base64: bool = False
):
pymupdf.open("test.pdf")
if __name__ == "__main__":
print(main())
Thank you, let me check
Im trying to run this with the vscode extension FYI
This is without the lib
And this is with the lib
hmh
This is with the lib but locally
Should i try to update windmill and see if thats enough?
Im using static requirements.txt if you want i can provide it:
Can you import pymupdf but dont open test.pdf?
Can you try to run the same on cloud/from webapp?
Mmmm it ran 👌
Strange
Ok, let me debug it
Let me try with my full script
Same issue with the full script
Could it be the lock files?
It is probably different issue
I will try to reproduce with provided packages
Are these inserted by hand, or output of resolver, e.g.
uv pip compile requirements.in -o requirements.txt ?
nvm, I can reproduce on cloud, I can work off thatI installed locally and then ran uv pip freeze > ./f/requirements.txt
What is the diffeence?
We include additional flags and cache the result while resolution, but it is ok to do it that way. I just wanted to know what top-level libraries are used.
Anyways I found failing library it is pathlib==1.0.1. This library is supported by python 3.4 and older, after that it became builtin and not maintained anymore.
For libraries that still referencing non-builtin pathlib python has some sort of fallback or override to that builtin pathlib in /usr/lib64/python3.11/pathlib.py
But for some reason in windmill that path is not found or does not contain pathlib.py. That explains why it runs on local machine but fails on cloud. I'm investigating this rn.
For example. This will work:
And this won't:
In first case we import pathlib as builtin, when in second one we import it explicitly from pypi.org
(I'm offline for next 2 hours, but the fix likely to be done today)
@Osmar Can you try this workaround?
Just add this:
Before importing that:
imprt pathlib
(Or before importing any package in case you dont import pathlib directly)Will try
Like that?
Same issue if it was that
What about webapp/cloud?
Testing
hmh, that is strange
`
--- PYTHON CODE EXECUTION ---
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/tmp/windmill/wk-default-c817fca2ec11-AGlx9/019447a2-dc3f-0623-afde-16753ade6c71/wrapper.py", line 4, in <module>
import loader
File "/tmp/windmill/wk-default-c817fca2ec11-AGlx9/019447a2-dc3f-0623-afde-16753ade6c71/loader.py", line 3, in <module>
from importlib.abc import MetaPathFinder, Loader
File "/usr/local/lib/python3.11/importlib/abc.py", line 19, in <module>
from .resources.abc import ResourceReader, Traversable, TraversableResources
File "/usr/local/lib/python3.11/importlib/resources/init.py", line 3, in <module>
from ._common import (
File "/usr/local/lib/python3.11/importlib/resources/_common.py", line 2, in <module>
import pathlib
File "/tmp/windmill/wk-default-c817fca2ec11-AGlx9/019447a2-dc3f-0623-afde-16753ade6c71/site-packages/pathlib.py", line 10, in <module>
from collections import Sequence
ImportError: cannot import name 'Sequence' from 'collections' (/usr/local/lib/python3.11/collections/init.py)``
Still when i run this local it runs as expected
might be actually related to lockfile generation..
Want me to regenerate?
may be just backup requirements.txt and let windmill generate lockfile
@Pyra Seems like its working now i reseted my uv to old requirements, added again the new packages, regenerated requirements.txt and then regenerated the lock files
Noise, can you share the new lockfile?
I would like to run it through diff
aiohappyeyeballs==2.4.3
aiohttp==3.11.8
aioprocessing==2.0.1
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.6.2.post1
async-class==0.5.0
attrs==24.2.0
axiom-py==0.8.1
brotli==1.1.0
certifi==2024.8.30
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.0
click==8.1.7
cryptography==44.0.0
cssselect==1.2.0
dacite==1.8.1
deprecation==2.1.0
faust-cchardet==2.1.19
frozenlist==1.5.0
gevent==24.11.1
geventhttpclient==2.3.3
gotrue==2.11.0
greenlet==3.1.1
h11==0.14.0
h2==4.1.0
hpack==4.0.0
hrequests==0.8.2
httpcore==1.0.7
httpx==0.27.2
hyperframe==6.0.1
idna==3.10
iso8601==2.1.0
jellyfish==1.1.0
jmespath==1.0.1
jwcrypto==1.5.6
lxml==5.3.0
markdown-it-py==3.0.0
mdurl==0.1.2
multidict==6.1.0
ndjson==0.3.1
numpy==2.1.3
orjson==3.10.12
packaging==24.2
parse==1.20.2
parsel==1.9.1
pillow==11.1.0
postgrest==0.18.0
propcache==0.2.0
psycopg==3.2.3
psycopg-binary==3.2.3
pyactiveresource==2.2.2
pycparser==2.22
pydantic==2.10.2
pydantic-core==2.27.1
pygments==2.18.0
pyhumps==3.8.0
pyjwt==2.10.1
pymupdf==1.25.1
pypdf2==3.0.1
python-dateutil==2.9.0.post0
pyyaml==6.0.2
realtime==2.0.6
reportlab==4.2.5
requests==2.32.3
requests-toolbelt==1.0.0
rich==13.9.4
selectolax==0.3.26
setuptools==75.6.0
shopifyapi==12.7.0
six==1.16.0
sniffio==1.3.1
sqlalchemy==2.0.36
sqlmodel==0.0.22
storage3==0.9.0
structlog==24.4.0
supabase==2.10.0
supafunc==0.7.0
tenacity==9.0.0
typing-extensions==4.12.2
ujson==5.10.0
urllib3==2.2.3
w3lib==2.2.1
websockets==13.1
wmill==1.431.1
xmltodict==0.14.2
yarl==1.18.0
zope-event==5.0
zope-interface==7.2
And my requirements file:
aiohappyeyeballs==2.4.3
aiohttp==3.11.8
aioprocessing==2.0.1
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.6.2.post1
async-class==0.5.0
attrs==24.2.0
axiom-py==0.8.1
brotli==1.1.0
certifi==2024.8.30
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.0
click==8.1.7
cryptography==44.0.0
cssselect==1.2.0
dacite==1.8.1
deprecation==2.1.0
faust-cchardet==2.1.19
frozenlist==1.5.0
gevent==24.11.1
geventhttpclient==2.3.3
gotrue==2.11.0
greenlet==3.1.1
h11==0.14.0
h2==4.1.0
hpack==4.0.0
hrequests==0.8.2
httpcore==1.0.7
httpx==0.27.2
hyperframe==6.0.1
idna==3.10
iso8601==2.1.0
jellyfish==1.1.0
jmespath==1.0.1
jwcrypto==1.5.6
lxml==5.3.0
markdown-it-py==3.0.0
mdurl==0.1.2
multidict==6.1.0
ndjson==0.3.1
numpy==2.1.3
orjson==3.10.12
packaging==24.2
parse==1.20.2
parsel==1.9.1
pillow==11.1.0
postgrest==0.18.0
propcache==0.2.0
psycopg==3.2.3
psycopg-binary==3.2.3
pyactiveresource==2.2.2
pycparser==2.22
pydantic==2.10.2
pydantic-core==2.27.1
pygments==2.18.0
pyhumps==3.8.0
pyjwt==2.10.1
pymupdf==1.25.1
pypdf2==3.0.1
python-dateutil==2.9.0.post0
pyyaml==6.0.2
realtime==2.0.6
reportlab==4.2.5
requests==2.32.3
requests-toolbelt==1.0.0
rich==13.9.4
selectolax==0.3.26
setuptools==75.6.0
shopifyapi==12.7.0
six==1.16.0
sniffio==1.3.1
sqlalchemy==2.0.36
sqlmodel==0.0.22
storage3==0.9.0
structlog==24.4.0
supabase==2.10.0
supafunc==0.7.0
tenacity==9.0.0
typing-extensions==4.12.2
ujson==5.10.0
urllib3==2.2.3
w3lib==2.2.1
websockets==13.1
wmill==1.431.1
xmltodict==0.14.2
yarl==1.18.0
zope-event==5.0
zope-interface==7.2Yep, that seems to work for me as well without that workaround ☺️
Does it work in vscode?
Yeah! Maybe this one was causing the issue
Yeah works well on vscode and online
Oh, yeah, that was the cause, I just thought your resolver put it there for a reason
Cool
Maybe i installed wrong version of an old lib and cause this pain oh man sorry for this
FYI Also found another edge case error on "Custom HTTP routes":
When i removed the user that deployed a custom route and then try to call that route i receive:
"Internal: Could not insert into queue 0194469a-4767-6d58-9f5b-76bec3cf6b08 with tag python3, schedul"
I can't help much with it, may be @Hugo can
Just want to report it thanks for your help!
You are welcome!
Hi @Osmar , that's expected
when using custom http routes the script will be run as the user who created/edited the route so if that user doesn't exist anymore it won't work (no permission to execute the job)
@Hugo (we can probably improve the error message however)