Juvio brings inline, PEP 723-style dependency management and automatic, ephemeral env setup to Jupyter notebooks.
Comments (18)
heisenzombie · 2h ago
I have been thinking about this for months now! Very excited to see you've implemented it, and I'm excited to try this out.
Could be fantastic for my use-case. We have a large repo of notebooks that are generally write-once, sometimes-re-run. Having a separate repo/venv/kernel per notebook would be a big overhead, so I currently just try to do a kind of semantic versioning where I make a new kernel on something like a 6-month cadence and try to keep breaking dependency changes to a minimum in that window. I can then keep around the old kernels for running old notebooks that depend on them. This is not at all an ideal state.
Thanks for sharing!
flakiness · 2h ago
The "git friendly format" is nice! How do markdown cells look like? Are they embedded as a python comment?
simlevesque · 5h ago
Seems awesome ! I'll try it soon.
okost1 · 5h ago
Thank you! I am looking forward to your feedback.
antman · 5h ago
Would it work on Jupyter lite?
okost1 · 5h ago
Unfortunately it won't, at least due to the fact UV is not available in the in-browser/wasm ecosystem. That would be awesome though. Maybe it is possible to make something close in terms of functionality using a custom pyodide kernel + micropip, but I did not look into that.
imcritic · 5h ago
> Why Use Juvio?
> No additional lock or requirements files are needed
Additional to what?
> Guaranteed reproducibility
Of what?
I probably need your project, but I don't understand what it is for.
okost1 · 5h ago
Hi. I appreciate your feedback. Basically, juvio stores all of the project requirements (versions of the packages and of the python interpreter) directly within the notebook itself using the PEP 723 spec. Then, when you open the notebook, a new ephemeral environment is created on the fly with all of the required dependencies. Therefore, you don't have to maintain a separate e.g. requirements.txt/conda.yaml/uv.lock file.
lyjackal · 4h ago
This is cool and something that I’ve wanted, but I don’t see hot listings requirements inline foregoes the need for a lock file to maintain reproducibility. What about version ranges? Versions of transitive dependencies?
Did you in the past? Normally Jupyter notebooks just include the package installation commands necessary to set up the environment from scratch. I've never seen a requirements.txt/lockfile distributed alongside a notebook.
mrbungie · 4h ago
That's common when they are distributed as single notebooks (i.e. via Google Colab). When distributed inside repos they usually contain a requirements.txt.
dylukes · 4h ago
I've done research involving sampling from all extant notebooks on GitHub and manually analyzing them, and have been a heavy user of them in non-CS research contexts for a long time (and Mathematica notebooks for years before iPython Notebooks grew in popularity).
Having manually sifted through hundreds of randomly sampled notebooks, I feel I can confidently speak on the distribution of characteristics in them, at least up through a couple years ago.
1. Notebooks on GitHub are not necessarily an entirely representative sample of notebooks at large. If the author is putting it on GitHub, there's already a significant selection bias towards certain topics, despite notebooks being used in practically any discipline you can think of at least to some extent.
2. Notebooks in repositories that contain requirements.txt files are a minority and tend to be intended/cleaned up for sharing (itself not the norm).
3. What's more common is just a handful of !pip install at the top.
4. Even more common is just some details on dependencies in an adjoining README.
A very non-trivial chunk of notebooks on GitHub are just copies of the "Hands On ML 3" textbook/exercise set. If my memory serves there are tens of thousands of copies of that one repository. The fork count shown by GitHub doesn't account for the plethora of copies that weren't made using it.
paddy_m · 4h ago
Specifically "Hands On ML 3", or just generally a copy of some exercise set from a class?
Hi. Thanks for bringing this up. To be honest, I have never tried juv, but judging from the readme the ideas of juv and juvio are slightly different. In juvio the ephemeral environment is created on kernel startup. Hence, one can have multiple notebooks within the same jupyterlab session, each with its own venv. This seems to be different with juv, but please correct me if I am wrong.
epistasis · 2h ago
I've been using juv on and off for for ~6 months. From what I can tell of juvio, it is a different model for using uv with jupyter notebooks.
I'm not sure which model fits best, I'll have to see how your juvio handles kernels in jupyter. Does the kernel name change, is it all the default kernel, and what changes when an install happens?
I'm not quite sure what you mean by cleaner git diffs, but hopefully that will become clear with experimentation.
For my particular method of working, I've mostly switched to having each small project (roughly a JIRA ticket) be a separate uv-managed project in a git repo, and I create a kernel for each of the uv projects. This allows me to examine multiple different tickets and kernels without having to launch multiple jupyter labs.
The whole kernel<->venv mapping is another layer of massive complexity on top of the current huge amount of complexity in Python packaging. uv makes it fast , but it does not provide the "correct" or even single route to managing venvs.
dockercompost · 2h ago
> In juvio the ephemeral environment is created on kernel startup. Hence, one can have multiple notebooks within the same jupyterlab session, each with its own venv.
Could be fantastic for my use-case. We have a large repo of notebooks that are generally write-once, sometimes-re-run. Having a separate repo/venv/kernel per notebook would be a big overhead, so I currently just try to do a kind of semantic versioning where I make a new kernel on something like a 6-month cadence and try to keep breaking dependency changes to a minimum in that window. I can then keep around the old kernels for running old notebooks that depend on them. This is not at all an ideal state.
Thanks for sharing!
> No additional lock or requirements files are needed
Additional to what?
> Guaranteed reproducibility
Of what?
I probably need your project, but I don't understand what it is for.
Having manually sifted through hundreds of randomly sampled notebooks, I feel I can confidently speak on the distribution of characteristics in them, at least up through a couple years ago.
1. Notebooks on GitHub are not necessarily an entirely representative sample of notebooks at large. If the author is putting it on GitHub, there's already a significant selection bias towards certain topics, despite notebooks being used in practically any discipline you can think of at least to some extent.
2. Notebooks in repositories that contain requirements.txt files are a minority and tend to be intended/cleaned up for sharing (itself not the norm).
3. What's more common is just a handful of !pip install at the top.
4. Even more common is just some details on dependencies in an adjoining README.
A very non-trivial chunk of notebooks on GitHub are just copies of the "Hands On ML 3" textbook/exercise set. If my memory serves there are tens of thousands of copies of that one repository. The fork count shown by GitHub doesn't account for the plethora of copies that weren't made using it.
https://github.com/manzt/juv
I'm not sure which model fits best, I'll have to see how your juvio handles kernels in jupyter. Does the kernel name change, is it all the default kernel, and what changes when an install happens?
I'm not quite sure what you mean by cleaner git diffs, but hopefully that will become clear with experimentation.
For my particular method of working, I've mostly switched to having each small project (roughly a JIRA ticket) be a separate uv-managed project in a git repo, and I create a kernel for each of the uv projects. This allows me to examine multiple different tickets and kernels without having to launch multiple jupyter labs.
The whole kernel<->venv mapping is another layer of massive complexity on top of the current huge amount of complexity in Python packaging. uv makes it fast , but it does not provide the "correct" or even single route to managing venvs.
This should be your primary selling point!