Jupyter.
Jupyter provides notebook interfaces combining code, output and narrative in one document. It is the standard environment for exploratory data work, and a poor one for anything that needs to run reliably.
Where Jupyter fits on a long engagement.
For exploration, notebooks are genuinely the right tool. Iterating on a transformation while seeing the result, keeping the reasoning beside the code, and sharing an analysis someone else can follow are all things a script does worse.
Out-of-order execution is the property that makes them unreliable. A notebook's state depends on the sequence cells were run in, not on their order in the document, so a notebook that produces a result on one machine can produce a different one elsewhere. A notebook that has not been restarted and run top to bottom is not reproducible.
What an assigned team does with Jupyter.
The recurring organisational problem is notebooks in production. A pipeline that exists as a notebook someone runs each morning is a person with a habit rather than a system, and it fails the day they are unavailable.
Moving that work into tested, scheduled jobs is the transition that makes analytics dependable, assigned under data engineering outsourcing.
What we use Jupyter for.
- Exploration with the reasoning attached Analysis someone else can follow, rather than a script with no narrative.
- Reproducibility enforced Restart-and-run-all as a rule, because execution order is otherwise invisible.
- Promotion out of the notebook Recurring work converted into scheduled jobs that do not depend on a person.
How Jupyter capacity is assigned.
Analytical capacity is assigned with a clear line between exploration and production, so recurring work does not remain in a notebook indefinitely.
Tell us what your roadmap needs Jupyter for.
A service delivery manager replies with the disciplines we would assign, the monthly capacity and what the first month looks like.
Loading the contact form… You can also email hello@azendo.co.
We reply within one working day. No obligation, and no newsletter.