Scala.
Scala is a JVM language combining object-oriented and functional programming with a strong static type system. In data work it is the language Spark is written in, and it is common in high-throughput streaming systems.
Where Scala fits on a long engagement.
Scala's place in data work is largely Spark's doing. The engine is written in it, the Scala API is the most complete and the fastest, and understanding Spark's behaviour at depth usually means reading Scala. For high-throughput jobs the difference against the Python API is measurable.
The language permits a very wide range of styles, from lightly functional Java to deeply abstract type-level programming, and a codebase that has drifted toward the latter is genuinely difficult for anyone else to maintain. Agreeing and holding a restrained style is more consequential on Scala than on most languages.
What an assigned team does with Scala.
Scala expertise is comparatively rare, which affects how a team is composed. A codebase that only one person can maintain is a risk regardless of how good that person is.
Building depth deliberately across a team rather than depending on an individual is what the Talent Success and Academy programme is for.
What we use Scala for.
- Spark jobs where throughput matters The native API, where the Python overhead is measurable at volume.
- Type safety across pipeline stages Schema mismatches caught at compile time rather than mid-run.
- A style restrained enough to maintain Conventions agreed, so the codebase does not become one person's dialect.
How Scala capacity is assigned.
Scala capacity is assigned inside data engineering under data engineering outsourcing, with codebase style agreed rather than left to individual preference.
Tell us what your roadmap needs Scala for.
A service delivery manager replies with the disciplines we would assign, the monthly capacity and what the first month looks like.
Loading the contact form… You can also email hello@azendo.co.
We reply within one working day. No obligation, and no newsletter.