A Brief History of ZIO

On June 5th, 2017, I opened an issue in the Scalaz repository on Github, arguing that the next major version of this library needed a powerful and fast data type for asynchronous programming. My friend Vincent Marquez encouraged me to contribute to Scalaz, and I volunteered to build one.

This was not my first foray into the space of asynchronous computing. Previously, I had designed and built the first version of Aff and assisted the talented Nathan Faubion with the second iteration.

The Aff library quickly became the number one solution for async and concurrent programming in the Purescript ecosystem, offering powerful features not available in Javascript Promises.

Before Aff, I had written a Future data type for Scala, which I abandoned after the Scala standard library incorporated a much better version. Before that, I wrote a Promise data type for the haXe programming language and a Raincheck data type for Java.

In every case, I made mistakes and subsequently learned… to make new and different mistakes!

That sunny afternoon in June 2017, I imagined spending a few months developing this new data type, at which point I would wrap it up in a tidy bow and hand it over to the Scalaz organization.

What happened next, however, I never could have imagined.

The Birth of ZIO

As I spent free time working on the core of the new project, I increasingly felt like the goal of my work should not be to just create a pure functional effect wrapper for asynchronous side-effects.

That had been done before, and while performance could be improved over Scalaz 7, libraries built on pure functional programming cannot generally be as fast as bare-metal, abstraction-free, hand-optimized procedural code.

Now, for developers like me who are sold on functional programming, an effect wrapper is enough, but if all it does is slap a Certified Pure label on imperative code, it’s not going to encourage broader adoption. Although solving the async problem is still useful in a pre-Loom world, lots of other data types already solved this problem, such as Scala’s own Future.

Instead, I thought I needed to focus the library on concrete pains that are well-solved by functional programming, and one pain stood out among all others: concurrency, including the unsolved problem of how to safely cancel executing code whose result is no longer needed, due to timeout or other failures.

Concurrency is a big space. Whole libraries and ecosystems have been formed to tame its wily ways. Indeed, some frameworks become popular precisely because they shield developers from concurrency, because it’s complex, confusing, and error-prone.

Given the challenges, I thought I should directly support concurrency in this new data type, and give it features that would be impossible to replicate in a procedural program without special language features.

This vision of a powerful library for safe concurrent programming drove my early development.

Contentious Concurrency

At the time I began working on the Scalaz 8 project, the prevailing dogma in the functional Scala ecosystem was that an effect type should have little or no support for concurrency.

Indeed, some argued that effect concurrency was inherently unsafe and must be left to streaming libraries, like FS2 (a popular library for doing concurrent streaming in Scala).

Nonetheless, having seen the amazing work coming out of Haskell and F#, I believed it was not only possible but essential for a modern effect type to solve four closely related concurrency concerns:

• Spawning a new independent ‘thread’ of computation
• Asynchronously waiting for a ‘thread’ to finish computing its return value
• Automatically canceling a running ‘thread’ when its return value is no longer needed
• Ensuring cancellation does not leak resources

In the course of time, I developed a small prototype of what became known as the Scalaz 8 IO monad, which solved these problems in a fast and purely functional package.

In this prototype, effects could be forked to yield a fiber (a cooperatively-yielding virtual thread), which could be joined or instantly interrupted, with a Haskell-inspired version of try/finally called bracket that provided resource safety, even in the presence of asynchronous or concurrent interactions.

I was very excited about this design, and I talked about it publicly before I released the code, resulting in some backlash from competitors who doubted resource safety or performance. However, on November 16th, 2017, I presented the first version at Scale by the Bay, opening a pull request with full source code, including rigorous tests and benchmarks, which allayed all concerns.

Despite initial skepticism and criticism, in time, all effect systems in Scala adopted this same model, including the ability to launch an effect to yield a fiber, which could be safely interrupted or joined, with support for finalizers to ensure resource safety.

This early prototype was not yet ZIO as we know it today, but the seeds of ZIO had been planted, and they grew quickly.

Typed Errors & Other Evolution

My initial design for the Scalaz 8 IO data type was inspired by the Haskell IO type and the Task type from Scalaz 7. In due time, however, I found myself reevaluating some decisions made by these data types.

For one, I was unhappy with the fact that most effect types have dynamically typed errors. The compiler can’t help you reason about error behavior if you pretend that every computation can always fail in infinite ways.

As a statically-typed functional programmer, I want to use the compiler to help me write better code. I can do a better job if I know where I have and have not handled errors, and if I can use typed data models for business errors.

Of course, Haskell can sort of give you statically-typed errors with monad transformers, and maybe type classes. Unfortunately, this solution increases barriers to entry, reduces performance, and bolts on a second, often confusing error channel.

Since I was starting from a clean slate in a different programming language, I had a different idea: what if instead of rigidly fixing the error type to Throwable, like the Scala Try data type, I let the user choose the error type, exactly like Either?

Initial results of my experiment were remarkable: just looking at type signatures, I could understand exactly how code was dealing with errors (or not dealing with them). Effect operators precisely reflected error handling behavior in type signatures, and some laws that traditionally had to be checked using libraries like ScalaCheck were now checked statically at compile time.

So on January 2nd, 2018, I committed what ended up being a radical departure from the status quo: introducing statically-typed errors into the Scalaz 8 effect type.

Over the months that followed, I worked on polish, optimization, bug fixes, tests, and documentation, and found growing demand to use the data type in production. When it became apparent that Scalaz 8 was a longer-term project, a few ambitious developers pulled the IO data type into a standalone library so they could begin using it in their projects.

I was excited about this early traction, and I didn’t want any obstacles to using the data type for users of Scalaz 7.x or Cats, so on June 11th, 2018, I decided to pull the project out into a new, standalone project with zero dependencies, completely separate from Scalaz 8.

I chose the name ZIO, combining the “Z” from “Scalaz”, and the “IO” from “IO monad”.

Contributor Growth

Around this time, the first significant wave of contributors started joining the project, including Regis Kuckaertz, Wiem Zine Elabidine, and Pierre Ricadat (among others)—many new to both open source and functional Scala, although some with deep backgrounds in both.

Through mentorship by me and other contributors, including in some cases weekly meetings and pull request reviews, a whole new generation of open source Scala contributors were born—highly talented functional Scala developers whose warmth positivity and can-do spirit started to shape the community.

ZIO accreted more polish and features to make it easier to build concurrent applications, such as an asynchronous, doubly-back-pressured queue, better error tracking and handling, rigorous finalization, and lower-level resource-safety than bracket.

Although the increased contributions led to an increasingly capable effect type, I personally found that using ZIO was not very pleasant, because of the library’s poor type inference.

Improved Type-Inference

Like many functional Scala developers at the time, I had absorbed the prevailing wisdom about how functional programming should be done in Scala, and this meant avoiding subtyping and declaration-site variance. (Indeed, the presence of subtyping in a language does negatively impact type inference, and using declaration-site variance has a couple drawbacks.)

However, because of this mindset, using ZIO required specifying type parameters when calling many methods, resulting in an unforgiving and joyless style of programming, particularly with typed errors. In private, I wrote a small prototype showing that using declaration-site variance could significantly improve type inference, which made me want to implement the feature in ZIO.

At the time, however, ZIO still resided in the Scalaz organization, in a separate repository. I was aware that such a departure from the status quo would be very controversial, so in a private fork, Wiem Zine Elabidine and I worked together on a massive refactoring in our first major collaboration.

On Friday July 20th, 2018, we opened the pull request that embraced subtyping and covariance. The results spoke for themselves: nearly all explicit type annotations had been deleted, and although there was still some controversy, it was difficult to argue with the results. With this change, ZIO started becoming pleasant to use, and the extra error type parameter no longer negatively impacted usability because it could always be inferred and widened seamlessly as necessary.

This experience emboldened me to start breaking other taboos: I started aggressively renaming methods and classes and removing jargon known only to pure functional programmers. At each step, this created yet more controversy, but also further differentiated ZIO from some of the other options in the landscape, including those in Scalaz 7.x.

From all this turbulent evolution, a new take on functional Scala entered the ZIO community: a contrarian but principled take that emphasizes practical concerns, solving real problems in an accessible and joyful way, using all of Scala, including subtyping and declaration-site variance.

Finally, the project began to feel like the ZIO of today, shaped by a rapidly growing community of fresh faces eager to build a new future for functional programming in Scala.

Batteries Included

Toward the latter half of 2018, ZIO got compositional scheduling, with a powerful new data type that represents a schedule, equipped with rich compositional operators. Using this single data type, ZIO could either retry effects or repeat them according to near arbitrary schedules.

Artem Pyanykh implemented a blazing fast low-level ring-buffer, which, with the help of Pierre Ricadat, became the foundation of ZIO’s asynchronous queue, demonstrating the ability of the ZIO ecosystem to create de novo high-performance JVM structures.

Itamar Ravid, a highly talented Scala developer, joined the ZIO project and added a Managed data type encapsulating resources. Inspired by Haskell, Managed provided compositional resource safety in a package that supported parallelism and safe interruption. With the help of Maxim Schuwalow, Managed has grown to become an extremely powerful data type.

Thanks to the efforts of Raas Ahsan, ZIO unexpectedly got an early version of what would later become FiberRef, a fiber-based version of ThreadLocal. Then Kai, a wizard-level Scala developer and type astronaut, labored to add compatibility with Cats Effect libraries so that ZIO users could benefit from all the hard work put into libraries like Doobie, http4s, and FS2.

Thanks to the work of numerous contributors spread over more than a year, ZIO became a powerful solution to building concurrent applications—albeit, one without concurrent streams.

ZIO Stream

Although Akka Streams provides a powerful streaming solution for Scala developers, it’s coupled to the Akka ecosystem and Scala’s Future, and doesn’t embrace the full compositional power of Scala.

In the functional Scala space, FS2 provides a streaming solution that works with ZIO but it’s based on Cats Effect, whose type classes can’t benefit from ZIO-specific features.

I knew that a ZIO-specific streaming solution would be more expressive and more type safe, with a lower barrier of entry for existing ZIO users. Given the importance of streaming to modern applications, I decided that ZIO needed its own streaming solution, one unconstrained by the limitations of Cats Effect.

Bringing a new competitive streaming library into existence would be a lot of work, and so when Itamar Ravid volunteered to help, I instantly said yes.

Together, in the third quarter of 2018, Itamar and I worked in secret on ZIO Stream, an asynchronous, back-pressured, resource-safe, and compositional stream. Inspired by the remarkable work of Eric Torreborre, as well as work in Haskell on iteratees, the initial release of ZIO Streams delivered high-performance, composable concurrent streams, and sinks, with strong guarantees of resource safety, even in the presence of arbitrary interruption.

We unveiled the design at Scale by the Bay 2018, and since then, thanks to Itamar and his army of capable contributors (including Regis Kuckaertz), ZIO Streams has become one of the highlights of the ZIO library—every bit as capable as other streaming libraries, but with much smoother integration with the ZIO effect type and capabilities.

Toward the end of 2018, I decided to focus on the complexity of testing code written using effect systems, which led to the last major revision of the ZIO effect type.

ZIO Environment

When exploring a contravariant reader data type to model dependencies, I discovered that using intersection types (emulated by the with keyword in Scala 2.x), one could achieve flawless type inference when composing effects with different dependencies, which provided a possible solution to simplifying testing of ZIO applications.

Excitedly, I wrote up a simple toy prototype and shared it with Wiem Zine Elabidine. “Do you want to help work on this?” I asked. She said yes, and together, we quietly added the third and final type parameter to the ZIO effect type: the environment type parameter.

I unveiled the third type parameter at a now-infamous talk, The Death of Finally Tagless, humorously presented with a cartoonish Halloween theme. In this talk, I argued that testability was the primary benefit of the so-called “tagless-final” technique, and that it could be obtained much more simply and in a more teachable way by just “passing interfaces”—the very same solution that object-oriented programmers have used for decades.

As with tagless-final, and under the assumption of discipline, ZIO Environment provided a way to reason about dependencies statically. But unlike tagless-final, it’s a joy to use because it fully infers, and doesn’t require teaching type classes, category theory, higher-kinded types, and implicits.

Some ZIO users immediately started using the ZIO Environment, appreciating the ability to describe dependencies using types without actually passing them. Constructing ZIO environments, however, proved to be problematic—impossible to do generically, and somewhat painful to do even when the structure of the environment was fully known.

A workable solution to these pains would not be identified until almost a year later.

Meanwhile, ZIO continued to benefit from numerous contributions, which added operators, improved documentation, improved interop, and improved semantics for core data types.

The next major addition to ZIO was software transactional memory.

Software Transactional Memory

The first prototype of the Scalaz IO data type included MVar, a doubly-back-pressured queue with a maximum capacity of 1, inspired by Haskell’s data type of the same name.

I really liked the fact that MVar was already “proven”, and could be used to build many other concurrent data structures (such as queues, semaphores, and more).

Soon after that early prototype, however, the talented and eloquent Fabio Labella convinced me that two simpler primitives provided a more orthogonal basis for building concurrency structures:

• Promise, a variable data type that can be set exactly one time (but can be awaited on asynchronously and retrieved any number of times);
• Ref, a model of a mutable cell that can store any immutable value, with atomic operations for updating the cell.

This early refactoring allowed us to delete MVar and provided a much simpler foundation. However, after a year of using these structures, while I appreciated their power, it became apparent to me that they were the “assembly language” of concurrent data structures.

These structures could be used to build lots of other asynchronous concurrent data structures, such as semaphores, queues, and locks, but doing so was extremely tricky, and required hundreds of lines of fairly advanced code.

Most of the complexity stems from the requirement that operations on the data structures must be safely interruptible, without leaking resources or deadlocking.

Moreover, although you can build concurrent structures with Promise and Ref, you cannot make coordinated changes across two or more such concurrent structures.

The transactional guarantees of structures built with Promise and Ref are non-compositional: they apply only to isolated data structures because they are built with Ref, which has non-compositional transactional semantics. Strictly speaking, their transactional power is equivalent to actors with mutable state: each actor can safely mutate its own state, but no transactional changes can be made across multiple actors.

Familiar with Haskell’s software transactional memory, and how it provides an elegant, compositional solution to the problem of developing concurrent structures, I decided to implement a version for ZIO with the help of my partner-in-crime Wiem Zine Elabidine, which we presented at Scalar Conf in April 2019.

Soon after, Dejan Mijic, a fantastic and highly motivated developer with a keen interest in high-performance, concurrency, and distributed systems, joined the ZIO STM team. With my mentorship, Dejan helped make STM stack-safe for transactions of any size, added several new STM data structures, dramatically improved the performance of existing structures, and implemented retry-storm protection for supporting large transactions on hotly contested transactional references.

ZIO STM is the only STM in Scala with these features, and although the much older Scala STM is surely production-worthy, it doesn’t integrate well with asynchronous and purely functional effect systems built using fiber-based concurrency.

The next major feature in ZIO would address a severe deficiency that had never been solved in the Scala ecosystem: the extreme difficulty of debugging async code, a problem present in Scala’s Future for more than a decade.

Execution Traces

Previously in presenting ZIO to new non-pure functional programmers (the primary audience for ZIO), I had received the question: how do we debug ZIO code?

The difficulty stems from the worthless nature of stack traces in highly asynchronous programming. Stack traces only capture the call stack, but in Future and ZIO and other heavily async environments, the call stack mainly shows you the “guts” of the execution environment, which is not very useful for troubleshooting errors.

I had thought about the problem and had become convinced it would be possible to implement async execution traces using information reconstructed from the call stack, so I began telling people we would soon implement something like this in ZIO.

I did not anticipate just how soon this would happen.

Kai came to me with an idea to do execution tracing in a radically different way than I imagined: by dynamically parsing the bytecode of class files. Although my recollection is a bit hazy, it seemed mere days before Kai had whipped up a prototype that seemed extremely promising, so I offered my assistance on hammering out the details of the full implementation, and we ended up doing a wonderful joint talk in Ireland to launch the feature.

Sometimes I have a tendency to focus on laws and abstractions, but seeing the phenomenally positive response to execution tracing was a good reminder to stay focused on the real world pains that developers have.

Summer 2019

Beginning in the summer of 2019, ZIO began seeing its first significant commercial adoption, which led to many feature requests and bug reports, and much feedback from users.

The summer saw many performance improvements, bug fixes, naming improvements, and other tweaks to the library, thanks to Regis Kuckaertz and countless other contributors.

Thanks to the work of the ever-patient Honza Strnad and others, FiberRef evolved into its present-day form, which is a much more powerful, fiber-aware version of ThreadLocal—but one which can undergo specified transformations on forks, and merges on joins.

I was very pleased with these additions. However, as ZIO grew, the automated tests for ZIO were growing too, and they became an increasing source of pain across Scala.js, JVM, and Dotty (the test runners at the time did not natively support Dotty).

So in the summer of 2019, I began work on a purely functional testing framework, with the goal of addressing these pains, the result of which was ZIO Test.

ZIO Test

Testing functional effects inside a traditional testing library is painful: there’s no easy way to run effects, provide them with dependencies, or integrate with the host facilities of the functional effect system (using retries, repeats, and so forth).

I wanted to change that with a small, compositional library called ZIO Test, whose design I had been thinking about since even before ZIO existed.

Like the ground-breaking Specs2 before it, ZIO Test embraced a philosophy of tests as values, although ZIO Test retained a more traditional tree-like structure for specs, which allows nesting tests inside test suites, and suites inside other suites.

Early in the development of ZIO Test, the incredible and extremely helpful Adam Fraser joined the project as a core contributor. Instrumental to fleshing out, realizing, and greatly extending the vision for ZIO Test, Adam has since become the lead architect and maintainer for the project, and co-author of this book.

Piggybacking atop ZIO’s powerful effect type, ZIO Test was implemented in comparatively few lines of code: concerns like retrying, repeating, composition, parallel execution, and so forth, were already implemented in a principled, performant, and type-safe way.

Indeed, ZIO Test also got a featherweight alternative to ScalaCheck based on ZIO Streams, since a generator of a value can be viewed as a stream. Unlike ScalaCheck, the ZIO Test generator has auto-shrinking baked in, inspired by the Haskell Hedgehog library; and it correctly handles filters on shrunk values and other edge case scenarios that ScalaCheck did not handle.

Toward the end of 2018, after nearly a year of real world usage, the ZIO community had been hard at work on solutions to the problem of making dynamic construction of ZIO environments easier.

This work directly led to the creation of ZLayer, the last major data type added to ZIO.

ZLayer

Two very talented Scala developers, Maxim Schuwalow and Piotr Gołębiewski, jointly worked on a ZIO Macros project, which, among other utilities, provided an easier way to construct larger ZIO environments from smaller pieces. This excellent work was independently replicated in Netflix’s highly-acclaimed Polynote by Scala engineer Jeremy Smith, in response to the same pain.

At Functional Scala 2019, several speakers presented on the pain of constructing ZIO Environments, which convinced me to take a hard look at the problem. Taking inspiration from an earlier attempt by Piotr, I created two new data types, Has and ZLayer.

Has can be thought of as a type-indexed heterogeneous map, which is typesafe but requires access to compile-time type tag information. ZLayer can be thought of as a more powerful version of Java and Scala constructors, which can build multiple services in terms of their dependencies.

Unlike constructors, ZLayer dependency graphs are ordinary values, built from other values using composable operators, and ZLayer supports resources, asynchronous creation and finalization, retrying, and other features not possible with constructors.

ZLayer provided a very clean solution to the problems developers were having with ZIO Environment—not perfect, mind you, and I don’t think any solution prior to Scala 3 can be perfect (every solution in the design space has different tradeoffs). This solution became even better when the excellent consultancy Septimal Mind donated Izumi Reflect to the ZIO organization.

The introduction of ZLayer was the last major change to any core data type in ZIO. Since then, although streams has seen some evolution, the rest of ZIO was quite stable.

Yet despite the stability, until August 2020, there was still one major unresolved issue at the very heart of the ZIO runtime system: a full solution to the problem of structured concurrency.

Structured Concurrency

Structured concurrency is a paradigm that provides strong guarantees around the lifespans of operations performed concurrently. These guarantees make it easier to build applications that have stable, predictable resource utilization.

Since I have long been a fan of Haskell structured concurrency (via Async and related), ZIO was the first effect system to support structured concurrency in numerous operations:

• By default, interrupting a fiber does not return until the fiber has been interrupted and all its finalizers executed.
• By default, timing out an effect does not return until the effect being timed out has been interrupted and all its finalizers executed.
• By default, when executing effects in parallel, if one of them fails, the parallel operation will not continue until all sibling effects have been interrupted.
• Etc.

Some of these design decisions were highly contentious and have not been implemented in other effect systems until recently (if at all).

However, there was one notable area where ZIO did not provide default structured concurrency: whenever an effect was forked (launched concurrently to execute on a new fiber), the lifespan of the executing effect was unconstrained.

Solving this problem turned out to require major surgery to the ZIO internal runtime system (which is a part of ZIO that few contributors understand completely).

In the end, we solved the problem in a satisfactory way, making ZIO the only effect system to fully support structured concurrency. But it required learning from real world feedback and prototyping no less than 5 completely different solutions to the problem.

So after three years of development, on August 3rd, 2020, ZIO 1.0 was released live in an online Zoom-hosted launch party that brought together and paid tribute to contributors and users across the ZIO ecosystem. We laughed, we chatted, I rambled for a while, and we toasted to users, contributors, and the past and future of ZIO.

Preparation for ZIO 2.0

Following the release of ZIO 1.0, the journey was far from over. While the core fundamentals were solid, I knew there were still opportunities to make ZIO even more powerful and accessible. Rather than rushing into immediate changes, we chose a methodical approach, releasing a series of milestone versions that allowed us to gather valuable feedback from the growing ZIO community.

Throughout our journey, we encountered many obstacles that influenced our development. As we move forward, I will no longer detail every step of the evolution process. Instead, I will focus on the challenges we faced that shaped it into what it is today.

The direction for ZIO 2.0 centered on four fundamental principles that I thought would shape the library’s next phase:

• First, we wanted to improve ergonomics and the developer experience dramatically. The library had proven itself capable, but we knew we could make it more intuitive and enjoyable to use.
• Second, although ZIO was already fast, we saw opportunities for even more aggressive performance optimization. In the world of high-performance distributed systems, every millisecond counts.
• Third, we recognized the need to enhance operational capabilities. Modern applications demand robust observability and diagnostics, and we wanted ZIO to excel in production environments.
• Finally, we set our sights on streaming. While ZIO Stream was already powerful, we saw the potential to make it even more expressive and performant.

Simplification of ZIO Environment and Dependency Injection

While ZIO 1.0’s Has and ZLayer provided a powerful foundation for dependency management, real-world feedback revealed a gap between power and simplicity. This realization led me to what might have seemed unthinkable months earlier—the complete removal of the Has data type.

So, we introduced a new type-level map called ZEnvironment, which was built into the ZIO itself instead of exposing Has on the surface. This led us to remove the Has data type from type signatures and simplify the API.

Talented Scala developer Kit Langton played a crucial role in what followed. Using Scala’s compile-time capabilities, Kit Langton helped develop a sophisticated auto-wiring system to construct dependency graphs automatically. This enabled us to eliminate entire categories of boilerplate codes when injecting dependencies since the early days of ZIO.

ZIO Becomes Composable Resourceful Effect

While the Managed data type in ZIO 1.0 was powerful for handling resource safety, it created an unnecessary burden. Developers had to constantly switch between two parallel worlds: ZIO for regular effects and Managed for resource management. Additionally, features were duplicated between both types—any enhancements made to ZIO often required a corresponding implementation in Managed.

As I had previously consolidated error handling, environments, and concurrency into the core ZIO effect type, I saw an opportunity to fold resource management directly into ZIO itself. This simplification would eliminate one other data type from the library while making resource-safe programming more natural and maintainable.

So, instead of maintaining two parallel worlds, I introduced scopes as first-class values through collaboration with Adam Fraser. By introducing and adding the Scope data type to ZIO’s environment, I enabled resource management directly within ZIO’s effects. This wasn’t just a minor improvement but a fundamental unification of the library’s core abstractions.

Developers could now handle resources using familiar ZIO operators and patterns without switching back and forth between ZIO and ZManaged types. Whether working with concurrent effects or managing resources, everything stayed within the same powerful abstraction. This unification meant that any code accepting a ZIO value could seamlessly work with scoped resources.

Service Pattern

Despite the powerful capabilities we introduced to the ZIO Environment and ZLayer, we were still concerned about the significant amount of boilerplate needed to define and implement services. The ceremony of defining services felt at odds with ZIO’s emphasis on developer ergonomics and joy.

The module pattern was a step in the right direction but still required more simplification. I borrowed three key elements from object-oriented design:

• Interfaces to define service contracts
• Classes to implement services in terms of other service interfaces
• Constructor-based dependency injection to wire services together

This realization led to the ZIO Service Pattern 2.0, merging functional and object-oriented principles that felt intuitive to developers. This pattern helped developers structure ZIO applications that naturally align with clean architecture principles, particularly onion architecture.

Smart Assertions

Continuing to make the API more ergonomic and simpler, Kit Langton started working on a macro-based approach to simplify writing tests with assertions. The result was smart assertions, which allowed developers to express expected behavior using ordinary Scala expressions that return Boolean values. These expressions were then translated behind the scenes into fundamental assertions, making test code more readable and easier to understand.

More Concrete Types

The journey toward simplification that began with ZIO’s earliest days—when we chose concrete data types over tagless-final style—continued to guide our design decisions after ZIO 1.0. We have seen that polymorphic data types like Ref and Queue can be powerful but can also introduce unnecessary complexity and developer friction. So, we took the bold step of eliminating these polymorphic variants, keeping only their concrete counterparts. This opinionated move further streamlined the library, making it more approachable for commercial teams.

Unified Streaming

Another area of our focus was ZIO Streams. I knew we needed to improve the ZIO Stream. During long discussions with ZIO contributors, I became convinced that we needed a more principled, lower-level abstraction that could unify all our streaming operations.

This led to the creation of ZChannel, a powerful abstraction inspired by Java NIO channels but reimagined in a purely functional way. Like its inspiration, a ZChannel supports both reading and writing operations but with the full power of ZIO’s type system and functional programming principles.

With the help of the growing ZIO team, particularly the talented contributor Daniel Vigovszky, we rebuilt our entire streaming infrastructure on top of this new foundation. ZStream, ZPipeline, and ZSink—while maintaining their distinct APIs that users had come to rely on—were now all unified under the hood as specialized channels.

I was particularly excited about how this unification made it easier for users to work with different streaming abstractions. If a developer needed to build something more advanced, they could now directly use channels, enabling their solutions to integrate seamlessly with all existing streaming data types.

Regional Settings and Contextual Scopes

Embracing regional settings through structured programming principles was another goal concerning ergonomics and developer experience. I envisioned a system where settings can be only referenced and controlled on their block scope. With the help of Adam Fraser, we tackled this challenge by first enhancing FiberRef to support compositional updates. This enhancement became the foundation for a complete overhaul of ZIO’s configuration system. Now, we could have nested scopes with different settings, each affecting the behavior of the ZIO application. Then, the same principles were applied to other aspects of ZIO, including concurrency, logging, and tracing.

Observability

I remember those days when the problem of useless stack traces had plagued effect systems for years, and ZIO was no exception. When an error occurred, developers would get stack traces full of framework internals rather than helpful information about where their code had failed.

A talented ZIO contributor, Rob Walsh, started working on a solution to this problem and presented his work on ZIO World. With his contribution, ZIO’s execution tracing took to the next level. ZIO became the first effect system to provide truly useful execution traces for asynchronous code, which pointed directly to the failing user code instead of getting lost in framework internals. This has a fantastic outcome: Any developer who could read a Java stack trace could now read ZIO’s cost-free async stack traces.

Toward our goal to make ZIO the foundation for writing cloud-native applications, we also tried to have built-in solutions for logging, metrics, and integrated fiber dumps. Now, it was easier than ever to understand what was happening in ZIO applications with built-in and out-of-the-box observability solutions.

Performance

Performance had always been a key focus for ZIO, but I wanted to push the boundaries even further. Adam Fraser took on this challenge, creating a sophisticated fiber-aware scheduler inspired by Rust’s Tokio. His implementation introduced work-stealing algorithms that could automatically balance workloads across operating system threads, maximizing fiber thread affinity while utilizing all available cores.

I also began a complete overhaul of the ZIO runtime to eliminate unnecessary work and bring you as close to bare metal as possible. This led to a significant performance improvement in ZIO applications and laid the foundation for post-Loom Java. This work led to what I consider the first “third generation” runtime in the effect ecosystem—one that minimizes JVM stack usage and delivers unprecedented performance among effect systems.

Another area of performance optimization we focused on was the challenge of developers handling blocking versus non-blocking operations.

In ZIO 1.0, we had implemented a reasonable solution at the time: developers had to use the blocking operator to signal blocking operations to the runtime. This allowed the runtime to shift heavy blocking work to a separate thread pool, protecting our core asynchronous operations from being blocked.

While I was proud that ZIO 1.0 pioneered the first tooling to manage this through its blocking service—an innovation that other runtimes later adopted—I wasn’t fully satisfied. Developers needed to explicitly label their blocking code, which proved problematic in practice. When developers forgot to mark blocking operations or simply didn’t realize they were importing blocking code, their applications could suffer serious performance degradation or even deadlock.

Adam Fraser and I began tackling this challenge head-on. We envisioned a runtime smart enough to manage this complexity automatically, removing the burden from developers entirely. The result was auto-blocking, where the runtime could intelligently detect blocking operations and automatically shift them to a dedicated blocking thread pool—without requiring any explicit labeling from developers. This dramatically simplifies ZIO applications, eliminating the need to distinguish between blocking and non-blocking operations manually. However, in later releases, this feature was disabled by default.

ZIO 2.0

Finally, we released ZIO 2.0 on June 24, 2022, after two years of continuous development and improvements based on community feedback. ZIO 2.0 was the solid foundation for the future of asynchronous and concurrent programming in Scala.

After the release of ZIO 2.0, we have had several maintenance releases—reaching version 2.1.13 at the time of writing. With these releases, I consciously decided to shift our focus. Rather than continuing to add new features, I wanted to ensure long-term stability through careful maintenance and optimization. So, new contributors attracted to this vision, including Kyriacos Petrou, an incredibly skilled developer who stepped forward to improve the performance of the ZIO runtime system. His work included a complete reimplementation of our Software Transactional Memory runtime, which significantly improved performance under high contention scenarios.

What is Next?

Looking ahead into 2025, I see ZIO evolving in response to changes in the Scala ecosystem. While ZIO has established itself as the “enterprise effect system” with growing adoption throughout 2024, we must adapt to a shifting landscape where we are in a situation that fewer companies are starting new Scala projects, and many are exploring alternative languages with stronger stability and tooling histories.

This evolution demands a thoughtful restructuring of the ZIO ecosystem. Rather than continuing to expand horizontally with new projects, we’re focusing on vertical integration and stability—bringing essential components closer to the core while ensuring long-term maintainability.

Some projects that have proven themselves indispensable to the ZIO experience—including ZIO Logging, ZIO Config, ZIO Metrics, and ZIO Profiling—will be promoted into ZIO core. This integration will provide a more cohesive experience while ensuring these critical components receive the same rigorous maintenance and guarantees as the core library.

We’re also investing heavily in mature projects that form the backbone of production ZIO applications. Libraries like ZIO HTTP, ZIO Schema, ZIO JSON, and Quill will continue to receive focused attention and improvements. In some areas, rather than maintaining our own implementations, we’re identifying and supporting ecosystem-neutral libraries that can provide excellent ZIO integration without the overhead of full maintenance.

While I have exciting ideas for ZIO 3—ideas that push beyond anything currently explored in the effect system ecosystem—our immediate priority is stabilization. Companies building on ZIO 2.1 should expect a very long life from the 2.x line, where backward compatibility and binary compatibility take precedence over chasing the latest trends. This stability-first approach means continued support for both Scala 2.13 and Scala 3.x, ensuring that enterprises can build with confidence on the ZIO platform.

The next chapter in ZIO’s story will focus less on breaking new ground and more on strengthening the foundations we’ve built. By focusing our community’s energy on actively maintained projects and core capabilities, we’re ensuring that ZIO continues to be the top choice for building resilient, scalable cloud-native applications.

Why ZIO

ZIO is a new library for concurrent programming. Using features of the Scala programming language, ZIO helps you build efficient, resilient, and concurrent applications that are easy to understand and test, and which don’t leak resources, deadlock, or lose errors.

Used pervasively across an application, ZIO simplifies many of the challenges of building modern applications:

• Concurrency. Using an asynchronous fiber-based model of concurrency that never blocks threads or deadlocks, ZIO can run thousands or millions of virtual threads concurrently.
• Efficiency. ZIO automatically cancels running computations when the result of the computations are no longer necessary, providing global application efficiency for free.
• Error Handling. ZIO lets you track errors statically, so the compiler can tell you which code has handled its errors, and which code can fail, including how it can fail.
• Resource-Safety. ZIO automatically manages the lifetime of resources, safely acquiring them and releasing them even in the presence of concurrency and unexpected errors.
• Streaming. ZIO has powerful, efficient, and concurrent streaming that works with any source of data, whether structured or unstructured, and never leaks resources.
• Troubleshooting. ZIO captures all errors, including parallel and finalization errors, with detailed execution traces and suspension details that make troubleshooting applications easy.
• Testability. With dependency inference, ZIO makes it easy to code to interfaces, and ships with testable clocks, consoles, and other core system modules.

ZIO frees application developers to focus on business logic, and fully embraces the features of the Scala programming languages to improve productivity, testability, and resilience.

Since it’s 1.0 release in August of 2020, ZIO has sparked a teeming ecosystem of ZIO-compatible libraries that provide support for GraphQL, persistence, REST APIs, microservices, and much more.