dry-rb news

dry-rb adopts Zeitwerk for code loading

2022-10-17T00:00:00+00:00

For the past few months we’ve been working on making all dry-rb gems auto-loadable through Zeitwerk. This is part of a bigger effort as Zeitwerk is also used by Hanami 2.0.

Using Zeitwerk to autoload not just your application code but also your dependencies leads to significantly faster boot times. That’s why it’s worth going through this process to have all the dry-rb gems use Zeitwerk ⚡

Updated gems

During the last few days we’ve released the following gems that now use Zeitwerk's GemLoader:

How to upgrade

In theory, everything should just work, but we’ve already found there can be various issues depending on how you require files from dry-rb gems. Luckily, the changes that may be required are very simple:

Change “cherry picking” requires of individual gem files (assuming you have them) to be just a single require of the gem’s entrypoint. Here's a couple of examples:

# before
require "dry/core/class_attributes"

# now
require "dry/core"

# before
require "dry/system/container"

# now
require "dry/system"

In case of dry-validation and its own dry-rb dependencies, please make sure that you upgrade everything to the current versions. Unfortunately, dry-validation 1.9.0 may still be installed with dry-schema < 1.11 due to how version requirements work in Rubygems. It’s best to have something like this in your Gemfile:
```
gem "dry-validation", "~> 1.9.0"
gem "dry-schema", "~> 1.11.2"
```
If you already use Zeitwerk and have teardown code in your test suite, please take a look how we’ve set it up in Hanami’s test suite to clear only the parts we need, without affecting loaders from other gems:

def autoloaders_teardown!
  # Tear down Zeitwerk (from zeitwerk's own test/support/loader_test)
  Zeitwerk::Registry.loaders.reject! do |loader|
    test_loader = loader.dirs.any? do |dir|
      dir.include?("/spec/") || dir.include?(Dir.tmpdir) ||
        dir.include?("/slices/") || dir.include?("/app")
    end

    if test_loader
      loader.unregister
      true
    else
      false
    end
  end
end

Reporting issues

If you find any crashes related to Ruby constants, please try to report them in the repository where the given missing constant is actually defined. If you’re unsure, just report them in the repository of the gem that you upgraded and gave you trouble.

Introducing dry-files

2021-05-04T00:00:00+00:00

We talked several times about the union of dry-rb + ROM + Hanami, well today we can share good news on that front: introducing dry-files.

It's a gem that abstracts low level file manipulations.

The code was originally created for hanami-utils, as a way to power Hanami command line. Then it was moved to dry-cli, when it was extracted from the Hanami code base. Today it finally made its own debut as a standalone gem.

dry-cli is a powerful framework to build Ruby command line interfaces. We use it as the main engine for the Hanami CLI, which also needs code generators. The initial idea was to have this optional dry-cli library to support code generators via file manipulations. But then we reached the point at which this library had more lines of code than dry-cli itself, so we decided to extract this library.

Here's a simple example:

# frozen_string_literal: true

require "dry/files"

files = Dry::Files.new
files.write("path/to/file", "Hello, World!") # intermediate directories are created, if missing

dry-files is shipped with an extensive API to touch, (re)write, read, and remove files/directories, inject/remove/append Ruby code lines and blocks, and so on.

Because of this abstraction we had the chance to introduce swappable adapters. One adapter (the default one) is for real file manipulations. It's meant to be used in production and integration tests. The other adapter is an in-memory file system. It's for very fast unit tests that cleanup by themselves.

# frozen_string_literal: true

require "dry/files"

files = Dry::Files.new(memory: true)
files.write("path/to/file", "Hello, World!") # create an in-memory file

Checkout the docs and the CHANGELOG to know more. Enjoy!

dry-system 0.19 released with Zeitwerk support and more, leading the way for Hanami 2.0

2021-04-22T00:00:00+00:00

We are very pleased to announce the release of dry-system 0.19.0! This release marks a huge step forward for dry-system, bringing support for Zeitwerk and other autoloaders, plus clearer configuration and improved consistency around component resolution for both finalized and lazy loading containers.

These changes will also pave the way for a seamless integration of Zeitwerk and dry-system into the new Hanami 2.0 application core. Until then, thanks to dry-rb serving as an independent foundation layer for Hanami, you can already try out these new features!

A dry-system container as of 0.19.0, configured for Zeitwerk, now looks like this:

require "dry/system/container"
require "dry/system/loader/autoloading"
require "zeitwerk"

class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__

    config.component_dirs.loader = Dry::System::Loader::Autoloading
    config.component_dirs.add_to_load_path = false

    config.component_dirs.add "lib"
  end
end

loader = Zeitwerk::Loader.new
loader.push_dir MyApp::Container.config.root.join("lib").realpath
loader.setup

Unlike the default loader, the new Dry::System::Loader::Autoloading does not require files itself when loading components. Instead, it references their class constants directly, allowing the missing constant resolution to trigger the autoloading behaviour of Zeitwerk and other autoloaders. This is all that's required to bring Zeitwerk and dry-system together! Combined with dry-system's auto-injector, you now have the best of both worlds: the convenience of auto-loading classes combined with all the loose-coupling benefits of injected dependencies.

The new component_dirs setting also allows multiple component dirs to be added (these are where dry-system looks when loading a container's components) and configured independently:

class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__

    # Defaults for all component dirs can be configured
    config.component_dirs.default_namespace = "my_app"

    # As well as settings for individual dirs
    config.component_dirs.add "lib" do |dir|
      dir.auto_register = proc do |component|
        !component.identifier.start_with?('entities')
      end
    end

    # Multiple component dirs can be added
    config.component_dirs.add "app"
  end
end

The auto_register and memoize component dir settings have been improved as part of this release, now accepting either simple truthy or falsey values, or a proc accepting a Dry::System::Component and returning a truthy or falsey value. Using a proc makes it easy to configure fine-grained behavior on a component-per-component basis. Check out also the new Dry::System::Identifier class as used above via component.identifier: this is a new class that provides namespace-aware methods for querying container component identifiers, which is particularly useful in cases like the above.

Finally, we've given a lot of attention to making sure dry-system containers work consistently regardless of whether they're finalized or lazy loading their components. For example, # auto_register: false magic comments are not respected in both cases, where previously they were ignored for a lazy loading container.

There's plenty more to learn about this release, including several breaking changes, so check out the changelog for all the details. And if you want to understand more of the thinking that went into these changes, also check out Tim’s open source status updates for this last November, December, January, and February (Yes, this release has been long in the making!).

With many internal improvements also in place for this release, we now see a clear picture of what's left before 1.0, and have filled out the dry-system 1.0 milestone with issues representing the remaining work. Please take a look and get in touch if you can help.

In the meantime, we hope you enjoy dry-system 0.19.0 and please let us know how you go with all the new features!

dry-schema and dry-validation 1.5.0 released

2020-03-11T00:00:00+00:00

We're happy to announce the release of dry-schema 1.5.0! It comes with plenty of new features, fixes, and general improvements. Here are some of the highlights.

Support for composing schemas

You can now compose schemas using logical operators. The only limitation is that xor is not supported yet as it wasn't clear how error messages are supposed to work. This feature is experimental until we finalize it in version 2.0.0.

In the meantime, please try it out! Here's a simple example:

RoleSchema = Dry::Schema.JSON do
  required(:id).filled(:string)
end

ExpirableSchema = Dry::Schema.JSON do
  required(:expires_on).value(:date)
end

UserSchema = Dry::Schema.JSON do
  required(:name).filled(:string)
  required(:role).hash(RoleSchema & ExpirableSchema)
end

UserSchema.(name: "Jane", role: { id: "admin", expires_on: "2020-05-01" }).errors.to_h
# {}

UserSchema.(name: "Jane", role: { id: "", expires_on: "2020-05-01" }).errors.to_h
# {role: {id: ["must be filled"]}}

UserSchema.(name: "Jane", role: { id: "admin", expires_on: "oops" }).errors.to_h
# {role: {expires_on: ["must be a date"]}}

Refer to the documentation for more information.

Errors about unexpected keys

Back in the dry-validation 0.x era, many people asked about returning errors for unexpected keys. Four years later, this feature is finally here! You can enable it with a simple config flag:

UserSchema = Dry::Schema.Params do
  # Enable key validation!
  config.validate_keys = true

  required(:name).filled(:string)

  required(:address).hash do
    required(:city).filled(:string)
    required(:zipcode).filled(:string)
  end

  required(:roles).array(:hash) do
    required(:name).filled(:string)
  end
end

input = {
  foo: 'unexpected',
  name: 'Jane',
  address: { bar: 'unexpected', city: 'NYC', zipcode: '1234' },
  roles: [{ name: 'admin' }, { name: 'editor', foo: 'unexpected' }]
}

UserSchema.(input).errors.to_h
# {
#  :foo=>["is not allowed"],
#  :address=>{:bar=>["is not allowed"]},
#  :roles=>{1=>{:foo=>["is not allowed"]}}
# }

Notice that it works even for arrays with hashes as elements, which is really nice!

Introspection extension

Another feature request that goes way back is easily seeing which keys are required and which are optional. This is now provided by a new :info extension, which shows both the keys and their associated types.

To enable it, you need to load the extension:

Dry::Schema.load_extensions(:info)

UserSchema = Dry::Schema.JSON do
  required(:email).filled(:string)
  optional(:age).filled(:integer)
  optional(:address).hash do
    required(:street).filled(:string)
    required(:zipcode).filled(:string)
    required(:city).filled(:string)
  end
end

UserSchema.info
# {
#   :keys=> {
#     :email=>{
#       :required=>true,
#       :type=>"string"
#     },
#     :age=>{
#       :required=>false,
#       :type=>"integer"
#      },
#     :address=>{
#       :type=>"hash",
#       :required=>false,
#       :keys=>{
#         :street=>{
#           :required=>true,
#           :type=>"string"
#         },
#         :zipcode=>{
#           :required=>true,
#           :type=>"string"
#         },
#         :city=>{
#           :required=>true,
#           :type=>"string"
#         }
#       }
#     }
#   }
# }

Summary

There's way more in the changelog so please check it out and if you're having any issues when upgrading, please do let us know.

Big thanks go to Rob Hanlon and the rest of the contributors who helped with this release!

Last but not least: dry-validation 1.5.0 was released too, which upgrades its own dependency on dry-schema to 1.5.0 and adds a couple of new features.

Please upgrade and enjoy using dry-schema and dry-validation!

Introducing dry-effects

2019-10-03T00:00:00+00:00

Today we're introducing another gem and supercharging our toolset: say hello to dry-effects!

dry-effects is an implementation of algebraic effects in Ruby. Sound scary? Fear not! After a few examples, it'll feel very natural and compelling.

Struggling with side effects

Writing purely functional code can be an attractive idea; it makes your code robust, testable, ... and useless! Indeed, if code doesn't perform any side effects, such as reading/writing data to the disc or network communications, the only thing it actually does is heating the CPU. On the other hand, side effects remove determinism from the code, making testing challenging. Here come algebraic effects, the underlying theory powering dry-effects.

Understanding effects

There are two main parts to effects and effectful systems:

Replace side effects with effects. These two are not the same, they are not even similar. Side effects are by definition not expected by the calling code. One cannot say if there are side effects judging by the interface. On the contrary, effects are explicitly included in interfaces; they are expected.
Running code with effects requires handling. This must be done explicitly, so that effects don't propagate straight to the outside world.

These two things combined give you full control over effects in your application.

Taming effects with dry-effects

dry-effects uses mixins for making (or introducing) and handling (or eliminating) effects.

For example, this code uses the effect of getting the current time:

class CreateSubscription
  include Dry::Effects.CurrentTime

  def call(values)
    subscription_repo.create(values.merge(start_at: current_time))
  end
end

To run it, there must be a handler:

# Rack middleware is a perfect example of a place
# where effects can be handled.

class WithCurrentTime
  include Dry::Effects::Handler.CurrentTime

  def initialize(app)
    @app = app
  end

  def call(env)
    with_current_time { @app.(env) }
  end
end

So how is this better than Time.now? You get testable code for free. An RSpec example:

include Dry::Effects::Handler.CurrentTime

subject(:create_subscription) { CreateSubscription.new }

example "creating subscription on New Year's Eve" do
  with_current_time(proc { Time.new(2019, 12, 31, 12) }) do
    create_subscription.(...)
  end
end

Why would you use dry-effects for this instead of specialized solutions? Because it provides a universal interface to all effects, it's not limited to Ruby. For instance, getting the current time in React would look very similar:

const CurrentTime = () => {
  const currentTime = useCurrentTime();

  return (
    <div className="current-time">
      {currentTime.getHours()}:{currentTime.getMinutes()}
    </div>
  );
};

Yes, React relies on algebraic effects under the hood; maybe you already use them!

What else?

dry-effects v0.1 is already out and comes with quite a few effects supported out of the box. Some of them are “classic” and some are experimental, 17 in total.

Here are some:

Accessing current time
Providing context
Sharing state
Providing environment (as opposed to accessing and manipulating ENV)
Caching
Locking
Deferred and parallel code execution

One of the most compelling examples is dependency injection:

class CreateUser
  include Dry::Effects.Resolve(:user_repo)

  def call(values)
    user_repo.create(values)
  end
end

Here CreateUser is not linked to a dependency resolution implementation in any way. To provide the dependency, add a handler:

include Dry::Effects::Handler.Resolve

subject(:create_user) { CreateUser.new }

let(:user_repo) { double(:user_repo, create: ...) }

example 'creating a user' do
  provide(user_repo: user_repo) do
    create_user.(...)
  end
end

You can provide multiple dependencies:

provide(user_repo: user_repo, post_repo: post_repo) { ... }

Handlers are also composable, you can nest them:

provide(user_repo: user_repo) { provide(post_repo: post_repo) { ... } }

This is not limited to handlers of the same type:

provide(user_repo: user_repo) { with_current_time { ... } }

Generally, effects and handlers work just the way you would expect; this is the most appealing thing about them (judging from experience!).

Why dry-effects?

These are early days for algebraic effects. We believe they have a prominent future. The concept comes from the functional world and, since dry-rb heavily leans toward functional programming, it perfectly fits our ecosystem. There is no existing production-ready library for Ruby, that's why we've built our own.

This post has mostly demonstrated using effects in application code, but they can be as easily used in libraries, providing a new level of flexibility to the users. This part is yet to be explored.

With infinite power comes infinite responsibility

Algebraic effects are quite new, and as a community we have zero to little experience in using them. They may help you with writing clean, decoupled, and testable code, but they can also turn your app into unmaintainable mess, drop your database, and burn your house, so please be careful!

As we gather experience, together we'll figure out what's good and what's bad. So please try it, and share what you learn with others!

Dive in!

The first version of dry-effects is already on RubyGems.org, go grab it. We have docs for most effects and specs for all of them. As always, share your experience and ask your questions in our chat and at our forum.

dry-validation 1.0.0 released

2019-06-10T00:00:00+00:00

We're very happy to announce the release of dry-validation 1.0.0!

This is a big release: it includes a rewritten schema DSL, released as dry-schema, and a completely redesigned validation system. If you're interested to know the reasoning behind these changes, please refer to the "Plans for dry-validation + dry-schema (a new gem!)" post on our forum. Yes, it's from February 2017, this took a while, but it was totally worth the wait. Continue reading to see why.

New-old schema DSL

The schema DSL has been rewritten from scratch and not only did it fix dozens of known issues, it also introduced a couple of new features. That said, some complex features that didn't fit anymore were removed. In dry-validation 1.0.0, the schema DSL is delegated to dry-schema and you can still define 3 types of schemas:

schema - a plain schema that does not perform any coercions
params - a schema with coercions optimized for HTTP params
json - a schema with coercions optimized for JSON

The syntax for defining keys with validations is almost identicial to the one you know from previous versions of dry-validation. However, there's a big conceptual difference between those earlier versions and how dry-validation 1.0.0 is intended to be used now.

Contracts with rules

We have a completely new concept called Contract that allows you to define a schema and domain validation rules. The new rule system is completely decoupled from the schema validation, but it's still type-safe, which means that when you define a rule you can assume the types of the values are correct. This removes the need to perform any additional checks in validation rules and you are going to love this.

Here's a simple example where we define a contract for a new user data:

class NewUserContract < Dry::Validation::Contract
  params do
    required(:name).filled(:string)
    required(:age).filled(:integer)
  end

  rule(:name) do
    key.failure("is too short") if value.length < 3
  end

  rule(:age) do
    key.failure("you must be at least 13 years old") if value < 13
  end
end

If you are familiar with the old version, your immediate reaction might be "oh that's more code, why not just define these checks in the schema?" That's a good question to ask! It's still possible to use all the known predicates so, technically speaking, you could perform these checks via the schema but it's not recommended. Starting with 1.0.0, we're moving to a new way of thinking about validations by splitting them into basic structural and type checks handled by schemas and domain validations handled by contracts and their rules. This is a good way of separating concerns to make your code cleaner, simpler and more reusable.

If the amount of code you need to write is a concern, don't worry, because we have a new macro system in place to DRY things up.

Improved messages

One of the biggest limitations in the previous version was the way you could provide custom error messages. Starting from 1.0.0, you have complete control over this process. You can now:

Provide a message as a plain string, e.g. key.failure("oops this is wrong")
Provide a message using a locale identifier, e.g. key.failure(:invalid)
Pass extra data when using locales, e.g. key.failure(:invalid, more: "info", goes: "here")
Pass additional metadata in addition to the message text, e.g. key.failure(text: "oops this is wrong", code: :red)

On top of this, we still support localized backends using plain YAML or I18n gem.

Base messages

Another nice improvement is support for base messages. This means you can provide a message that will be associated with the whole input, instead of a specific key.

Here's an example:

class EventContract < Dry::Validation::Contract
  option :today, default: Date.method(:today)

  params do
    required(:start_date).value(:date)
    required(:end_date).value(:date)
  end

  rule do
    if today.saturday? || today.sunday?
      base.failure('creating events is allowed only on weekdays')
    end
  end
end

Now we can access base errors (assuming it's a weekend):

contract = EventContract.new

contract.call(start_date: Date.today+1, end_date: Date.today+2).errors
# #<Dry::Validation::MessageSet
#   messages=[
#     #<Dry::Validation::Message text="creating events is allowed only on weekdays" path=[nil] meta={}>
#   ]
#   options={}
# >

Macros

As mentioned above, you can use the new macro system to reduce code duplication. Currently, there's only one built-in macro, called :acceptance, but we'll be adding more.

Here's an example how you could use the :acceptance macro:

class NewUserContract < Dry::Validation::Contract
  schema do
    required(:email).filled(:string)
    required(:terms).filled(:bool)
  end

  rule(:terms).validate(:acceptance)
end

contract = NewUserContract.new

contract.call(email: "jane@doe.org", terms: "false").errors.to_h
# => {:terms=>["must accept terms"]}

contract.call(email: "jane@doe.org", terms: "true").errors.to_h
# => {}

Defining your own macros is very simple and you're encouraged to do so. Let's say we want to encapsulate checking if a string is of a minimum length, here's how you could do it with a macro:

class ApplicationContract < Dry::Validation::Contract
  register_macro(:min_length) do |macro:|
    key.failure("is too short") if value.length < macro.args[0]
  end
end

Now we can use our :min_length macro in other contract classes:

class NewUserContract < ApplicationContract
  schema do
    required(:email).filled(:string)
    required(:password).filled(:string)
  end

  rule(:password).validate(min_length: 12)
end

contract = NewUserContract.new

contract.call(email: "jane@doe.org", terms: "false", password: "secret").errors.to_h
# => {:password=>["is too short"]}

The posibilities are endless and I'm sure we'll soon have a nice collection of macros either built into the main gem or provided as an extension.

Improved validation of array elements

Validating array elements can be tricky business, but it's become nice and simple in dry-validation 1.0.0. It works using the same mechanism as other value types - an array element will not be checked by a rule unless the corresponding schema checks passed.

To validate array elements, use Rule#each:

class NewSongContract < Dry::Validation::Contract
  params do
    required(:artist).filled(:string)
    required(:title).filled(:string)
    required(:tags).array(:string)
  end

  rule(:tags).each do
    key.failure("tag length must be at least 3") if value.length < 3
  end
end

Now let's see it in action:

contract = NewSongContract.new

contract.(artist: "Queen", title: "Bohemian Rhapsody", tags: ["rock", 123, "ab"]).errors.to_h
# => {:tags=>{1=>["must be a string"], 2=>["tag length must be at least 3"]}

Notice that our rule did not crash on 123 value even though Integer does not implement length - instead, we got a nice error that the second element must be a string. This is how type safety in rules work.

Upgrading from dry-validation 0.x

Please refer to the comprehensive guide "dry-rb 1.0: upgrading validations, types and schemas," written by Igor Morozov. He's done a terrific job explaining the process.

Additionally, check out:

dry-validation 1.0.0 CHANGELOG
dry-types 1.0.0 CHANGELOG
dry-schema 1.0.0 CHANGELOGs

If you need help with upgrading, please do not hesitate to ask questions either on our discussion forum or community chat.

Thank you

Thank you to all the contributors and early adopters who helped us shape dry-validation. This has been a big effort and we're very happy with the results. Please check it out and let us know what you think!