Skip to content
Alex Shpilkin edited this page May 30, 2022 · 111 revisions

Those who have installed and are using the Nix package manager often use its command-line tool nix-shell to enter into a Nix shell, which is just a Bash session with environment variables set up for a project. We can configure Direnv to load environment variables from the Nix shell for a project. Nix has support for lots of different programming languages, so once we get Direnv integrated with Nix, we can do all of our language-specific configuration with Nix. Our .envrc can be short and simple.

Consider reading some of the official Nix documentation if you're new to Nix and the benefits of using it. That will help you understand the rest of this wiki page better.

Summary of options

It would be nice if there was only one recommended option for integrating Direnv with Nix. Instead, over the years, there's been a lot of parallel efforts to implement Nix integration for Direnv. We have a lot of options do a lot of the same things, but each slightly different. Using one of these approaches could save you time investing in your own effort.

Some factors to consider

Below is a table to highlight differences with respect to the features discussed below.

Feature Standard Library Nixify Lorri Lorelei Nix-direnv Hand-rolled
Avoids naming conflict with use_nix NA
Caches nix-shell invocation
Prevents Nix GC
Saves last previous caches
Can autodetect files to watch
Invalidates cache by modification time
Invalidates cache by content change
Background process pre-calculating
Supports Nix Flakes

Before we begin, note that most of these extensions can be installed and used concurrently. Nix-direnv and Nixify both overwrite the name use_nix used by the standard library, so these three options conflict and can't be installed concurrently. However, other combinations work fine, for example you can use Nix-direnv for its use_flake function and Lorelei for its use_nix_gcrooted function.

Direnv's standard library comes with limited support for Nix integration. The provided use_nix function can build a Direnv environment from a Nix shell, but there's a few problems we encounter when using this basic support:

  • The nix-shell invocation needed to set up environment variables with Nix can sometimes be inconveniently slow, which can be annoying when entering, exiting, and re-entering a Direnv environment that hasn't changed.
  • Nix might garbage collect a package that is needed by one of our Direnv environments.

To solve these problems, most options cache the calculated Direnv environment in a file, generally in a project's .direnv folder. Then when we enter the directory, we can load these cached variables to avoid a pause for Nix evaluation.

The various options also prevent Nix from garbage collecting the environment by creating a GC root. Nix provides a special directory, /nix/var/nix/gcroots, for putting symlinks to packages we want to prevent Nix from garbage collecting. Transforming a Nix environment into a Direnv environment such that Nix GC roots can be managed is definitely done differently by each project. Lorri's approach seems to be the most robust, a hack that seems to work correctly. Lorelei links to code in Lorri's codebase as a library to benefit from this approach without dual maintaining its implementation. As for the rest, well, to understand the state of things, someone should take a look at each option's issue tracker to see if there's any known problems with managing GC roots. Maybe you have some time?

We may also want to prevent garbage collection for the packages referenced by a number of older caches as well. This way, we may prevent Nix from rebuilding packages if we make a cache-invalidating change, run a Nix garbage collection, and then revert our change. The level of support for this varies between options.

The options also differ in how they invalidate the cached direnv environment:

  • Lorri and Loreli autodetect which files to watch. Other options require specifying the files explicitly by the user.
  • Some options invalidate when the modification time of the file is newer than that of the cache.
  • Some options additionally cache hashes of watched files' content, and invalidate the cache when watched files have new content.

These approaches have different ergonomics, particularly if the calculation of environment variables draws from any more than your watched files. It can be easier to touch a watched file to invalidate a cache than make a whitespace change.

Lorri is probably the most advanced method. It runs a daemon that keeps track of all your Nix projects and caches Direnv environments in the background. With other methods to cache Direnv environments, we can't avoid the first time pause for evaluation of the cache. We don't face another pause until a watched file changes. Lorri's background process can help us avoid having to experiencing any of these pauses at all. However, this convenience comes with at the cost of an additional layer of complexity.

Another option for speeding things up is the evaluation cache of Nix Flakes. Flakes are only supported by nix-direnv.

Quickstart Guides

Following the links to each third-party project should get you to each project's official documentation for installation and/or a tutorial for usage. What follows below is supplemental guidance.

Setting up a project to use Nix

Nix supports a variety of languages by providing a lot of library code you can use in the Nixpkgs GitHub repository. If you want a Nix expression specialized to a particular language ecosystem you should learn more by reading the Nixpkgs Manual. That should help you figure out how to write a default.nix and/or shell.nix for your project.

Once you have a default.nix or shell.nix authored, you can run nix-shell to enter into a development environment for your project. The next step would be to integrate Direnv with your Nix-configured project using any of the methods detailed in this wiki page.

If you just want to put a few packages on your PATH using Nix in a way agnostic to any particular language ecosystem, you can use the following for your project's shell.nix:

with import <nixpkgs> {};
mkShell {
  nativeBuildInputs = [
    # dependencies you want available in your shell
  ];
}

Nix shell integration with the Direnv standard library

As discussed in the manpage for the Direnv standard library, if you have a project with either a default.nix or shell.nix file ready to be called with a simple nix-shell invocation, all you need to do is create a .envrc file with the following simple entry:

use nix

Then follow the typical process of calling direnv allow, and you should experience a pause as your Direnv environment is evaluated. Then, when you edit your shell.nix or default.nix files, this environment should be recomputed (they are set up as watched files).

Note, because Nix is generally managing our environment comprehensively, it knows about all the environment variables needed by a project for different programming languages. Consequently, you shouldn't need any layout calls in your .envrc files.

We can use a simple use nix in the .envrc of any project set up with a shell.nix or default.nix and get the appropriate environment set up. Even better, we get that environment in our current shell as maintained by Direnv instead of a subshell.

Nix shell integration with Lorri

Please see the README file for the Lorri source code, which has the best documentation for installation and getting started.

Nix shell integration with Lorelei

Please see the README file for the Lorelei source code, which has the best documentation for installation and getting started.

Nix shell or flakes integration with Nix-direnv

Please see the README file for the Nix-direnv source code, which has the best documentation for installation and getting started.

Nix shell integration with Nixify

Nixify was designed to create projects from scratch via a project scaffold. If you have a directory that doesn't have a default.nix, shell.nix, or .envrc file, you can set up that directory as a Nix project configured for Direnv with the following:

nix-env -f https://github.com/kalbasit/nur-packages/archive/master.tar.gz -iA nixify
nixify .    # this should give you a shell.nix, .envrc and a .nixpkgs-version.json
direnv allow

This will copy some template code into your project. Your .envrc will be a copy of Nixify's maintained template for .envrc. This means that your projects will have a somewhat busy .envrc file. You have the option of moving all the functions of this code code into your ~/.config/direnv/lib/nix.sh so you can use it as a library. If you do so, be careful not to to accidentally to also copy the use_nix invocation at the bottom. That use_nix call will then be what you'll have left in your project's .envrc file.

Some troubleshooting guidance:

  • You will find that shell hooks are not executed. The workaround is to just call shellHook in your .envrc file.
  • Rust's cargo may fail to download crates due to SSL errors. A workaround is to comment the use_nix part of direnvrc when adding new crates in Cargo.toml.

Using a per-project Nix profile

The other methods in this wiki page build a Direnv environment from either a project's Nix shell or Nix flake. These are static specifications of the environment specified by a shell.nix, default.nix, or flake.nix file. Instead, we can dynamically maintain a set of installed programs local to our project's Direnv environment.

A Nix profile is a directory that Nix installs programs into using Nix's nix-env tool. The profile has the typical structure of Unix systems following the Filesystem Hierarchy Standard (executable files in $NIX_PROFILE/bin, etc). We can set up environment variables like NIX_PROFILE in our Direnv environment so that nix-env installs packages to a project-local .direnv/nix profile (instead of the default ~/.nix-profile). We can then use Direnv's load_prefex function to find everything installed to this project-local profile.

To support this, put the following code in ~/.config/direnv/lib/nix-profile.sh:

use_nix_profile() {
  source "$HOME/.nix-profile/etc/profile.d/nix.sh"
  export NIX_PATH=/nix/var/nix/profiles/per-user/$USER/channels
  export NIX_PROFILE="$(direnv_layout_dir)/nix"
  load_prefix "$NIX_PROFILE"
}

Your project's .envrc file will have the following single line:

use nix_profile

From within the Direnv environment, whenever nix-env --install is invoked, all the dependencies are installed only under the project-local profile. Note that user-local dependencies previously on our PATH before entering the Direnv environment will still be available. Also, note that Nix environmental variables suited for development such as NIX_BINTOOLS, NIX_LDFLAGS NIX_CFLAGS_COMPILE (used by Nix's C compiler's wrapper) are not set by this approach. These variables are made available with an approach drawing variables from either a Nix shell or flake.

Hand-rolled Nix shell integration

Before third-party projects had matured, people were just copying and pasting snippets of shell functions into their ~/.config/direnv configuration as well as their project's .envrc files. Here's one example.

Example ~/.config/direnv/lib/nix-shell.sh
# Usage: use nix_shell
#
# To force the reload the derivation, run `touch shell.nix`
use_nix_shell() {
  local shellfile=shell.nix
  local wd=$PWD/.direnv/nix
  local drvfile=$wd/shell.drv
  local outfile=$ws/result

  # same heuristic as nix-shell
  if [[ ! -f $shellfile ]]; then
    shellfile=default.nix
  fi

  if [[ ! -f $shellfile ]]; then
    fail "use nix_shell: shell.nix or default.nix not found in the folder"
  fi

  if [[ -f $drvfile && $(stat -c %Y "$shellfile") -gt $(stat -c %Y "$drvfile") ]]; then
    log_status "use nix_shell: removing stale drv"
    rm "$drvfile"
  fi

  if [[ ! -f $drvfile ]]; then
    mkdir -p "$wd"
    # instantiate the derivation like it was in a nix-shell
    IN_NIX_SHELL=1 nix-instantiate \
      --show-trace \
      --add-root "$drvfile" --indirect \
      "$shellfile" >/dev/null
  fi

  direnv_load nix-shell "$drvfile" --run "$(join_args "$direnv" dump)"
  watch_file "$shellfile"
}

This code can seem very small and easy to maintain. But be aware that these solutions often have limitations. As you address these limitations, you very well can end up writing code similar to what's already in Lorri/Lorelei, Nix-direnv, or Nixify.

Hand-rolled Nix flakes integration

Nix-direnv is putting a good amount of effort into Nix flakes support (it is currently the only third-party maintained project with support for flakes).

In this section, we'll show how you can support flakes yourself with some streamlined code. Note that this approach won't protect Nix from garbage collecting packages referenced by our Direnv environments. If you want that feature, you should consider using Nix-direnv instead.

Flakes are an experimental feature for Nix, not yet officially released, and available as a special package nixpkgs.nixFlakes as a drop-in replacement for nixpkgs.nix (the normal package we use to install Nix). A compelling benefit of flakes are that they implicitly cache the evaluation, and thus load faster than nix-shell. This means we don't have to worry about caching in our Direnv/Nix integrations when using flakes.

If you have a project, you add support for flakes to it with a flake.nix file.

Example project-level flake.nix file
{
  description = "Flake description";
  edition = 201909;
  outputs = { self, nixpkgs }: {
    # setup the devShell for x86_64-linux.
    devShell.x86_64-linux =
      with nixpkgs.legacyPackages.x86_64-linux;
      mkShell {
        buildInputs = [
          # add your dependencies here
        ];

        shellHook = ''
          # add extension
        '';
      };
  };
}

Once our project supports flakes, we can integrate Direnv with the project easily with the following .envrc file:

# reload when these files change
watch_file flake.nix
watch_file flake.lock
# load the flake devShell
eval "$(nix print-dev-env)"

However, as mentioned the trade-off for this simplicity is the lack of protection from a Nix garbage collection (which Nix-direnv provides).