this post was submitted on 08 Mar 2024
25 points (96.3% liked)

Nix / NixOS

1765 readers
1 users here now

Main links

Videos

founded 1 year ago
MODERATORS
 

How easy would it be to flakify the Lemmy repo to add a fourth build option for those of us in the Nix world?

I had the (perhaps foolishly ambitious) idea of creating a rolling fork of Lemmy with the intent of modifying the codebase for use in an open source pub sub implementation of retail inventory. But I have to get standard Lemmy working first...and I like to use Nix for everything I do in the dev world (where feasible).

So, I forked the repo and was immediately brought into dev environment hell.

They only offer a choice between:

A.) Docker B.) Ansible C.) Building from scratch.

Two hours of fighting with the scratch build instructions and I eventually had to admit defeat due to some vague dependencies (and general malaise). Though I have completely flakified my Purescript and Haskell dev environments, I have found Rust to be a lot more challenging even on simple projects.

Anyway, I decided to come here to ask: **How easy would it be to flakify the Lemmy repo to add a fourth build option for those of us in the Nix world? **

Can I reference the build instructions from nixpkgs to get close to my intended goal? I need all of the help I can get. Be as pedantic or defeatist as you will. I currently have no skin in this game and merely want to help the Lemmy devs welcome people that are more nixy like myself (if nothing else).


edit: here's a naive attempt. I haven't tested anything other than it being a valid flake. Here's the link to my rolling flakification fork.

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
    systems.url = "github:nix-systems/default";

    # Dev tools
    treefmt-nix.url = "github:numtide/treefmt-nix";
  };

  outputs = inputs:
    inputs.flake-parts.lib.mkFlake { inherit inputs; } {
      systems = import inputs.systems;
      imports = [
        inputs.treefmt-nix.flakeModule
      ];
      perSystem = { config, self', pkgs, lib, system, ... }:
        let
          cargoToml = builtins.fromTOML (builtins.readFile ./Cargo.toml);
          nonRustDeps = [
            pkgs.libiconv
            # Derived from Documentation on Dev installation for Linux
            pkgs.cacert
            pkgs.postgresql
            pkgs.openssl
            pkgs.pkg-config
            pkgs.libpqxx
          ];
          rust-toolchain = pkgs.symlinkJoin {
            name = "rust-toolchain";
            paths = [ pkgs.rustc pkgs.cargo pkgs.cargo-watch pkgs.rust-analyzer pkgs.rustPlatform.rustcSrc pkgs.cacert pkgs.postgresql pkgs.openssl pkgs.pkg-config pkgs.libpqxx ];
          };
        in
        {
          # Rust package
          packages.default = pkgs.rustPlatform.buildRustPackage {
            inherit (cargoToml.package) name version;
            src = ./.;
            cargoLock.lockFile = ./Cargo.lock;
          };

          # Rust dev environment
          devShells.default = pkgs.mkShell {
            inputsFrom = [
              config.treefmt.build.devShell
            ];
            shellHook = ''
              # For rust-analyzer 'hover' tooltips to work.
              export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc}
              export CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true

              echo
              echo "๐ŸŽ๐ŸŽ Run 'just <recipe>' to get started"
              just
            '';
            buildInputs = nonRustDeps;
            nativeBuildInputs = with pkgs; [
              just
              rust-toolchain
            ];
            RUST_BACKTRACE = 1;
          };

          # Add your auto-formatters here.
          # cf. https://numtide.github.io/treefmt/
          treefmt.config = {
            projectRootFile = "flake.nix";
            programs = {
              nixpkgs-fmt.enable = true;
              rustfmt.enable = true;
            };
          };
        };
    };
}

and here's the lemmy-ui flake (which, judging by past forays into tsc with nix last summer, should be a lot easier to create and work with than the Rust library):

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    systems.url = "github:nix-systems/default";
  };

  outputs = {
    systems,
    nixpkgs,
    ...
  } @ inputs: let
    eachSystem = f:
      nixpkgs.lib.genAttrs (import systems) (
        system:
          f nixpkgs.legacyPackages.${system}
      );
  in {
    devShells = eachSystem (pkgs: {
      default = pkgs.mkShell {
        buildInputs = [
          pkgs.nodejs
          # You can set the major version of Node.js to a specific one instead
          # of the default version
          # pkgs.nodejs-19_x

          # You can choose pnpm, yarn, or none (npm).
          pkgs.nodePackages.pnpm
          # pkgs.yarn

          pkgs.nodePackages.typescript
          pkgs.nodePackages.typescript-language-server
        ];
      };
    });
  };
}
top 10 comments
sorted by: hot top controversial new old
[โ€“] pr06lefs@lemmy.ml 6 points 8 months ago* (last edited 8 months ago) (2 children)

Are you just after a dev environment? Or are you looking to make lemmy a nix module, so you could make it a service on a nixos machine?

I made some flakes that might be a starting point for nix develop.

https://github.com/bburdette/lemmy/tree/flake.nix

https://github.com/bburdette/lemmy-ui/tree/flake.nix

Both repos require git submodule init and git submodule update .

After nix develop, I was able to build the rust with cargo build.

The lemmy-ui one you have to npm install pnpm so it makes a node_modules dir, then you can exit and reenter nix develop to get it to set the path. I used nodejs_20 but you might not need to. When I run it with pnpm dev, it complains about missing translation files. Some files got installed with a submodule, but maybe they're in the wrong place? dunno, stopped there.

Beyond that, I would try using their docker compose thing. Probably that will use the compiled typescript and/or rust code so you can work in a normal env and have those containers use your code. Haven't tried it.

[โ€“] demesisx@infosec.pub 10 points 8 months ago

This looks great. Iโ€™ll try to incorporate some of what you did in a little while. Thanks!

[โ€“] einsteinx2@programming.dev 1 points 8 months ago

Heads up with modern git you can now just use git clone --recursive and it will clone all sub modules automatically.

[โ€“] onlinepersona@programming.dev 6 points 8 months ago* (last edited 8 months ago) (1 children)

Hmm, do you need to contribute? I think there should be a way to have a shell.nix that uses the lemmy-server package as input.

Damn, the nixpkgs manual is horrific. Ctrl+F "mkShell" requires hitting "next result" for ages to finally get to the "documentation". Sheesh.

Anyway, have you tried putting this into the repo

shell.nix

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
  inputsFrom = [ pkgs.lemmy-server ];
}

then executing nix-shell ?

If so, you could try to make a default.nix from the lemmy-server package and open a PR with it. Maybe they'll accept it.

Edit: I see you wrote "flakes". Can't help you with that. I find the documentation on that even worse than normal nix documentation since it's cobbled together from so many different sources.

CC BY-NC-SA 4.0

[โ€“] demesisx@infosec.pub 13 points 8 months ago (1 children)

Iโ€™ll definitely give that a try.

I was hoping to spell out everything in the flake. So, maybe Iโ€™d use the lemmy-server nix build instructions to help me get a proper flake working. I definitely want to contribute because, at least for me, it kept me from joining their dev efforts in any meaningful way.

Good luck and have fun :)

CC BY-NC-SA 4.0

[โ€“] ours@jlai.lu 1 points 7 months ago (1 children)

If you're using flakes and since lemmy is packaged in nixpkgs, you can spawn a dev shell with:

nix develop nixpkg#lemmy-server
[โ€“] demesisx@infosec.pub 11 points 7 months ago* (last edited 7 months ago) (1 children)

Thanks for the advice.

Iโ€™m working on getting my own fork of Lemmy therefore Iโ€™m attempting to repackage it independently of Nixpkgs so I can get a flake in their actual repo that allows devs to spin up a Lemmy dev environment. I want my dev environment to reflect the changes being made on Lemmy in that dev environment when I make them. Unless Iโ€™m missing something, with your suggestion, Iโ€™d have no way to test a major change to Lemmyโ€™s codebase in my fork (as I intend to).

[โ€“] ours@jlai.lu 1 points 7 months ago (1 children)

This gives you a build environment for lemmy, see the example. You can clone upstream and use that shell to build lemmy. Quite convenient to test quick changes. However, that won't help distributing your fork.

[โ€“] demesisx@infosec.pub 11 points 7 months ago

Thanks for the pointers.