Run a script with systemd timers on Nixos - web-scraping

Question

I have a small shellscript scrape.sh that scrapes a website and puts the resulting data into a new directory:
website='test.com'
dir_name="datev_data/$(date -u +'%Y-%m-%dT%H:%M:%S')"
mkdir $dirname
wget --directory-prefix="$dir_name" "$website"
(I don't realy care where the data ends up as long as it is gets a new directory every time and I can get to the data. I therefore put it into my home directory /home/kaligule for now.)
Running this script by hand works fine, so now I want to run this script every hour on my nixos-server. Therefore I put the following into my config there (Inspired by this post):
systemd.services.test_systemd_timers = {
serviceConfig.Type = "oneshot";
script = ''
echo "Will start scraping now."
{pkgs.bash}/bin/bash /home/kaligule/scrape.sh
echo "Done scraping."
'';
};
systemd.timers.test_systemd_timers = {
wantedBy = [ "timers.target" ];
partOf = [ "test_systemd_timers.service" ];
timerConfig.OnCalendar = [ "*-*-* *:00:00" ];
};
Then I test it out:
sudo nixos-rebuild switch # everything is fine
sudo systemctl start test_systemd_timers # run it immediatelly for testing
I get:
Job for test_systemd_timers.service failed because the control process exited with error code.
See "systemctl status test_systemd_timers.service" and "journalctl -xe" for details.
The first suggested command gives me this:
● test_systemd_timers.service
Loaded: loaded (/nix/store/f8348svxpnn6qx08adrv5s7ksc2zy1sk-unit-test_systemd_timers.service/test_systemd_timers.service; linked; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2021-04-02 14:50:02 CEST; 2min 36s ago
TriggeredBy: ● test_systemd_timers.timer
Process: 5686 ExecStart=/nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start (code=exited, status=127)
Main PID: 5686 (code=exited, status=127)
IP: 0B in, 0B out
CPU: 11ms
Apr 02 14:50:02 regulus systemd[1]: Starting test_systemd_timers.service...
Apr 02 14:50:02 regulus test_systemd_timers-start[5686]: Will start scraping now.
Apr 02 14:50:02 regulus test_systemd_timers-start[5687]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
Apr 02 14:50:02 regulus systemd[1]: Failed to start test_systemd_timers.service.
The second suggested command gives me:
Apr 02 14:54:42 regulus systemd[1]: Starting test_systemd_timers.service...
░░ Subject: A start job for unit test_systemd_timers.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit test_systemd_timers.service has begun execution.
░░
░░ The job identifier is 34454.
Apr 02 14:54:42 regulus test_systemd_timers-start[5734]: Will start scraping now.
Apr 02 14:54:42 regulus test_systemd_timers-start[5735]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit test_systemd_timers.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 127.
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit test_systemd_timers.service has entered the 'failed' state with result 'exit-code'.
Apr 02 14:54:42 regulus systemd[1]: Failed to start test_systemd_timers.service.
░░ Subject: A start job for unit test_systemd_timers.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit test_systemd_timers.service has finished with a failure.
░░
░░ The job identifier is 34454 and the job result is failed.
Apr 02 14:54:42 regulus sudo[5731]: pam_unix(sudo:session): session closed for user root
So because I got the Will start scraping now. in the logs I think the job started but wasn't able to run the script. My questions are:
where (and with which permissions) should I put my script?
how should I addapt my config so that the script runs as I described?
(Of course I am thankfull for every feedback to my approach.)

Answer 1

Your problem is that, in your script, {pkgs.bash} should be ${pkgs.bash}; without the $, you won't get variable interpolation.

Answer 2

On NixOS, systemd services run with a very minimal default environment. You can add the missing packages in systemd.services.<name>.path
systemd.services.test_systemd_timers = {
serviceConfig.Type = "oneshot";
path = [
pkgs.wget
pkgs.gawk
pkgs.jq
];
script = ''
echo "Will start scraping now."
${pkgs.bash}/bin/bash /home/kaligule/scrape.sh
echo "Done scraping."
'';
};