Fail to restream with nginx-rtmp-module - nginx

I'm unable to start restream using command exec. Also I tried exec_pull.
It didn't help. My target is to restream rtmp://ktv.s4c.link/live/livestream url to my local nginx, to rtmp://localhost:1935/hls.
/tmp/logs/ffmpeg.log is empty.
I guess that exec not even called, but why?
rtmp {
server {
listen 1935;
chunk_size 4096;
application live {
allow play all;
live on;
record off;
#pull rtmp://ktv.s4c.link/live/livestream;
exec ffmpeg -i rtmp://localhost:1935/live -f flv rtmp://localhost:1935/hls 2>>/tmp/logs/ffmpeg.log;
}
application hls {
allow play all;
live on;
record off;
}
}
I'm using nginx-1.12.0.
I followed by this tutorial, watched this

So, first of all, be aware of that Nginx run all your exec commands as nobody user. And you have to test it in command line like this:
sudo -u nobody ffmpeg -i rtmp://ktv.s4c.link/live/livestream -f flv rtmp://localhost:1935/hls
And then if this command line works - print to nginx.conf.
Secondly, use the full path. It's very important, because a child process than Nginx runs has different environment compared with your current user environment.
sudo -u nobody /usr/bin/ffmpeg -i rtmp://ktv.s4c.link/live/livestream -f flv rtmp://localhost:1935/hls
Thirdly, my mistake was that I used FFmpeg from the repository and that FFmpeg has 2.8.11-0ubuntu0.16.04.1 version. Related to this answer
FFmpeg can be compiled with librtmp, that's why FFmpeg can require live=1 and quotes("). So the final command that I copied to nginx.conf was:
sudo -u nobody /usr/bin/ffmpeg -i "rtmp://ktv.s4c.link/live/livestream live=1" -f flv rtmp://localhost:1935/hls
My final nginx.conf
rtmp {
server {
listen 1935;
chunk_size 4096;
application live {
live on;
record off;
pull rtmp://ktv.s4c.link/live/livestream;
exec_play /usr/bin/ffmpeg -i "rtmp://ktv.s4c.link/live/livestream live=1" -f flv rtmp://localhost:1935/hls 2>>/tmp/ffmpeg-$name.log;
}
application hls {
live on;
record off;
}
}
As you can mention I added FFmpeg logs 2>>/tmp/ffmpeg-$name.log.

Related

Using nginx-rtmp-module to save rtmp stream to mp4 instead of flv

I am trying to save an incoming rtmp stream on my server as an mp4 file. I am using the nginx-rtmp module and changing the container from flv to mp4 on the exec_record_done directive. This is done using ffmpeg as follows
ffmpeg -i input.flv output.mp4
I am sending h264 video streams and aac audio streams (in an flv container), so as to keep it compatible even with mp4
I would like to know if it is possible to directly save the incoming stream as an mp4 and avoid the transcoding at the end of each session
I am guessing this will help avoid the abrupt CPU spikes that I see when a recording is done
nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 1024;
}
rtmp {
server {
listen 1935;
chunk_size 4096;
allow publish all;
allow play all;
application live {
live on;
record all;
record_suffix .flv;
record_path /tmp/videos/;
record_unique on;
exec_record_done ffmpeg -y -i $path /home/ubuntu/videos/$basename.mp4;
}
}
}
I thought this would be simple enough because the underlying streams remain the same. Any help is appreciated :)
It's because your FFmpeg command is incorrect, try this:
exec_record_done ffmpeg -i $path -c copy /home/ubuntu/videos/$basename.mp4;
Remember to add the -c copy to tell FFmpeg convert FLV to MP4 without transcoding.
Another solution is copying the HLS files and covert HLS to MP4 by yourself.

NGinx RTMP live stream text overlay and push to multiple

I have been banging my head against this wall for a long time. Hoping you all can get me over.
I have a live stream coming from an IP Camera to my computer.
Nginx publishes to YouTube and to an FFmpeg stream that takes a frame every minute to use for a static webcam image.
Here is the code with the exec_push that I've tried to use with no success. The YouTube stream and frame capture work fine. I have FFmpeg installed with freetype. This is all on MacOS X 10.15.4 Catalina with home-brew FFmpeg --HEAD installed.
Update: I should also say I have tried outputting the overlay using command line FFmpeg and it works great with this command:
/usr/local/bin/ffmpeg -i rtmp://localhost:1935/live/68.1. -vf drawtext="fontfile=/System/Library/Fonts/Supplemental/Arial.ttf:text='Stack Overflow': fontcolor=white: fontsize=24: box=1: boxcolor=black#0.5: boxborderw=5: x=(w-text_w)/2: y=(h-text_h)/2" /Users/user/Desktop/test.mp4
So it seems that the output portion is the part FFmpeg doesn't like in Nginx.conf
My thought is I should be passing the overlayed FFmpeg stream to the "overlay" app and have the stream published to Youtube and the frame capture from there. (And also potentially recorded).
Update: When I have tried point to a sh file to run the command rather than the direct FFmpeg exec_push I get:
[alert] 56849#0: kevent() error on 15 filter:-1 flags:4002 (2: No such file or directory)
Thanks so much!
rtmp {
server {
listen 1935;
chunk_size 4096;
application live {
live on;
record off;
exec_push /usr/local/bin/ffmpeg -i rtmp://localhost:1935/live/68.1. -vf drawtext="fontfile=/System/Library/Fonts/Supplemental/Arial.ttf:textfile=/Users/Shared/overlayescaped.txt: reload=1: fontcolor=white: fontsize=20: box=1: boxcolor=black#1: boxborderw=75: x=70: y=925" -c:v libx264 -maxrate 6000k -bufsize 4000k -c:a aac -b:a 160k -ar 44100 -b:a 128k -f mp4 rtmp://localhost:1935/overlay/test;
#push rtmp://localhost:1935/overlay;
}
application overlay {
live on;
record off;
push rtmp://a.rtmp.youtube.com app=live2 playpath=yourstreamkey;
exec_push /usr/local/bin/ffmpeg -i rtmp://localhost:1935/overlay/$name -vf fps=1/60 /Users/Shared/stream/netcam.jpg;
}
}
}
The answer was:
a) I must invoke the Ffmpeg command through a file for this to work. I'm not entirely sure why, but that is just the way it is.
b) I wasn't able before to get logging info from Ffmpeg. It was because I was logging to the wrong spot. I needed to log to /tmp/ because of the unprivileged (nobody) user used by Nginx. Makes sense.
c) At that point once the command was working from a file, I could see the actual errors that Ffmpeg was throwing and could troubleshoot them. Which had a lot to do with option placement, spacing, and ensuring it is a flv container, not an mp4 container.
Here is the Nginx rtmp configuration I ended up with:
rtmp {
server {
listen 1935;
chunk_size 4096;
application live {
live on;
record off;
meta copy;
exec /Users/Shared/ffmpegcommand.sh $name;
}
application overlay {
live on;
record off;
meta copy;
push rtmp://a.rtmp.youtube.com app=live2 playpath=stream-key;
exec_push /usr/local/bin/ffmpeg -i rtmp://localhost:1935/overlay/$name -vf fps=1/60 /Users/Shared/stream/netcam.jpg;
}
}
}
And here is the Ffmpeg command I am using in the command file for the text overlay (now using -filter_complex, as -vf wasn't the proper option in this case).
/usr/local/bin/ffmpeg -i rtmp://localhost:1935/live/68.1. -filter_complex drawtext="fontfile=/System/Library/Fonts/Supplemental/Verdana.ttf: textfile=/Users/Shared/overlayescaped.txt: reload=1: fontcolor=white: fontsize=17: box=1: boxcolor=black#1: boxborderw=80: x=80: y=935" -c:v libx264 -level 4.1 -maxrate 6000k -bufsize 4000k -c:a copy -f flv rtmp://localhost:1935/overlay/newlive 2>>/tmp/ffmpeg.error
I also modified the audio options so that they copy straight from source as no encoding is needed.
Finally, I created the overlay text file from a text file I already had. The existing overlay had a % symbol for humidity so I had to escape that character using sed in a bash script.
escovlfiletmp='/Users/Shared/overlayescapedtmp.txt'
escovlfile='/Users/Shared/overlayescaped.txt'
overlaysearch="% B:"
overlayreplace="\\\\\\% B:"
sed -e "s/${overlaysearch}/${overlayreplace}/g" ${overlayfile} > ${escovlfile}
I've attached a screen cap of the final video stream result. The entire black area is the overlay.
Very happy.
Thank you for all the resources on this website and elsewhere. It took me 4 days and many hours of constant searching but managed to piece it all together.
enter image description here

NGiNX RTMP module exec_xxx commands not working

I a having trouble nginx 1.10.3 and rtmp module exec_xxx commands. I have nginx.conf
with the following code.
rtmp {
...
server {
...
application dash { #creates a rtmp application
exec_options on;
exec_pull /bin/bash /usr/local/nginx/conf/ping.sh pull;
exec_push /bin/bash /usr/local/nginx/conf/ping.sh push;
exec_static /bin/bash /usr/local/nginx/conf/ping.sh static;
exec_publish /bin/bash /usr/local/nginx/conf/ping.sh publish;
...
I can read the rtmp DASH video externally that I publish internally from local host. So I know the conf file is working. I can also verify the directives are active with 'sudo -i nginx -T | grep exec_'. But the ping.sh (shown below) is not being executed.
#!/bin/bash
touch ./test.txt
/bin/echo "got message 1=$1 2=$2 3=$3"
/bin/echo "got message 1=$1 2=$2 3=$3" >>/usr/local/nginx/conf/exec_log.txt
The ping.sh command works when executed from ~/nginx/. How can I tell why the exec_pull static and other 'exec_xxx' commands are not working?

Static webpage on Nginx Docker Container Missing CSS

I am trying to host the Swagger UI on a docker container using Nginx.
When I access my webpage via hostAddress.com it returns the webpage as plain text and inspecting it tells me that it can't find any of the javascript or css files despite them seeming to be present in the container as I have ssh into the container to check.
My dockerfile
FROM nginx
COPY src /usr/share/nginx/html
COPY config/nginx.conf /etc/nginx
EXPOSE 80
nginx.config
events {
worker_connections 4096; ## Default: 1024
}
http {
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html index.htm;
include /etc/nginx/mime.types;
location /swagger {
try_files $uri /index.html;
}
#Static File Caching. All static files with the following extension will be cached for 1 day
location ~* .(jpg|jpeg|png|gif|ico|css|js)$ {
expires 1d;
}
}
}
Here what you can do achieve that.
Like it was mentioned the comment, you can download the artifacts from
https://github.com/ianneub/docker-swagger-ui/blob/master
Create a directory say ngix
Copy Dockerfile, run.sh into that directory
Edit the Dockerfile to make the customization that you need to change the location as shown below:
FROM nginx:1.9
ENV SWAGGER_UI_VERSION 2.1.2-M2
ENV URL **None**
RUN apt-get update \
&& apt-get install -y curl \
&& curl -L https://github.com/swagger-api/swagger-ui/archive/v${SWAGGER_UI_VERSION}.tar.gz | tar -zxv -C /tmp \
&& mkdir /usr/share/nginx/html/swagger \
&& cp -R /tmp/swagger-ui-${SWAGGER_UI_VERSION}/dist/* /usr/share/nginx/html/swagger \
&& rm -rf /tmp/*
COPY run.sh /run.sh
CMD ["/run.sh"]
If you compare the above changes with original Dockerfile, there are changes made in the lines 9, 10 to include additional path i.e., swagger. Of course, you may change as needed.
Next, run the following docker commands to build and run
Build Image:
docker build -t myswagger .
Run it
docker run -it --rm -p 3000:80 --name testmyswagger -e "URL=http://petstore.swagger.io/v2/swagger.json" myswagger
Now you should be able to access your swagger using http://localhost:3000/swagger/index.html

Have nginx access_log and error_log log to STDOUT and STDERR of master process

Is there a way to have the master process log to STDOUT STDERR instead of to a file?
It seems that you can only pass a filepath to the access_log directive:
access_log /var/log/nginx/access.log
And the same goes for error_log:
error_log /var/log/nginx/error.log
I understand that this simply may not be a feature of nginx, I'd be interested in a concise solution that uses tail, for example. It is preferable though that it comes from the master process though because I am running nginx in the foreground.
Edit: it seems nginx now supports error_log stderr; as mentioned in Anon's answer.
You can send the logs to /dev/stdout. In nginx.conf:
daemon off;
error_log /dev/stdout info;
http {
access_log /dev/stdout;
...
}
edit: May need to run ln -sf /proc/self/fd /dev/ if using running certain docker containers, then use /dev/fd/1 or /dev/fd/2
If the question is docker related... the official nginx docker images do this by making softlinks towards stdout/stderr
RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log
REF: https://microbadger.com/images/nginx
Syntax: error_log file | stderr | syslog:server=address[,parameter=value] | memory:size [debug | info | notice | warn | error | crit | alert | emerg];
Default:
error_log logs/error.log error;
Context: main, http, stream, server, location
http://nginx.org/en/docs/ngx_core_module.html#error_log
Don't use: /dev/stderr
This will break your setup if you're going to use systemd-nspawn.
For a debug purpose:
/usr/sbin/nginx -g "daemon off;error_log /dev/stdout debug;"
For a classic purpose
/usr/sbin/nginx -g "daemon off;error_log /dev/stdout info;"
Require
Under the server bracket on the config file
access_log /dev/stdout;
When running Nginx in a Docker container, be aware that a volume mounted over the log dir defeats the purpose of creating a softlink between the log files and stdout/stderr in your Dockerfile, as described in #Boeboe 's answer.
In that case you can either create the softlink in your entrypoint (executed after volumes are mounted) or not use a volume at all (e.g. when logs are already collected by a central logging system).
In docker image of PHP-FPM, i've see such approach:
# cat /usr/local/etc/php-fpm.d/docker.conf
[global]
error_log = /proc/self/fd/2
[www]
; if we send this to /proc/self/fd/1, it never appears
access.log = /proc/self/fd/2
Based on the official docker Nginx image this is already in place
For more information you can see the Dockerfile written for the 1-alpine version which soft link the access logs and error logs to the stdout and stderr respectively. Other docker tags also have it.
ref: https://github.com/nginxinc/docker-nginx/blob/1.23.1/stable/alpine/Dockerfile#L118-L119

Resources