Nginx custom 404 page fails when in a higher directory - nginx

Question

I am trying to create a custom 404 page with Nginx. My nginx.conf file looks like this:
http {
server {
location / {
root /nginx/html;
}
error_page 404 /custom_404.html;
location = /custom_404.html {
root /nginx/html;
internal;
}
}
}
When I check with a URL like "/page_that_doesnt_exist", it works fine.
But if I add a trailing "/", eg "/page_that_doesnt_exist/" or "/page_that_doesnt_exist/and_more_stuff" it fails, returning a blank screen (not the default nginx 404 page).
When I check the server messages, it tells me the following:
"GET /page_that_doesnt_exist/ HTTP/1.1" 404 280 "-"
[error] 6#6: *3 open() "/nginx/html/page_that_doesnt_exist/error404.js" failed (2: No such file or directory), client: 172.17.0.1, server: , request: "GET /page_that_doesnt_exist/error404.js HTTP/1.1", host: "localhost:8080", referrer: "http://localhost:8080/page_that_doesnt_exist/"
"GET /page_that_doesnt_exist/error404.js HTTP/1.1" 404 280 "http://localhost:8080/page_that_doesnt_exist/"
Which I take to mean that instead of nginx searching for my "custom_404.html" file in my "nginx/html" folder, it is adding "page_that_doesnt_exist" to the directory. Thus it now searches in "nginx/html/page_that_doesnt_exist/" and doesn't find a "custom_404.html" file in that directory.
What am I doing wrong here?