How to return binary data from custom wordpress rest api endpoint - wordpress

Question

I am writing a custom endpoint for a REST api in wordpress, following the guide here: https://developer.wordpress.org/rest-api/extending-the-rest-api/adding-custom-endpoints/
I am able to write a endpoint that returns json data. But how can I write an endpoint that returns binary data (pdf, png, and similar)?
My restpoint function returns a WP_REST_Response (or WP_Error in case of error).
But I do not see what I should return if I want to responde with binary data.

Answer 1

Late to the party, but I feel the accepted answer does not really answer the question, and Google found this question when I searched for the same solution, so here is how I eventually solved the same problem (i.e. avoiding to use WP_REST_Response and killing the PHP script before WP tried to send anything else other than my binary data).
function download(WP_REST_Request $request) {
$dir = $request->get_param("dir");
// The following is for security, but my implementation is out
// of scope for this answer. You should either skip this line if
// you trust your client, or implement it the way you need it.
$dir = sanitize_path($dir);
$file = $request->get_param("file");
// See above...
$file = sanitize_path($file);
$sandbox = "/some/path/with/shared/files";
// full path to the file
$path = $sandbox.$dir.$file;
$name = basename($path);
// get the file mime type
$finfo = finfo_open(FILEINFO_MIME_TYPE);
$mime_type = finfo_file($finfo, $path);
// tell the browser what it's about to receive
header("Content-Disposition: attachment; filename=$name;");
header("Content-Type: $mime_type");
header("Content-Description: File Transfer");
header("Content-Transfer-Encoding: binary");
header('Content-Length: ' . filesize($path));
header("Cache-Control: no-cache private");
// stream the file without loading it into RAM completely
$fp = fopen($path, 'rb');
fpassthru($fp);
// kill WP
exit;
}

Answer 2

I would look at something called DOMPDF. In short, it streams any HTML DOM straight to the browser.
We use it to generate live copies of invoices straight from the woo admin, generate brochures based on $wp_query results etc. Anything that can be rendered by a browser can be streamed via DOMPDF.