I build a rest service in Deno (Oak) and also serve static files. However, when I run deno compile I would like to have those static files included into the single binary file that is ejected. Is this possible?
From what I can tell, neither Deno nor Oak have intentional support for this.
A downside of even doing this is that your binary file may become large. This isn't only an issue with distribution but may also slow loading and executing the binary.
Nevertheless, one way you can make "static" files available in a compiled binary is to encode the files as JavaScript modules (similar to using WebAssembly in Deno).
e.g. The following module encodes a static file, named example.txt, storing its file type, txt, and its contents, hello world\n. The contents are base64 encoded (thank you jsejcksn for the suggestion). You can encode and decode the contents other ways as well or even use different encodings depending on the file type if you like.
example.txt.ts:
export default {
type: "txt",
data: "aGVsbG8gd29ybGQK",
};
You can programmatically create modules like this from static files.
e.g. encode-as-module.ts:
import { extname } from "https://deno.land/std#0.155.0/path/mod.ts";
import { encode } from "https://deno.land/std#0.155.0/encoding/base64.ts";
const [inputPath, outputPath = `${inputPath}.ts`] = Deno.args;
const type = extname(inputPath).slice(1);
const bytes = await Deno.readFile(inputPath);
const script = /* JavaScript */ `export default {
type: "${type}",
data: "${encode(bytes)}",
};
`;
await Deno.writeTextFile(outputPath, script);
Usage:
deno run --allow-read --allow-write encode-as-module.ts example.txt
Once you have your static files encoded as modules you can then change your Oak app from serving them using send() to serving them using context.response (passing the type and body). More work will need to be done here to encode a list of all the static files, etc. but I think what's already provided here illustrates the idea sufficiently.
Related
Barrel/index files seem to create issues when used with next.js. It doesn't seem established if it's purely a webpack issue or both webpack and next.js
According to this issue tree shaking stops working if we use barrel files. I also created a small repo where I have an issue with an index file. Not sure if it's a tree shaking issue.
Steps to reproduce the issue:
npm install
npm run dev
in browser, visit http://localhost:3000/about-pro, expect to see blank page with errors or warnings in browser's console
go to server's console(where you run npm run dev)
see an error of sort "Module not found: Can't resolve 'fs'" (1) (2) (3)
1- this comes from the await serialize in getAboutPageData file. Which itself is only called within getStaticProps
2 - googling for this issue, you'll find solutions such as modifying next.config.js file. It still doesn't work. Feel free to uncomment the next.config.js file and see for yourself
3 - to "solve" the issue, go to about-pro.tsx, in the imports, import AboutPage from its own file instead of from the barrel/index file
If I only import getAboutPageData from the barrel/index file, then it works fine. But as soon as I import e.g. AboutPage from it, it starts throwing unrelated issues.
Can I continue using barrel/index files with next.js and if yes, is there a simple and intuitive way to do that?
The issue is not in the barrel files themselves but in the library that you're using combined with barrel files.
If you take a look at the readme file https://github.com/hashicorp/next-mdx-remote#examples you can find a warning:
IMPORTANT: Be very careful about putting any mdx-remote code into a separate "utilities" file. Doing so will likely cause issues with nextjs' code splitting abilities - it must be able to cleanly determine what is used only on the server side and what should be left in the client bundle. If you put mdx-remote code into an external utilities file and something is broken, remove it and start from the simple example above before filing an issue.
So in order to make your code work you need to remove the export of getAboutPageData from your barrel file, like this:
export { default as AboutPage } from './AboutPage';
// export { default as getAboutPageData } from './getAboutPageData';
and move the code that uses the library inside the about-pro.tsx file.
import { AboutPage } from '../modules/about';
import { serialize } from 'next-mdx-remote/serialize';
const AboutPro = (props) => {
return <AboutPage {...props} />;
};
export const getStaticProps = async () => {
const serializedContent = await serialize(`# Header1`);
const data = serializedContent;
return { props: {} };
};
export default AboutPro;
I think the issue is that the modules imported in barrel files are executed both client and server side. Probably removing side effects from the barrel file could solve the issue, but I don't know enough about Next.js to be able to do it correctly.
In deno you can load related modules or other code by just referencing the relative path to those ES6 modules. Deno will handle loading them appropriately. What's the way to do this for non-es6 modules? For example: say I wanted to include some custom css with my deno project? Deno doesn't allow doing import mycss from "./relative.css";.
Deno file operations do work for local files, but they're evaluated relative to the cwd not the current file, and they don't work for arbitrary URLs. fetch, on the other hand, should be perfect, but currently doesn't support file schemes and the decision isn't being actively considered. Combining these yields the only solution I can come up with, but I really don't like it:
async function loadLocal(relative: string): Promise<string> {
const url = new URL(relative, import.meta.url);
if (url.protocol === 'file:') {
return await Deno.readTextFile(url.pathname);
} else {
const resp = await fetch(url.href);
return await resp.text();
}
}
This seems like it should mostly work, but it seems like a terrible way to hack in something that I expected would be supported by design in deno. It also must be redeclared in each file, or have the callers URL passed in, although there might be a way to avoid that. It doesn't work on windows without modifying the path delimiter.
Update
Deno.emit seems close to what I would want, however for some reason it has different behavior than standard importing:
If the rootSpecifier is a relative path, then the current working directory of the Deno process will be used to resolve the specifier. (Not relative to the current module!)
It also still requires that the paths be to valid modules, instead of arbitrary text.
As #Zwiers pointed out, deno 1.6 now supports fetch with the file protocol, so this is now irrelevant.
I have an Angular 2 application in which i wish to use wrapbootstrap. I do however have a problem with the fonts (bootstrap, font-awesome, google) as i do not know how to implement them.
When using the css file for wrapbootstrap is says it cannot find font awesome:
"Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost:8000/fonts/font-awesome/fontawesome-webfont.woff2?v=4.4.0"
I cannot make sense of this as i can see the missing file(s) in resources in the chrome console on that exact address.
The font files are currently in a folder vi the following relative path from the css (application.css) file using them:
Which fits the required path in the css file:
I hope someone out there can provide some guidance as i am lost.
Thanks in advance
Solved
the problem was apparently the location of my fonts folder.
my file structure are as follows:
and i had firstly added the fonts/ relative to where the application.css file was. It had to be located in the root of my app (src)
Adding fonts installed with package manager is quite often a task. For instance, using font-awesome or any other similar library is a typical task one will need.
For this purpose, you can go through the following steps:
In tools/config/project.config.ts:
...
export class ProjectConfig extends SeedConfig {
PROJECT_TASKS_DIR = join(process.cwd(), this.TOOLS_DIR, 'tasks', 'project');
FONTS_DEST = `${this.APP_DEST}/fonts`;
FONTS_SRC = [
'node_modules/bootstrap/dist/fonts/**'
];
...
Create a file tools/tasks/project/build.fonts.ts:
import * as gulp from 'gulp';
import Config from '../../config';
export = () => {
return gulp.src(Config.FONTS_SRC)
.pipe(gulp.dest(Config.FONTS_DEST));
};
In gulpfile.ts (or in seed.tasks.json for newer versions of the Seed)
// Build dev.
gulp.task('build.dev', done =>
runSequence('clean.dev',
'tslint',
'build.assets.dev',
'build.fonts', // Added task;
'build.js.dev',
'build.index.dev',
done));
// Build prod.
gulp.task('build.prod', done =>
runSequence('clean.prod',
'tslint',
'build.assets.prod',
'build.fonts', // Added task;
'build.html_css.prod',
'build.js.prod',
'build.bundles',
'build.bundles.app',
'build.index.prod',
done));
// Build test.
gulp.task('build.test', done =>
runSequence('clean.dev',
'tslint',
'build.assets.dev',
'build.fonts', // Added task;
'build.js.test',
'build.index.dev',
done));
src
I thought this would be easy.
I want to create simple files the user can download by clicking a link.
Write what you want into the servers assets/app folder and then generate a simple link
Download> me!
Writing files into Meteor's server side asset folder is easy. And the download link above will always download a file with the name you specified.
You will get a yourNewFile.txt in the client's download folder. But, unfortunately its content will not be what you wrote on the server (new.txt).
Meteor has the strange behavior of downloading its startup html page as the content if the name of your content wasn't originally in the public folder. I think this is bug .... put the above anchor into a default Meteor project and click the link .. don't even create a public folder. You get a downloaded file with the name you asked for...
So, if you put stubs in the public folder (you know the names of the assets you are going to create) then you can create them dynamically.
I don't know the names before hand. Is there any way to get Meteor to 'update' its assets list with the new names I want to use?
I know there are packages that can do this. I'd like to just do it myself as above, really shouldn't be this hard.
The public/ folder intended use is specifically for static assets. Its content is served by the node http server.
If you want to dynamically generate assets on the server, you can rely on iron:router server side routes.
Here is a simple example :
lib/router.js
Router.route("/dynamic-asset/:filename",function(){
var filename = this.params.filename;
this.response.setHeader("Content-Disposition",
"attachment; filename=" + filename);
this.response.end("Hello World !");
},{
name: "dynamic-asset",
where: "server"
});
In server-side route controllers, you get access to this.response which is a standard node HTTP response instance to respond to the client with the correct server generated content. You can query your Mongo collections using the eventual parameters in the URL for example.
client/views/download/download.html
<template name="download">
{{#linkTo route="dynamic-asset" target="_blank" download=""}}
Download {{filename}}
{{/linkTo}}
</template>
client/views/parent/parent.html
<template name="parent">
{{> download filename="new.txt"}}
</template>
The linkTo block helper must be called in a context where the route parameters are accessible as template helpers. It will generate an anchor tag having an href set to Router.path(route, dataContext). It means that if our server-side route URL is /dynamic-asset/:filename, having a data context where filename is accessible and set to "new.txt" will generate this URL : /dynamic-asset/new.txt.
In this example we set the current data context of the download template to {filename: "new.txt"} thanks to the template invocation syntax.
Note that target="_blank" is necessary to avoid being redirected to the dynamic asset URL inside the current tab, and the download HTML attribute must be set to avoid considering the link as something the browser should open inside a new tab. The download attribute value is irrelevant as it's value will be overriden server-side.
Here is the raw Picker (meteorhacks:picker) route and method I used to get this running. I've kept it lean and its just what I got working and probably not the best way to do this ... the synchronous methods (like readFileSync) throw exceptions if things are not right, so they should be wrapped in try-catch blocks and the mkdirp is a npm package loaded through meteorhacks:npm package hence the Meteor.npmRequire. Thanks again to saimeunt for the directions.
Picker.route('/dynamic-asset/:filename', function(params, req, res, next) {
console.log('/dynamic-asset route!');
var fs = Npm.require('fs');
var path = Npm.require('path');
var theDir = path.resolve('./dynamic-asset');
var filename = params.filename;
var fileContent = fs.readFileSync(theDir + '/' + filename, {encoding:'utf8'});
res.end(fileContent);
});
The Meteor method that creates the file is
writeFile: function(fname, content) {
console.log('writeFile', fname);
var fs = Npm.require('fs');
var path = Npm.require('path');
var mkdirp = Meteor.npmRequire('mkdirp');
// verify/make directory
var theDir = path.resolve('./dynamic-asset');
mkdirp(theDir);
fs.writeFileSync(theDir + '/' + fname, content);
return 'aok';
}
and the hyper link I generate on the client if the file gets created looks like this:
Download lane file now
I incorrectly stated in my original question at the top that you could use stubs and write files into the assets folder. Its not so .. you will only get back the stub ... sorry.
I have a data.json file that I would like to load and that I have placed in the lib/ folder. What should I do in order to load that JSON into a variable in the server?
Thanks
There are three ways you can go about this, it depends what you're most comfortable with & your use case.
The first is to store it as a JS Object
if your json data is { "name":"bob" } you could use myjson = {"name":"bob"} in a .js file in the /lib folder and just call myjson when you need it.
Using an http call
You need the Meteor http package, installed via meteor add http.
Server Side code
myobject = HTTP.get(Meteor.absoluteUrl("/myfile.json")).data;
Client Side Code
HTTP.get(Meteor.absoluteUrl("/myfile.json"), function(err,result) }
console.log(result.data);
});
Another way to do it is to fetch the json file ajax style (you would have to put it in your /public folder though and use Meteor.http to call it.
Read the file directly
Lastly you could read the file directly, you store your myfile.json in a private directory in your project's root:
var myjson = {};
myjson = JSON.parse(Assets.getText("myfile.json"));
If you want to access this on the client side you would have to interface it with a Meteor.methods and Meteor.call
So whichever way you want, the first is the easiest but I'm not too sure how you want to use it or whether you want to pick the file or something
As I am new to all this I suspect this is not the correct way to do this, however this has worked for me...
Three coffee script files, two in the server directory:
server.coffee:
Meteor.startup ->
insertSample = (jsondata) ->
Fiber(->
Documents.insert
name: "Sample doc"
data: jsondata
).run()
if Documents.find().count() is 0
insertJSONfile("tests/test.json", insertSample)
and insertJSONfile.coffee:
fs = __meteor_bootstrap__.require("fs")
insertJSONfile = (file, insert) ->
jsondata = undefined
fs.readFile file, (err, data) ->
throw err if err
jsondata = JSON.stringify(JSON.parse(data))
insert(jsondata)
and model.coffee in the root dir:
#Documents = new Meteor.Collection("documents")
On startup this should load and insert the JSON file (in my case I've stored this in the tests directory) into a field in the documents collection.
I would love to hear from others on how this should be done properly.
I assume you want the json content to be represented as an object and not as a simple string.
I use js-yaml (https://github.com/nodeca/js-yaml), assuming you install the npm package. You can also just copy it manually.
yaml = __meteor_bootstrap__.require('js-yaml')
fs = __meteor_bootstrap__.require('fs')
content = fs.readFileSync(file, 'utf8')
object = yaml.load(content)
and that's it! I personally persist my json into meteor collections.