Grunt error with fs.unlinkSync on Sails - gruntjs

I'm using skipper to receive the files, sharp to resize (and save) and fs unlink to remove the old image. But I got a very weird error this time that concerns me a lot:
error: ** Grunt :: An error occurred. **
error:
Aborted due to warnings.
Running "copy:dev" (copy) task
Warning: Unable to read "assets/images/users/c8e303ca-1036-4f52-88c7-fda7e01b6bba.jpg" file (Error code: ENOENT).
error: Looks like a Grunt error occurred--
error: Please fix it, then restart Sails to continue running tasks (e.g. watching for changes in assets)
error: Or if you're stuck, check out the troubleshooting tips below.
error: Troubleshooting tips:
error:
error: *-> Are "grunt" and related grunt task modules installed locally? Run npm install if you're not sure.
error:
error: *-> You might have a malformed LESS, SASS, CoffeeScript file, etc.
error:
error: *-> Or maybe you don't have permissions to access the .tmp directory?
error: e.g., (edited for privacy)/sails/.tmp ?
error:
error: If you think this might be the case, try running:
error: sudo chown -R 1000 (edited for privacy)/sails/.tmp
Grunt stopped running and to have that in production is a big NoNo... I believe that this is caused because of concurrency with fs.unlinkSync(fname). The error is also intermittent and very hard to reproduce in some machines (IO ops/sec maybe?).
I have the following controller action:
var id = 1; // for example
req.file('avatar').upload({
dirname: require('path').resolve(sails.config.appPath, 'assets/images')
}, function(err, files){
var avatar = files.pop();
//file name operations here. output is defined as the path + id + filetype
//...
sharp(avatar.fd)
.resize(800, 800)
.toFile(output, (err, info)=>{
if(err){
res.badRequest();
} else {
fs.unlinkSync(avatar.fd);
res.ok();
}
});
});
Now I've been thinking about a few solutions:
Output the new image directly to .temp
Unlink when files exists on .tmp. Explanation: Grunt already copied the old file so removing it would be safe!
But I don't know if this is some spaghetti code or even if a better solution exists.
EDIT: My solution was, as proposed by arbuthnott, wrap a controller like this:
get : function(req, res){
var filepath = req.path.slice(1,req.path.length);
//remove '/' root identifier. path.resolve() could be used
if(fs.existsSync(path.resolve(filepath))){
return res.sendfile(filepath);
} else {
return res.notFound();
}
}

I think you are on the right track about the error. You are making some rapid changes to in the assets folder. If I read your code right:
Add an image with user-generated filename to assets/images (ex cat.jpg)
Copy/resize the file to an id filename in assets/images (ex abc123.jpg)
Delete the original upload (cat.jpg)
(I don't know the details of sharp, there may be more under the hood there)
If sails is running in dev mode, then Grunt will be trying to watch the whole assets/ folder, and copy all the changes to .tmp/public/. It's easy to imagine Grunt may register a change, but when it gets around to copying the added file (assets/images/cat.jpg) it is already gone.
I have two suggestions for the solution:
One:
Like you suggested, upload your original to the .tmp folder (maybe even a custom subfolder of .tmp). Still place your sized copy into /assets/images/, and it will be copied to /.tmp/public/ where it can be accessed as an asset by the running app. But Grunt will ignore the quick add-then-delete in the .tmp folder.
Two:
Do a bit of general thinking about both what you want to include in version control, and what Grunt tasks you want to be running in production. Note that if you use sails lift --prod then Grunt watch is turned off by default, and this error would not even occur. Generally, I don't feel like we want Grunt to do too much in production, it is more of a development shortcut. Specifically, Grunt watch can use a lot of resources on a production server.
The note about version control is just that you probably want some of the contents of assets/images/ to be in version control (images used by the site, etc), but maybe not in the case of user-uploaded avatars. Make sure you have a way to differentiate these contents (subdirectories or whatever). Then they can be easily .git-ignore'd or whatever is appropriate.
Hope this helps, good luck!

Related

Why does `getResourceAsStream` sometimes load a resource even when there is a typo in the resource path?

I have a Jar (we'll call it a.jar) with a resource in it at path foo/bar.txt and a function as follows:
object FooBarLoader {
fun loadFooBarText() = javaClass.getResourceAsStream("foo//bar.txt")
?.bufferedReader()
?.readLines()
?.joinToString("\n")
}
When I test the function in a unit test (JUnit 4, running with Gradle 6), it loads the text from the resource file despite the obvious typo (the // in the middle of the resource path).
I also have a CLI application (in b.jar) that has a dependency on a.jar. When the CLI application calls loadFooBarText(), it got a null result due to the resource not being found. This was fixed by fixing the typo (// -> /) in the function in a.jar. No other changes were needed to fix it.
So, my question is why did the wrong path work in one situation (unit tests of a.jar) and not the other (call from b.jar)?
How do you run the unit test with a.jar ? Just run it in your IDE or use command java -jar a.jar ?
If you ran it just in IDE,I think difference is the search path between local files and zip files .
Your first application searches the file in your target directory and the second application searches it in the jar which is a compressed file.
When searching files in local path, command will be changed to right one by system.
The two commands below are the same in both Windows/Linux.
cd work//abc/ddd
cd work/abc/ddd
But when searching files in a jar file which is actually compressed zip file, path should be a restrict written or else the program will find nothing.

How do I compile LESS files every time I save a document?

I've installed Less via npm like this
$ npm install -g less
Now every time that I want to compile source files to .css, I run
$ lessc styles.less styles.css
Is there any way via the command line to make it listen when I save the document to compile it automatically?
The best solution out there I've found is the one recommended on the official LESS website: https://github.com/jgonera/autoless. It is dead simple to use. Also it listens to the changes in the imported files to compile.
Have a look at this article:
http://www.hongkiat.com/blog/less-auto-compile/
It offers GUI solutions (SimpLESS, WinLESS, LESS.app, and ChrunchApp) and a node solution. (deadsimple-less-watch-compiler)
Are you using less alone or with Node.JS ? Because if you are using it with node, there are easy ways to resolve this problem. The first two I can think of are (both these solutions go in your app.js) :
using a middleware, like stated in this stack overflow discussion
var lessMiddleware = require('less-middleware');
...
app.configure(function(){
//other configuration here...
app.use(lessMiddleware({
src : __dirname + "/public",
compress : true
}));
app.use(express.static(__dirname + '/public'));
});
another method consists of making a system call as soon as you start your nodeJS instance (the method name may differ based on your NodeJS version)
// before all the treatment is done
execSync("lessc /public/stylesheets/styles.less /public/stylesheets/styles.css");
var app = express();
app.use(...);
In both cases, Node will automatically convert the less files into css files. Note that with the second option, Node was to be relaunched for the conversion to happen, whereas the first option will answer your need better, by always checking for a newer version in a given directory.

Why does Meteor complain that an insert method for a collection is already defined?

Can anyone tell me why the code below throws the following error? :
Error: A method named '/players/insert' is already defined
I'm new to Meteor and coffeescript so I may be overlooking something simple.
Here's my port of the leaderboard example to coffeescript:
###
Set up a collection to contain player information. On the server,
it is backed by a MongoDB collection named "players."
###
Players = new Meteor.Collection("players")
if Meteor.is_client
Template.leaderboard.players = ->
Players.find({}, {sort: {score: -1, name: 1}})
Template.leaderboard.selected_name = ->
player = Players.findOne(Session.get "selected_player")
player and player.name
Template.player.selected = -> if Session.equals("selected_player", this._id) then "selected" else ''
Template.leaderboard.events = {
'click input.inc': ->
Players.update(Session.get("selected_player"), {$inc: {score: 5}})
}
Template.player.events = {
'click': ->
Session.set("selected_player", this._id)
}
# On server startup, create some players if the database is empty.
if Meteor.is_server
Meteor.startup ->
if Players.find().count() is 0
names = [
"Ada Lovelace"
"Grace Hopper"
"Marie Curie"
"Carl Friedrich Gauss"
"Nikola Tesla"
"Claude Shannon"
]
Players.insert({name: name, score: Math.floor(Math.random()*10)*5}) for name in names
The full stack trace is as follows:
[[[[[ ~/dev/meteor/leaderboard ]]]]]
Running on: http://localhost:3000/
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: A method named '/players/insert' is already defined
at app/packages/livedata/livedata_server.js:744:15
at Function.<anonymous> (app/packages/underscore/underscore.js:84:24)
at [object Object].methods (app/packages/livedata/livedata_server.js:742:7)
at new <anonymous> (app/packages/mongo-livedata/collection.js:111:13)
at app/leaderboard.js:4:11
at /Users/alex/dev/meteor/leaderboard/.meteor/local/build/server/server.js:109:21
at Array.forEach (native)
at Function.<anonymous> (/Users/alex/dev/meteor/leaderboard/.meteor/local/build/server/underscore.js:76:11)
at /Users/alex/dev/meteor/leaderboard/.meteor/local/build/server/server.js:95:7
Exited with code: 1
I'm running Meteor version 0.4.0 (8f4045c1b9)
Thanks in advance for assistance!
You would also get this error, regardless of using coffeescript or plain javascript, if you duplicated your files. For example, copying your sources files to a subdirectory named Backup would produce this error, because Meteor merges files from subdirectories.
This appears to be a configuration issue with coffeelint (installed globally with npm).
I originally installed coffeelint to check that my coffeescript code was correct and had no errors.
I installed coffeelint as per the instructions with:
sudo npm install -g coffeelint
coffeelint worked fine when run stand-alone against .coffee files.
However, when running any Meteor project with coffeescript package added I got the above error.
On a whim, I thought the error might be due to conflict with my exisiting node install.
I decided to uninstall coffeelint first with:
sudo npm uninstall -g coffeelint
and then deleted the previously meteor-generated leaderboard.js file.
After re-starting meteor the coffeescript example above worked as expected without errors.
try moving (ie copying and deleting the original )
Players = new Meteor.Collection("players")
one time below if Meteor.is_client
and another time below if Meteor.is_server
I don't know exactly why, as I'm new to Meteor too, but that worked for me, I assume that the server side needs it's own reference,as well as the client,
although declaring outside the scope should do the same (maybe a bug, remember they're still at 0.5.0 preview , which makes me think you might wanna upgrade and try some new smart packages that are with the new version, it looks like you're using 0.4), but when the files in my server wouldn't recognize anything I defined the root directory of meteor (which pushes these files to both client and server), I defined the server's own reference, and I got the same error, and until I moved the 'public' declaration of the reference to give the server and the client each their own copy, nothing worked.
Hopefully that helps...

Meteor reading CSV files to populate database don't work after deploy

So I have a bunch of data that I want to load into database from CSV. I've hacked together a solution that works in local development, but when I deploy to meteor.com, it no longer works.
I'm loading the csv file in the folder /server/data/:
function readData(name){
var fs = __meteor_bootstrap__.require('fs');
var path = __meteor_bootstrap__.require('path');
var base = path.resolve('.');
var data = fs.readFileSync(path.join(base, '/server/data/', name));
return CSVToArray(data);
}
After I deploy to meteor.com, i got:
INFO Error: ENOENT, no such file or directory '/meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/public/data/categories.csv'
at Object.openSync (fs.js:240:18)
at Object.readFileSync (fs.js:128:15)
at readData (app/server/models.js:10:16)
at app/server/categories.js:6:7
at /meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/server.js:132:63
at Array.forEach (native)
at Function.<anonymous> (/meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/underscore.js:76:11)
at /meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/server.js:132:7
Any idea how I can get meteor to see the csv file after deployment?
I realize this question is old, but it still ranks high on certain keyword searches. So, if you're using Meteor 0.6.5+, you can use the new Assets API.
The issue is that meteor only bundles files that it knows about (ie. JS/CSS/HTML/+more depending on which packages you use) up when it deploys.
Try putting the file you need in the public directory (this directory is exempt from the above rule).
Thanks to SamuelDavis and Tom Coleman's tips. I ended up figuring out what the problem is. Turns out the bundled app is no longer formated as client, public, and server. I ended up debugging it by running meteor bundle to create a tarball. extract the tarball and took a look inside to find where the data folder is. Tom was also right that the data folder needed to be in the public folder in order to get bundled in.
It appears that the base directory is not in the same location that contains the file '/server/data/xxx.csv'.
Before you try anything else, log the base path after calling "var base = path.resolve('.'). If that value is what you expected, log the files that appear in that directory. Again if the files are what you expected, navigate into the /server folder and print out those directories and so forth.
This should pinpoint you to which folder and/or directory is missing and should indicate where you should place the CSV file in future.

PHPUnit - does nothing, no errors, no output

Sorry for another 'phpunit doesn't work' question. It used to work for years now. Today I reinstalled PEAR and phpunit for reasons not connected to this problem. Now when I run phpunit as I usually did. Nothing happens. The cli just shows me a new line, no output whatsoever.
Has anyone encountered this problem or has an idea what could have caused it.
PHPUnit Version: 3.5.15
PEAR Version: 1.9.4
PHP Version: 5.3.8
Windows 7
I'm on OSX and MAMP. To get error messages I had to adjust the following entries in php.ini:
display_errors = On
display_startup_errors = On
Please not that this has to go into /Applications/MAMP/bin/php/php5.3.6/conf/php.ini .
For future reference, for those who are facing any problem with PHPUnit, and PHPUnit is failing silently, just add this three lines inside phpunit.xml:
<phpunit ....... >
...
...
<php>
<ini name="display_errors" value="true"/>
</php>
</phpunit>
After that run the tests again, and now you can see why PHPUnit is failing,
AND ... ENJOY UNIT TESTING :)
I know the orignal poster's question is already answered, but just for any people searching in the future: one thing that can cause PHPUnit to fail silently (i.e. it just stops running tests without telling you why) is that it has an error handler that it set up before each test run, which is intended to capture errors and display them at the end of the test run. The problem is that certain errors halt execution of the whole test run.
What I generally do when this happens, as a first step, is reset the error handler to something that will immediately output the error message. In my base test class I have a method called setVerboseErrorHandler, which I'll call at the top of the test (or in setUp) when this happens. The below requires php 5.3 or higher (due to the closure), so if you're on 5.2 or lower you can make it a regular function.
protected function setVerboseErrorHandler()
{
$handler = function($errorNumber, $errorString, $errorFile, $errorLine) {
echo "
ERROR INFO
Message: $errorString
File: $errorFile
Line: $errorLine
";
};
set_error_handler($handler);
}
Create the simplest test class you can without a bootstrap.php or phpunit.xml to first verify that your new installation works. PHPUnit will stop without any message if it cannot instantiate all of the test cases--one for each test method and data provider--before running any tests.
You have already figured out how to get it to work, but my solution was a little different.
First thing you can do is check the exit status. If it's not 0, then PHP exited and because of the INI configuration settings set, none of the PHP error messages were outputted. What I did was to enable the "display_errors" INI setting, and set "error_reporting" to E_ALL. I was then able to identify errors such as PHP not being able to parse a certain script. Once I fixed that, PHPUnit ran properly.
I managed to spectacularly paint myself in a corner with a custom "fatal error handler" that in certain rare conditions turned out to output nothing. Those conditions, in accordance with Murphy's Law, materialized once I had forgotten the handler was in place.
Since it wasn't really a "PHPunit problem", none of the other answers helped [although #David's problem was at the bottom the same thing], even though the symptoms were the same - phpunit terminating with no output, no errors, no log and no clue.
In the end I had to resort to a step-by-step tracing of the whole test suite by adding this in the bootstrap code:
register_shutdown_function(function() {
foreach ($GLOBALS['lastStack'] as $i => $frame) {
print "{$i}. {$frame['file']} +{$frame['line']} in function {$frame['function']}\n";
}
});
register_tick_function(function() {
$GLOBALS['lastStack'] = debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS, 8);
});
declare(ticks=1);
If anyone ever manages to do worse than this, and somehow block stdout as well, this modification should work:
register_shutdown_function(function() {
$fp = fopen("/tmp/path-to-debugfile.txt", "w");
foreach ($GLOBALS['lastStack'] as $i => $frame) {
fwrite($fp, "{$i}. {$frame['file']} +{$frame['line']} in function {$frame['function']}\n");
}
fclose($fp);
});
an old thread this one, but one I stumbled across when having the same problem.
I had the same problem, nothing being returned to console including print, print_r, echo etc.
solved it by using --stderr as a test-runner option.
Check that you haven't written any logic into your code that just dies, with no output. For example,
<?php
if (!array_key_exists('SERVER_NAME', $_SERVER)) {
die();
}
This was exactly my case; I'd made some assumptions about the environment which were correct when running the code via Apache, but weren't fulfilled when running from CLI and the code did not echo any output.
PHPUnit tried to include the bootstrap file before giving the usual init output, but died during the bootstrapping proccess, hence exiting with status 0 and no output.
If when you run from command line a recent version of phpunit like this
> php phpunit
or
> ./phpunit
or
> php ./phpunit.phar
or
> ./phpunit.phar
And you immediatly return to the prompt with no messages, this is probably due to a "suhosin secutiry" setup.
phpunit is now a "phar" package including all libraries. To be able to run such file when php has the suhosin security module enabled, you must first set this
suhosin.executor.include.whitelist = phar
into you php.ini file (for example, with debian/ubuntu, you may have to edit file /etc/php5/conf.d/suhosin.ini
i tried everything here, but nothing worked until i tried phpunit --no-configuration simpletest.php. that finally gave me some output, which implies that my phpunit.xml.dist file is broken. (i'll come back and update this once i debug it.)
the contents of simpletest.php are below, but any test file should work.
<?php
use PHPUnit\Framework\TestCase;
final class FooTest extends TestCase
{
public function testFoo()
{
$this->assertEquals('x', 'y');
}
}
Check if the phpunit you're running and the one you installed are the same:
$ pear list phpunit/phpunit
...
script /path/to/phpunit
...
Try to execute exactly that phpunit with the full path.
Then check your PATH variable and see if the correct directory is in it. If not, fix that.
If that does not help, use write something into the phpunit executable, e.g. "echo 123;" and run phpunit. Check if you see that.
For me the conflict was with Xdebug's directive
xdebug.remote_enable=1
If you are using composer, check to make sure your included PHP files are not ending the code executions. The same goes for when you included certain PHP files explicitly.
In my case, I was working on a WordPress plugin and one of the PHP files I included directly in composer.json (which I don't want to load through PSR-4 because WordPress's coding standards don't support it yet) had this code on top;
if (!defined('ABSPATH')) {
exit(); // exit if accessed directly
}
And since ABSPATH will not be defined when I run the tests directly, the code was exiting.
This is because, since I told Composer to always load these files each time, this part of the code will execute, while the other files included though autoload PSR will load on demand.
So check to make sure any of the files you included are not stopping the code execution. If it happens, then even when you run phpunit --info the code will still exit and you won't see any output.
I was facing a seems problem. I could run phpunit from root directory but not from anywhere else. so I put the "--configuration" tag, and point it to my xml configuration.
$ ./<path_to_phpunit>phpunit --configuration <path_to_phpunitxml>/phpunit.xml
The path to phpunit is optinal, I used it because I installed locally by composer.
In /composer/vendor/phpunit/phpunit/src/TextUI/Command.php in main() function in catch (Throwable $t) {} block var_dump (echo / print_r) exception. If an exception exists, you, probably, will solve the current problem.

Resources