Is there a way (e.g. flag) to terminate whole nightwatch tests when a single failed test occur? Or to get at least a status code that the some test failed, within the program?
Thanks!
In your "test_settings" section of your nightwatch.json, add the following entry:
"end_session_on_fail": true
It looks promising.
You can also try creating a global in global.js and set it to true:
"abortOnAssertionFailure: true"
You can run each test suit by different npm script, such as-
"test1": "nightwatch --retries=1 ./src/tests/test1.js",
"test2": "nightwatch --retries=1 ./src/tests/test2.js"
and then you can run this npm script-
"run_tests": "npm run test1 && npm run test2"
is one test case will fail in 'test1' the npm command will fail as well and no other tests case/suits will run.
Related
In node there is a --check option to check if code is valid for execution without actually running it if valid.
I have been looking for an equivalent in deno (deno help, deno run help, deno lint help)
The closest i can figure out is deno lint, but it will happily lint the following typescript code (in test.ts), which deno run subsequently refuses to run (obviously).
What is the canonical deno "dry run"?
test.ts:
const message = "hello!";
message();
check if code is valid for execution
"dry run"
These concepts don't exist. I think what you want is to have your TypeScript code analyzed using static type-checking (without actually running the module). This is called compilation and the Deno subcommand you want in this case is cache:
deno cache module.ts
Re-cache the entrypoint module and all dependencies:
deno cache --reload module.ts
Using your example test.ts:
% deno cache test.ts
Check file:///Users/deno/test.ts
error: TS2349 [ERROR]: This expression is not callable.
Type 'String' has no call signatures.
message();
~~~~~~~
at file:///Users/deno/test.ts:2:1
Thanks for the cypress team.
I have one issue with e2e headless test on command.
When I run the test with --watch option, it passes well.
But it is failed for the --headless option.
I need to run this on CI, so it has to resolve this issues soon.
Here are commands that I run.
yarn nx run checkout-e2e:e2e --watch
yarn nx run checkout-e2e:e2e --headless
I have 3 test files totally, and the specific thing is that the first test doesn't pass after long time delay(around 60 sec) but the rest ones pass.
Here is one code file to run.
The other files are almost same.
const classname = 'buybox--buy-box-example-one-size'
const url = `?path=/story/${classname}`
describe('BuyBox Example visual regression', () => {
it('Matches the snapshot', () => {
cy.visit(url)
cy.viewport(1000, 1500)
cy.get(`#${classname}`)
cy.get('body').type('S')
cy.get('body').type('A')
cy.wait(3000)
cy.get('body').matchImageSnapshot('buyboxOneSizeDesktopExample')
cy.viewport(375, 667)
cy.get(`#${classname}`)
cy.get('body').type('S')
cy.get('body').type('A')
cy.wait(5000)
cy.get('body').matchImageSnapshot('buyboxOneSizeExample')
})
})
My Dev Env:
Windows 10
VS code
Cypress 6.0.1
Node 14.16.1
Storybook 6.2.8
Here is the error screenshot in headless mode.
https://prnt.sc/131k32j
Please reply if anyone knows its reason and solution.
Thanks.
I've tried to solve this issue and found it finally.
Go to plugin/index.js and comment this line...
// on('file:preprocessor', preprocessTypescript(config))
Fixing e2e test failed on headless mode
I am not sure why this works for me but I hope that this will be helped for everyone who has same issue.
If anyone knows the reason of this solution, comment here please.
Thanks.
I have been experimenting with javascript frameworks for test automation and one of them is testCafe. I have been able to set up a simple TestCafe project and run some test cases for my application. However, now, the requirement is to have some kinda BDD support built in it. I looked up a few testCafe-cucumber integration projects on GitHub but I can't get them to run. here are a few that I tried:-
1) https://github.com/rquellh/testcafe-cucumber
- I cloned the repo,
- did npm install,
- run the test cases using "npm test",
- blank browser launches but test doesn't run. I see this error in VS code console:
× Before # features\support\hooks.js:46
Error: function timed out, ensure the promise resolves within 20000 milliseconds
at Timeout._onTimeout (C:\Users\Mo\Desktop\TestCafe\github\testCafeBDD\testcafe-cucumber\node_modules\cucumber\src\user_code_runner.js:61:18)
at ontimeout (timers.js:482:11)
at tryOnTimeout (timers.js:317:5)
at Timer.listOnTimeout (timers.js:277:5)
× After # features\support\hooks.js:60
ReferenceError: testController is not defined
Then I tried another gitHub project namely this one : https://github.com/kiwigrid/gherkin-testcafe
the run command in readme doesn't work for me, it doesn't even recognize the "gherkin-testcafe".
When I run my TestCafe test cases without the cucumber I have this line in my package.json
"scripts": {
"test": "testcafe chrome Tests/ -e --proxy https.proxy.mycompany.com:8000"
},
the proxy is mentioned because I am behind a proxy and without this the browser launches but does not run any test cases. I found this fix on testCafe site
I am guessing(not sure yet) this could be the issue with cucumber integration as well. None of these frameworks work as they don't set up the proxy anywhere. Can someone point me in the right direction? if the proxy needs to be set up then where in the framework does it need to go- an example would be helpful?
TestCafe/Cucumber integrations rely on starting TestCafe runner programmatically.
In the repo, search for this sequence:
const runner = tc.createRunner();
return runner
.src('./test.js')
.screenshots('reports/screenshots/', true)
.browsers(browser)
.run()
.catch(function(error) {
console.error(error);
});
or search for this sequence:
await runner
.browsers(browsers)
.specs(specs)
.steps(steps)
.concurrency(concurrency)
.startApp(app, appInitDelay)
.tags(tags)
.run(...)
Chain the useProxy method on runner object (do it before the run()method):
const runner = tc.createRunner();
return runner
.src('./test.js')
.screenshots('reports/screenshots/', true)
.browsers(browser)
.useProxy('username:password#proxy.mycorp.com')
.run()
.catch(function(error) {
console.error(error);
});
I am trying to get the results of a php unit tests folder , but not using the CLI, and instead using a php file.
I would like to get the answer ( OK, 4 tests passed ) in a variable or something so I can decide whether the script should execute or not, what's the best way to do this? i don't want to use batch files, I want to force the execution of the tests, inside the library itself.
I started with
require_once 'PHPUnit/Autoload.php';
When I included the tests, don't know how to start them thought.
Any advice?
You can check PHPUnits exit code, 0 means no test failed.
To run the tests from a php file, then to check the exit code, try something like this:
//Composer installs phpunit to /vendor/bin/phpunit
exec('/vendor/bin/phpunit', $result, $exitCode);
if ($exitCode == 0) {
// continue exiting the script
} else {
// there was a test failure, more info will be in $result
}
If you're looking for an enterprise quality solution, look into a Continuous Integration product like Jenkins or Bamboo.
I want to write a grunt task, which will start a server, and run some tests based on that server:
grunt.initConfig({
shell: {
sbtRun: {
options: {
stdout: true
},
command: './sbt run'
}
}
});
grunt.loadNpmTasks('grunt-shell');
grunt.registerTask('run-test-only', ...);
grunt.registerTask('start-server-and-test', ['shell:sbtRun', 'run-test-only']);
But the problem is, the task ./sbt run is not running in daemon. When I run:
grunt start-server-and-test
It will blocking in the shell:sbtRun task forever.
Is there any way to start it in a child process? So the second task run-test-only will be called, and the server will be destroyed automatically after testing?
I was facing a similar issue. I wanted to connect to a database and store the result of a query in a .csv. I created a separate file to do all that work and I was successful in running that file through the command line using:
node filename.js
I used grunt-shell and grunt-exec to run this command but everytime I saw the same error:
const child = require('child_process').exec;
Then, I found another grunt plugin i.e grunt-execute which was able to create a child process and the task was completed successfully. You could find more about grunt-execute here, https://www.npmjs.com/package/grunt-execute.
Hope this helps.