Always times out at 10 seconds regardless of setting - webdriver

I am running a simple WebDriverIO script, and inserting any amount of async behaviour is making it time out at the 10 sec threshold (or before?). I want to control the timeout setting, but no matter what I try I cannot increase it.
As I am using ChromeDriver, not all Selenium settings are applicable, and setting browser.timeouts('implicit', 30000) (or script or pageLoad) will throw an error: unknown error: unknown type of timeout:pageLoad.
The only other timeouts I have found are
mochaOpts.timeout
waitforTimeout
This is my test:
it.only('should be able to register', ()=>{
// Mocha timeout
this.timeout(50000)
browser.url('/encounter/new');
browser.waitUntil( function() {
return browser.isExisting('[name=lastName]');
});
browser.setValue('#problem', 'something fishy');
// this is problematic: comment this out and everything works
// also works with very small timeouts
browser.executeAsync(function(done){
setTimeout(done, 1000);
});
browser.click('#appdetailsheader button');
console.log(browser.getUrl(), browser.log('browser'))
browser.waitUntil( function() {
return !browser.isExisting('[name=lastName]');
});
console.log(browser.getTitle(), browser.getUrl());
console.log(browser.log('browser'))
});

I can totally understand you frustration. WebdriverIO is extremely modular & configurable, but this comes with an increased level of complexity which often leads to confusion.
For this:
// Mocha timeout
this.timeout(50000);
!!! This has no effect because you are configuring/setting your Mocha timeout in an arrow function which is discouraged by Mocha. Read more about it here.
Solution (pick as applicable to your setup):
run your script with WebdriverIO test-runner and set mochaOpts: { timeout: <desiredTimeout>}, or you can even override it from your test run: wdio wdio.config.js --mochaOpts.timeout=<desiredTimeout>;
set your timeout in your root describe statement, or even better, in a before hook: before(function() { this.timeout(<desiredTimeout>); (...)});;
if you're running your test case using Mocha, pass the timeout either into your CLI command (if you're using it to run your tests): mocha yourTestFile.js --timeout <desiredTimeout>, or change it's value in your mocha.opts file;
Note: I'm sure there are even more ways to do this, but these are a few that worked for me.
For this:
browser.waitUntil( function() {
return browser.isExisting('[name=lastName]');
});
!!! This will always wait for the existence of the element with attribute name="lastName" for the default 1000 ms before timing out. This value can be changed via waitforTimeout.
Solution (pick as applicable to your setup):
explicitly give your waitUntil.../waitfor... commands the timeout: browser.waitUntil( function() { return browser.isExisting('[name=lastName]');}, <desiredTimeout>, <errorMessage>);;
run your script with WebdriverIO test-runner and set waitforTimeout: <desiredTimeout>, or you can even override it from your test run: wdio wdio.config.js --waitforTimeout=<desiredTimeout>;
Finally, I tried to run a few test cases with obscene timeout values (50000 ms) and it worked as expected for every one of the issues you mentioned above.
waitforTimeout example:
Logs (1 failing (57s)):
[chrome #0-0] ConnectWeb Devices Content Test Suite
[chrome #0-0] 1) "before all" hook
[chrome #0-0]
[chrome #0-0]
[chrome #0-0] 1 failing (57s)
[chrome #0-0]
[chrome #0-0] 1) ConnectWeb Devices Content Test Suite "before all" hook:
[chrome #0-0] Oups! An error occured.
Timed out waiting for element ('span[connectqa-device="events"]') to exist
Note: I've never used Selenium timeouts in WebdriverIO before (implicit, pageLoad, script), but I never had this necessity before as waitforTimeout & Mocha's timeout have been more than effective for my testing scenarios.
Small mention: This statement inserting any amount of async behaviour is making it time out at the 10 sec threshold is not true. First off, WDIO is completely asynchronous. You might be using the sync: true flag, but behind the scenes, everything is still async.
This is a vast topic and I tried to cover as much as possible given the information at hand. Sorry if I didn't completely answer your question. Tell me in the comments and I'll try to update the answer with the relevant info.
Hope it helps. Cheers!

Related

How do I compare the visual differences of an element on a page using CodeceptJS?

I have just started to use CodeceptJS and I'm having some trouble getting the seeVisualDiffForElement command working.
This is my test code so far:
Feature('MyTest');
Scenario('Creating base images and comparing using the same command but ran twice', (I) => {
I.amOnPage('https://testsite-90d35.firebaseapp.com/');
I.wait(1);
I.saveScreenshot("Test1.png");
I.wait(1);
I.seeVisualDiff("Test1.png", {tolerance: 0, prepareBaseImage: true});
});
Scenario ('Comparing the blue button on the page', async (I) => {
I.amOnPage('https://testsite-90d35.firebaseapp.com/testPage.html');
I.wait(1);
I.saveScreenshot("Test1.png");
I.wait(1);
I.seeVisualDiffForElement(".btnBlue","Test1.png", {tolerance: 0, prepareBaseImage: false});
});
When I run this code it opens the Chromium browser (as I'm using it with Puppeteer) and runs through the first scenario just fine. Takes the screenshot and saves it as a base image. Then I have to close the browser for it to run the next scenario. After the 2nd scenario has ran it fails the test but still creates the screenshots. The screenshots are different as they should be but the diff screenshot doesn't have any outlined changes.
In the documentation for CodeceptJS it states "seeVisualDiffForElement only works when the page for baseImage is open in the browser, so that webdriver can fetch coordinates of the provided selector". My browser is open but not the same browser that created the base image (as I have to close it to run the 2nd scenario). Could this be why the test fails and doesn't recognise any changes? Or can anyone see something else I'm doing wrong?
I've tried to run the tests as a single scenario as well but it gives me the same output.
I was informed that element testing only works with WebDriver, but I was using Puppeteer, hence the errors.

Why is Puppeteer failing simple tests with: "waiting for function failed: timeout 500ms exceeded"?

While trying to set up some simple end-to-end tests with Jest and Puppeteer, I've found that any test I write will inexplicably fail with a timeout.
Here's a simple example test file, which deviates only slightly from Puppeteer's own example:
import puppeteer from 'puppeteer';
describe('Load Google Puppeteer Test', () => {
test('Load Google', async () => {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
await page.goto('https://google.co.uk');
await expect(page).toMatch("I'm Feeling Lucky");
await browser.close();
});
});
And the response it produces:
TimeoutError: Text not found "I'm Feeling Lucky"
waiting for function failed: timeout 500ms exceeded
I have tried adding in custom timeouts to the goto line, the test clause, amongst other things, all with no effect. Any ideas on what might be causing this? Thanks.
What I would say is happening here is that using toMatch expects text to be displayed. However, in your case, the text you want to verify is text associated with a button.
You should try something like this:
await expect(page).toMatchElement('input[value="I\'m Feeling Lucky"]');
Update 1:
Another possibility (and it's one you've raised yourself) is that the verification is timing out before the page has a chance to load. This is a common issue, from my experience, with executing code in headless mode. It's very fast. Sometimes too fast. Statements can be executed before everything in the UI is ready.
In this case you're better off adding some waitForSelector statements throughout your code as follows:
await page.waitForSelector('input[value="I\'m Feeling Lucky"]');
This will ensure that the selector you want is displayed before carrying on with the next step in your code. By doing this you will make your scripts much more robust while maintaining efficiency - these waits won't slow down your code. They'll simply pause until puppeteer registers the selector you want to interact with / verify as being displayed. Most of the time you won't even notice the pause as it will be so short (I'm talking milliseconds).
But this will make your scripts rock solid while also ensuring that things won't break if the web page is slower to respond for any reason during test execution.
You're probably using 'expect-puppeteer' package which does the toMatch expect. This is not a small deviation. The weird thing is that your default timeout isn't 30 seconds as the package's default, check that.
However, to fix your issue:
await expect(page).toMatch("I'm Feeling Lucky", { timeout: 6000 });
Or set the default timeout explicitly using:
page.setDefaultTimeout(timeout)
See here.

Sometimes Initialization of Google Earth Plugin fails in IE10

This is my code for the initialization of google earth plugin.
Sometimes Initialization of Google Earth Plugin fails in IE10(I have it in compatability mode) IE7 Standards. This error happens only in IE and no other browser.
90% of the time createInstance() method creates the google earth plugin instance and control goes to mygeeEarthPluginInitCb() method but few times mostly after restarting the machine or after few hours of inactivity if I load the page createInstance fails and control goes to geeEarthPluginFailureCb() method.
This is causing an error page, a very intermittent one.
function geeInit() {
alert("google.earth.createInstance : Start");
google.earth.createInstance(geeDivIds.map, mygeeEarthPluginInitCb,
geeEarthPluginFailureCb, earthArgs);
alert("google.earth.createInstance : End");
}
function mygeeEarthPluginInitCb(object) {
alert("Success mygeeEarthPluginInitCb: Inside");
geeEarthPluginInitCb(object);
gex = new GEarthExtensions(ge);
createSearchResultsMarkers(null, 'results');
var lookAt = ge.createLookAt('');
lookAt.setLongitude(Number('-73.784190'));
lookAt.setLatitude(Number('42.643446'));
lookAt.setRange(25000.00);
ge.getView().setAbstractView(lookAt);
initRadSearchValsOnLoad();
}
function geeEarthPluginFailureCb(message) {
alert("Failure geeEarthPluginFailureCb: Inside" + message);
if (google.earth.isInstalled()) {
} else {
var result = confirm('Google Earth Plugin is not'
+ ' installed.Please download and install it.');
if (result == true) {
window.location.href = 'install.html';
}
}
}
Remove all the alert lines, e.g.
alert("google.earth.createInstance : Start");
and
alert("google.earth.createInstance : End");
alert is a special method that blocks execution and user interaction - it could well be that it is blocking the initialisation of the plugin. This is something I have seen before.
Perhaps try using the console, or else outputting data to the document in some way that avoids blocking. e.g.
console && console.log("google.earth.createInstance, "End");
Google acknowledged the issue and mentioned they are working on a fix.
For right now there is a temporary fix below is the shorter version of Google's response.
******** Start Google's Response *************
"We have been able to reproduce this issue, intermittently. It is now pending additional investigation for the Google Earth client team, to find the root cause here. Unfortunately, it is not possible to provide an estimate for a deadline when this will be fixed. This issue definitely has a high priority since it impacts all Google Earth users with custom globes (GEE, and GME), and we have let the team know that this is now critical for your applications.
The only workaround that we can see, right now, is to refresh the page when the plugin fails to load (or you could do that programmatically: implement a timeout, and if after 5 seconds, the Earth API has not yet loaded, reload the plugin, or refresh the page). You could also consider using the Google Earth client, but I'm not sure if this is something that would be applicable to your use case."
**********End Google's Response ***************

Google Chrome restores session cookies after a crash, how to avoid?

On Google Chrome (I saw this with version 35 on Windows 8.1, so far I didn't try other versions) when browser crashes (or you simply unplug power cable...) you'll be asked to recover previous session when you'll open it again. Good feature but it will restore session cookies too.
I don't want to discuss here if it's a bug or not anyway IMO it's a moderate security bug because a user with physical access to that machine may "provoke" a crash to stole unclosed sessions with all their content (you won't be asked to login again).
Finally my question is: how a web-site can avoid this? If I'm using plain ASP.NET authentication with session cookies I do not want they survive to a browser crash (even if computer is restarted!).
There is not something similar to a process ID in the User Agent string and JavaScript variables are all restored (so I can't store a random seed, generated - for example - server side). Is there anything else viable? Session timeout will handle this but usually it's pretty long and there will be an unsafe window I would eliminate.
I didn't find anything I can use as process id to be sure Chrome has not been restarted but there is a dirty workaround: if I setup a timer (let's say with an interval of five seconds) I can check how much time elapsed from last tick. If elapsed time is too long then session has been recovered and logout performed. Roughly something like this (for each page):
var lastTickTime = new Date();
setInterval(function () {
var currentTickTime = new Date();
// Difference is arbitrary and shouldn't be too small, here I suppose
// a 5 seconds timer with a maximum delay of 10 seconds.
if ((currentTickTime - lastTickTime) / 1000 > 10) {
// Perform logout
}
lastTickTime = currentTickTime;
}, 5000);
Of course it's not a perfect solution (because a malicious attacker may handle this and/or disable JavaScript) but so far it's better than nothing.
New answers with a better solution are more than welcome.
Adriano's suggestion makes is a good idea but the implementation is flawed. We need to remember the time from before the crash so we can compare it to the time after the crash. The easiest way to do that is to use sessionStorage.
const CRASH_DETECT_THRESHOLD_IN_MILLISECONDS = 10000;
const marker = parseInt(sessionStorage.getItem('crashDetectMarker') || new Date().valueOf());
const diff = new Date().valueOf() - marker;
console.log('diff', diff)
if (diff > CRASH_DETECT_THRESHOLD_IN_MILLISECONDS) {
alert('log out');
} else {
alert ('ok');
}
setInterval(() => {
sessionStorage.setItem('crashDetectMarker', new Date().valueOf());
}, 1000)
To test, you can simulate a Chrome crash by entering chrome://crash in the location bar.
Don't forget to clear out the crashDetectMarker when the user logs out.

Unit testing with Symfony's flashBag sessions

I'm creating an application in Silex with unit tests.
Running unit tests works fine against the regular session handler:
$app->register(new Silex\Provider\SessionServiceProvider(), array(
'session.storage.options' => array(
'cookie_lifetime' => 1209600, // 2 weeks
),
));
and setting this flag in my unit tests:
$this->app['session.test'] = true;
If I don't set that session.test flag, my unit tests throw a headers already sent error and all fail. With it on, my tests run well.
The issue is I am attempting to use the flashBag feature (session info that lasts only until first request then get removed):
$foo = $app['session']->getFlashBag()->all();
The flashBag does not seem to respect the session.test flag, and attempts to send headers, which cause all my unit tests to fail:
24)
Yumilicious\UnitTests\Validator\PersonAccountTest::setConstraintsPassesWithMinimumAttributes
RuntimeException: Failed to start the session because headers have
already been sent.
/webroot/yumilicious/vendor/symfony/http-foundation/Symfony/Component/HttpFoundation/Session/Storage/NativeSessionStorage.php:142
/webroot/yumilicious/vendor/symfony/http-foundation/Symfony/Component/HttpFoundation/Session/Storage/NativeSessionStorage.php:262
/webroot/yumilicious/vendor/symfony/http-foundation/Symfony/Component/HttpFoundation/Session/Session.php:240
/webroot/yumilicious/vendor/symfony/http-foundation/Symfony/Component/HttpFoundation/Session/Session.php:250
/webroot/yumilicious/src/app.php:38
/webroot/yumilicious/tests/Yumilicious/UnitTests/Base.php:13
/webroot/yumilicious/vendor/silex/silex/src/Silex/WebTestCase.php:34
/webroot/yumilicious/vendor/EHER/PHPUnit/src/phpunit/phpunit.php:46
/webroot/yumilicious/vendor/EHER/PHPUnit/bin/phpunit:5
I've narrowed it down to this bit of code: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpFoundation/Session/Storage/NativeSessionStorage.php#L259
Specifically, line 262. Commenting out that single line allows my tests to work properly and all pass green.
I've searched quite a bit to get this to work, but am not having any luck. I think it's because the flashBag stuff is new (https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpFoundation/Session/Session.php#L305) and the old methods are being deprecated.
Any suggestions on getting my unit tests to work would be awesome.
For testing you need to replace the session.storage service with an instance of MockArraySessionStorage:
use Symfony\Component\HttpFoundation\Session\Storage\MockArraySessionStorage;
$app['session.storage'] = new MockArraySessionStorage();
This is because the native one tries to send a cookie via header which of course fails in a test environment.
EDIT: There is now a session.test parameter that you should set to true. That will automatically make the session use a mock storage.
I had this happen too, if i am not mistaking i fixed by having my unittests run via a different environment, wich has
framework:
test: ~
session:
storage_id: session.storage.mock_file
set in the config_test.yml
I came across similar problem today and temp fix would be to comment out block of code in
\Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage
in start() method
/*
if (ini_get('session.use_cookies') && headers_sent()) {
throw new \RuntimeException('Failed to start the session because headers have already been sent.');
}
*/
This solution keeps tests "green" and from looks of it the application session functionality as is.

Resources