Webdriver - Infinite Page loads on submit - webdriver

I am testing a login page. At certain times (read as intermittent) loading of the home page takes infinite time. In this case, the below command never completes -
driver.findElement(By.id("Login")).submit();
I am aware of both implicit & explicit waits.
These waits are applicable only for findElement or DOM related actions but not for submit command ( (Ref: https://sqa.stackexchange.com/questions/2606/what-is-seleniums-default-timeout-for-page-loading)
Debug Log Trace -
15 May 2013 16:42:08 DEBUG wire:77 - << "{"name":"submitElement","sessionId":"949f6c8f-a8fc-4e13-b4da-bf6c19c893fe","status":0,"value":""}"
15 May 2013 16:42:08 DEBUG DefaultClientConnection:152 - Connection shut down
15 May 2013 16:42:08 DEBUG ThreadSafeClientConnManager:272 - Released connection is not reusable.
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:434 - Releasing connection [HttpRoute[{}->http://127.0.0.1:7055]][null]
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:679 - Notifying no-one, there are no waiting threads
15 May 2013 16:42:08 DEBUG SalesForce:153 - Verify next page
15 May 2013 16:42:08 DEBUG ThreadSafeClientConnManager:221 - Get connection: HttpRoute[{}->http://127.0.0.1:7055], timeout = 120000
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:350 - [HttpRoute[{}->http://127.0.0.1:7055]] total kept alive: 0, total issued: 0, total allocated: 0 out of 2000
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:523 - No free connections [HttpRoute[{}->http://127.0.0.1:7055]][null]
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:369 - Available capacity: 2000 out of 2000 [HttpRoute[{}->http://127.0.0.1:7055]][null]
15 May 2013 16:42:08 DEBUG ConnPoolByRoute:549 - Creating new connection [HttpRoute[{}->http://127.0.0.1:7055]]
15 May 2013 16:42:08 DEBUG DefaultClientConnectionOperator:145 - Connecting to 127.0.0.1:7055
15 May 2013 16:42:08 DEBUG RequestAddCookies:132 - CookieSpec selected: best-match
15 May 2013 16:42:08 DEBUG RequestAuthCache:75 - Auth cache not set in the context
15 May 2013 16:42:08 DEBUG DefaultHttpClient:643 - Attempt 1 to execute request
15 May 2013 16:42:08 DEBUG DefaultClientConnection:264 - Sending request: GET /hub/session/949f6c8f-a8fc-4e13-b4da-bf6c19c893fe/title HTTP/1.1
15 May 2013 16:42:08 DEBUG wire:63 - >> "GET /hub/session/949f6c8f-a8fc-4e13-b4da-bf6c19c893fe/title HTTP/1.1[\r][\n]"
15 May 2013 16:42:08 DEBUG wire:63 - >> "Accept: application/json, image/png[\r][\n]"
15 May 2013 16:42:08 DEBUG wire:63 - >> "Cache-Control: no-cache[\r][\n]"
15 May 2013 16:42:08 DEBUG wire:63 - >> "Host: 127.0.0.1:7055[\r][\n]"
15 May 2013 16:42:08 DEBUG wire:63 - >> "Connection: Keep-Alive[\r][\n]"
15 May 2013 16:42:08 DEBUG wire:63 - >> "[\r][\n]"
15 May 2013 16:42:08 DEBUG headers:268 - >> GET /hub/session/949f6c8f-a8fc-4e13-b4da-bf6c19c893fe/title HTTP/1.1
15 May 2013 16:42:08 DEBUG headers:271 - >> Accept: application/json, image/png
15 May 2013 16:42:08 DEBUG headers:271 - >> Cache-Control: no-cache
15 May 2013 16:42:08 DEBUG headers:271 - >> Host: 127.0.0.1:7055
15 May 2013 16:42:08 DEBUG headers:271 - >> Connection: Keep-Alive
Basically DefaultClientConnection does not receive a 200 OK & the command just hangs.
Looking for a solution to close the browser in case of no response from the browser for a submit command after waiting for a specific duration.

I'm guessing the id "Login" is that of a form.
Could you instead select the form's submit button (assuming one exists) and call .click() on the button, rather than calling .submit() on the form?
And then you should be able to use implicit or explicit waits.
EDIT:
After submitting the form, you can make a custom method to wait for the response, and close the browser after timing out, if necessary.
int tryCount = 0;
boolean desiredResponseReceived = false;
while (desiredResponseReceived == false && tryCount < 20) {
String statusText = (String) js.executeScript("return xhr.statusText;");
if (statusText.indexOf("200") != -1) {
desiredResponseReceived = true;
}
else {
Thread.sleep(250);
tryCount++;
}
}
if (desiredResponseReceived == false) {
driver.quit();
}
As written, this method will close the browser if "200" doesn't appear in the xhr response within 5 seconds. (because it checks every 250 milliseconds, 20 times, before it finally reaches the if (desiredResponseReceived == false) and closes the browser).

Related

ASP.NET Core: Huge delay before HTTP request is handled

I have an ASP.NET Core application running on Ubuntu-server 18.04 on an Intel NUC.
The application receives commands and start up to 4 MPV video player processes on background threads.
When a controller is called, it basically writes a command in one of 4 command queues and returns. Then the command is consumed by a background thread which start/kill the process using System.Diagnostics.Process. So the controller operation should be really fast.
At the beginning of the middlewares chain I have a diagnostic middleware that measures the operation duration. This one always reports values near 3-4ms, which is what I expect as I only write in a queue
The problem is that, sometimes, the caller of the API reports times that can span up to several seconds.
I have a network capture showing that there is a huge delay between when the HTTP request is received by the OS and when it is handled by the application.
Logs of the POST requests made by the caller
1 Fr Aug 7 10:37:14 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;53;76
2 Fr Aug 7 10:37:17 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;54;101
3 Fr Aug 7 10:37:20 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;55;74
4 Fr Aug 7 10:37:23 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;56;47
5 Fr Aug 7 10:37:26 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;57;84
6 Fr Aug 7 10:37:29 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;58;67
7 Fr Aug 7 10:37:32 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;59;91
8 Fr Aug 7 10:37:35 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;60;57
9 Fr Aug 7 10:37:39 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;61;50
10 Fr Aug 7 10:37:42 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;62;71
11 Fr Aug 7 10:37:45 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;63;69
12 Fr Aug 7 10:37:49 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;64;948
13 Fr Aug 7 10:37:52 CEST 2020;rtsp://192.168.8.12/VideoInput/1/H264/1;65;66
14 Fr Aug 7 10:37:55 CEST 2020;rtsp://192.168.8.9/VideoInput/1/H264/1;66;56
The last number is the request time. We can see that most of the case it is < 100ms but here we have a value near 1000ms at line 12
TCP dump, where the slow request have been highlighted
We can see that the numbers are consistent with the caller.
Now, the logs from ASP.NET Core, where we can see that the request is logged approx 900ms later
[10:37:45.198 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request starting HTTP/1.1 POST http://192.168.8.200:9000/api/v1/video/affect?playerIndex=4 application/json 120
[10:37:45.198 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executing endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:45.199 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Route matched with {action = "AffectCamera", controller = "Player"}. Executing controller action with signature System.Threading.Tasks.Task`1[System.Boolean] AffectCamera(VideoPlayer.Controllers.AffectCameraInputDTO, Int32) on controller VideoPlayer.Controllers.PlayerController (VideoPlayer).
[10:37:45.201 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor - Executing ObjectResult, writing value of type 'System.Boolean'.
[10:37:45.201 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Executed action VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer) in 2.0618ms
[10:37:45.202 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executed endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:45.202 INF] VideoPlayer.Utils.RequestDiagnosticMiddleware - Request POST /api/v1/video/affect => 200 took 3ms -- IP source: 192.168.8.204
[10:37:45.202 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request finished in 4.2783ms 200 application/json; charset=utf-8
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[10:37:49.110 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request starting HTTP/1.1 POST http://192.168.8.200:9000/api/v1/video/affect?playerIndex=1 application/json 118
[10:37:49.111 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executing endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:49.112 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Route matched with {action = "AffectCamera", controller = "Player"}. Executing controller action with signature System.Threading.Tasks.Task`1[System.Boolean] AffectCamera(VideoPlayer.Controllers.AffectCameraInputDTO, Int32) on controller VideoPlayer.Controllers.PlayerController (VideoPlayer).
[10:37:49.115 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor - Executing ObjectResult, writing value of type 'System.Boolean'.
[10:37:49.115 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Executed action VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer) in 2.4829ms
[10:37:49.116 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executed endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:49.116 INF] VideoPlayer.Utils.RequestDiagnosticMiddleware - Request POST /api/v1/video/affect => 200 took 4ms -- IP source: 192.168.8.204
[10:37:49.117 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request finished in 6.534ms 200 application/json; charset=utf-8
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[10:37:49.127 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request starting HTTP/1.1 POST http://192.168.8.200:9000/api/v1/video/affect?playerIndex=2 application/json 118
[10:37:49.128 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executing endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:49.129 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Route matched with {action = "AffectCamera", controller = "Player"}. Executing controller action with signature System.Threading.Tasks.Task`1[System.Boolean] AffectCamera(VideoPlayer.Controllers.AffectCameraInputDTO, Int32) on controller VideoPlayer.Controllers.PlayerController (VideoPlayer).
[10:37:49.132 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor - Executing ObjectResult, writing value of type 'System.Boolean'.
[10:37:49.132 INF] Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker - Executed action VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer) in 2.6272ms
[10:37:49.133 INF] Microsoft.AspNetCore.Routing.EndpointMiddleware - Executed endpoint 'VideoPlayer.Controllers.PlayerController.AffectCamera (VideoPlayer)'
[10:37:49.133 INF] VideoPlayer.Utils.RequestDiagnosticMiddleware - Request POST /api/v1/video/affect => 200 took 5ms -- IP source: 192.168.8.204
[10:37:49.133 INF] Microsoft.AspNetCore.Hosting.Diagnostics - Request finished in 6.9064ms 200 application/json; charset=utf-8
Any idea what can cause this delay, how I can improve it or analyze it further?
EDIT
It seems it is somehow related with the fact I read the MPV process standard and error outputs (to get logs from the player)
I tried to redirect the output and not consume it and everything works fine. As soon as I try to consume the output the issue appears randomly. Even if I don't do any processing with this output.
I tried to read it using different methods
With the xxxDataReceived events
On a background thread using StandardOutput / StandardError properties
With ReadLine()
With Read() and a buffer
In all the cases lags appear.
If I redirect the output, but don't consume it or if I don't redirect the output, everything is fine.
The process doesn't output a lot. Less than 1 line/second

nginx response status variable

I have a question. We use nginx+uwsgi stack and see many errors like:
Aug 30 00:00:55 imfmce-va-81-2 uwsgi: Tue Aug 30 00:00:55 2016 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /provisioning/user/f205970b-6a9f-42b5-830f-c2bec9967b32 (ip 10.216.153.254) !!!
I understand that error occurs when client close connection before reading response or by uwsgi_read_timeout, but I don't understand why in access log I cannt see any error, nginx just log 200 OK:
Aug 30 00:00:55 imfmce-va-81-2 provisioning: active [ 55544 ] 10.216.153.254 Sync-Wopi-SyncLocksTask hostpilot 73af65e4-5984-4b2c-baf4-c88cf8385898 - ECDHE-RSA-AES256-GCM-SHA384 GET /provisioning/user/f205970b-6a9f-42b5-830f-c2bec9967b32 - 0,0,1,0 200 - 1 OK - 321 515 844
We use next format log line:
log_format ss_log_format "active\t[ \$pid ]\t\$remote_addr\t\$http_user_agent\t\$upstream_http_x_user_identity\t\$http_x_client_id\t\$http_x_request_id\t\$ssl_cipher\t\$request_method\t\$uri\t\$args\t\$upstream_http_x_durations\t\$status\t\$upstream_status\t\$http_x_error_code\t\$connection_requests\t\$request_completion\t\$content_length\t\$request_length\t\$body_bytes_sent\t\$bytes_sent";
I would like you to understand that,
we don't need to fix this error, we just need have right access logs.

AgileCRM return "HTTP/1.1 500 Internal Server Error" on every call

From the agile CRM doc, trying a few simple curl calls like this one (anonymised):
curl https://****.agilecrm.com/dev/api/contacts/*********** -H "Accept :application/xml" -v -u ********#***********:********************
And always same problem :
< HTTP/1.1 500 Internal Server Error
< Date: Mon, 29 Feb 2016 15:10:50 GMT
< Content-Type: text/html; charset=UTF-8
< Server: Google Frontend
< Content-Length: 323
< Alt-Svc: quic=":443"; ma=2592000; v="30,29,28,27,26,25"
<
Difficult to imagine any error in something that basic, but it doesn't work.
Any idea?
When registering from mobile, you often have 1st letter to be capitalized.
Then, in the agilecrm interface/backoffice, if you do not put the same cap at same place, you can't login. But if you put cap in your email in API call, the server crashes (Internal Server Error).
So beware, expected capitalisation isn't consistent across the different AgileCRM parts.
Authentication process at Agile CRM interface and Rest API level is same now. Now it is working with uppercase and lower case email with Rest API.

Email issues with Wordpress running on AWS Elastic Beanstalk

I'm running Wordpress 4.2 on AWS Elastic Beanstalk. I'm using a plugin that sends emails using the WP email API, which uses the server's (i.e. Elastic Beanstalk's) internal mail system.
The problem: emails that the app attempts to send to my university email (let's call it .uni.edu) fail, and they do work when sent to my standard gmail.
What's interesting is that I have no issues running the same application locally. The emails associated with the app send fine -- so obviously there's some issue with the way Elastic Beanstalk in particular is attempting to relay the messages to the .uni.edu server. Perhaps the .uni.edu server is picking it up as spam (when sent from Elastic Beanstalk, not when being sent through my local system) and bouncing it back, but I'm not experienced enough to diagnose this.
Does anyone have suggestions for either directly applying some kind of fix to this problem, or creating some sort of setup that is a workaround (e.g. setting something up on Elastic Beanstalk to send emails in a non-default way that is less likely to have issues working with the university email server)?
Here's a log from /var/mail:
From MAILER-DAEMON#ip-172-31-41-109.ec2.internal Fri Apr 24 21:39:52 2015
Return-Path: <MAILER-DAEMON#ip-172-31-41-109.ec2.internal>
Received: from localhost (localhost)
by ip-172-31-41-109.ec2.internal (8.14.4/8.14.4) id t3OLdq1k026047;
Fri, 24 Apr 2015 21:39:52 GMT
Date: Fri, 24 Apr 2015 21:39:52 GMT
From: Mail Delivery Subsystem <MAILER-DAEMON#ip-172-31-41-109.ec2.internal>
Message-Id: <201504242139.t3OLdq1k026047#ip-172-31-41-109.ec2.internal>
To: <webapp#ip-172-31-41-109.ec2.internal>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
boundary="t3OLdq1k026047.1429911592/ip-172-31-41-109.ec2.internal"
Subject: Returned mail: see transcript for details
Auto-Submitted: auto-generated (failure)
This is a MIME-encapsulated message
--t3OLdq1k026047.1429911592/ip-172-31-41-109.ec2.internal
The original message was received at Fri, 24 Apr 2015 21:39:42 GMT
from localhost [127.0.0.1]
----- The following addresses had permanent fatal errors -----
<univstudent#uni.edu>
(reason: 553 5.1.8 <webapp#ip-172-31-41-109.ec2.internal>... Domain of sender address webapp#ip-172-31-41-109.ec2.internal does not exist)
----- Transcript of session follows -----
... while talking to apathy.uni.edu.:
>>> MAIL From:<webapp#ip-172-31-41-109.ec2.internal> SIZE=1191
<<< 553 5.1.8 <webapp#ip-172-31-41-109.ec2.internal>... Domain of sender address webapp#ip-172-31-41-109.ec2.internal does not exist
501 5.6.0 Data format error
--t3OLdq1k026047.1429911592/ip-172-31-41-109.ec2.internal
Content-Type: message/delivery-status
Reporting-MTA: dns; ip-172-31-41-109.ec2.internal
Received-From-MTA: DNS; localhost
Arrival-Date: Fri, 24 Apr 2015 21:39:42 GMT
Final-Recipient: RFC822; univstudent#uni.edu
Action: failed
Status: 5.1.8
Diagnostic-Code: SMTP; 553 5.1.8 <webapp#ip-172-31-41-109.ec2.internal>... Domain of sender address webapp#ip-172-31-41-109.ec2.internal does not exist
Last-Attempt-Date: Fri, 24 Apr 2015 21:39:52 GMT
--t3OLdq1k026047.1429911592/ip-172-31-41-109.ec2.internal
Content-Type: message/rfc822
Return-Path: <webapp#ip-172-31-41-109.ec2.internal>
Received: from ip-172-31-41-109.ec2.internal (localhost [127.0.0.1])
by ip-172-31-41-109.ec2.internal (8.14.4/8.14.4) with ESMTP id t3OLdg1k026045
for <univstudent#uni.edu>; Fri, 24 Apr 2015 21:39:42 GMT
Received: (from webapp#localhost)
by ip-172-31-41-109.ec2.internal (8.14.4/8.14.4/Submit) id t3OLdgcq026044;
Fri, 24 Apr 2015 21:39:42 GMT
To: univstudent#uni.edu
Subject: [classifiedads] Password Reset
X-PHP-Originating-Script: 498:class-phpmailer.php
Date: Fri, 24 Apr 2015 21:39:42 +0000
From: WordPress <wordpress#classifiedads-test.elasticbeanstalk.com>
Message-ID: <bace4f13cdc099117cd4ac9c70e2531c#classifiedads-test.elasticbeanstalk.com>
X-Priority: 3
X-Mailer: PHPMailer 5.2.7 (https://github.com/PHPMailer/PHPMailer/)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Someone requested that the password be reset for the following account:
http://classifiedads-test.elasticbeanstalk.com/
Username: univstudentseas
If this was a mistake, just ignore this email and nothing will happen.
To reset your password, visit the following address:
<http://classifiedads-test.elasticbeanstalk.com/wp-login.php?action=rp&key=dzaUvJpcjAe243gNOysZ&login=univstudentseas>
--t3OLdq1k026047.1429911592/ip-179-35-41-109.ec2.internal--
This sounds very similar to another recent question about email:
http://stackoverflow.com/questions/29808262/unable-to-send-emails-from-linux/29810884#29810884
Does your ec2 instance have an associated external DNS? Is there a public IP associated with it. You'll need to apply some type of fix so that your outgoing email message has a domain other than webapp#ip-172-31-41-109.ec2.internal

HTTP streaming / chunked responses on Heroku with clojure

I'm making a clojure web app that streams data to clients using chunked HTTP responses. This works great when I run it locally using foreman, but doesn't work properly when I deploy it to Heroku.
A minimal example exhibiting this behaviour can be found on my github here. The frontend (in resources/index.html) performs an AJAX GET request and prints the response chunks as they arrive. The server uses http-kit to send a new chunk to connected clients every second. By design, the HTTP request never completes.
When the same code is deployed to Heroku, the HTTP connection is closed by the server immediately after the first chunk is sent. It seems to be Heroku's routing mesh which is causing this disconnection to occur.
This can also be seen by performing the GET request using curl:
$ curl -v http://arcane-headland-2284.herokuapp.com/stream
* About to connect() to arcane-headland-2284.herokuapp.com port 80 (#0)
* Trying 54.243.166.168...
* Adding handle: conn: 0x6c3be0
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x6c3be0) send_pipe: 1, recv_pipe: 0
* Connected to arcane-headland-2284.herokuapp.com (54.243.166.168) port 80 (#0)
> GET /stream HTTP/1.1
> User-Agent: curl/7.31.0
> Host: arcane-headland-2284.herokuapp.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Date: Sat, 17 Aug 2013 16:57:24 GMT
* Server http-kit is not blacklisted
< Server: http-kit
< transfer-encoding: chunked
< Connection: keep-alive
<
* transfer closed with outstanding read data remaining
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
The time is currently Sat Aug 17 16:57:24 UTC 2013 <-- this is the first chunk
Can anybody suggest why this is happening? HTTP streaming is supposed to be supported in Heroku's Cedar stack. The fact the code runs correctly using foreman suggests it is something in Heroku's routing mesh causing it to break.
Live demo of the failing project: http://arcane-headland-2284.herokuapp.com/
This was due to a bug in http-kit which will be fixed shortly.
https://devcenter.heroku.com/articles/request-timeout may be relevant: "long-polling" requests like yours have to send data every 55 seconds or be terminated.

Resources