I'm requesting a HTML report from an OpenVAS server the following way:
<?xml version="1.0" encoding="utf-8"?>
<commands><authenticate><credentials><username>***</username><password>***</password></credentials></authenticate><get_reports report_id="454a3397-b8a4-408e-9f55-f08972765d30" format_id="b993b6f5-f9fb-4e6e-9c94-dd46c00e058d"/></commands>
However when I read the response I receive only 8192 bytes, which happens not to hold all the HTML code. Is this is a hardcoded limit? How do I get around it?
I have found a similar issue. The cause was related to running
openvas-mkcert-client -n admin -i
Note the trailing -i, this is what causes the break. I have found no solution.
Related
Background
We run a kubernetes cluster that handles several php/lumen microservices. We started seeing the app php-fpm/nginx reporting 499 status code in it's logs, and it seems to correspond with the client getting a blank response (curl returns curl: (52) Empty reply from server) while the applications log 499.
10.10.x.x - - [09/Mar/2020:18:26:46 +0000] "POST /some/path/ HTTP/1.1" 499 0 "-" "curl/7.65.3"
My understanding is nginx will return the 499 code when the client socket is no longer open/available to return the content to. In this situation that appears to mean something before the nginx/application layer is terminating this connection. Our configuration currently is:
ELB -> k8s nginx ingress -> application
So my thoughts are either ELB or ingress since the application is the one who has no socket left to return to. So i started hitting ingress logs...
Potential core problem?
While looking the the ingress logs i'm seeing quite a few of these:
2020/03/06 17:40:01 [crit] 11006#11006: ngx_slab_alloc() failed: no memory in vhost_traffic_status_zone "vhost_traffic_status"
Potential Solution
I imagine if i gave vhost_traffic_status_zone some more memory at least that error would go away and on to finding the next error.. but I can't seem to find any configmap value or annotation that would allow me to control this. I've checked the docs:
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/
Thanks in advance for any insight / suggestions / documentation I might be missing!
here is the standard way to look up how to modify the nginx.conf in the ingress controller. After that, I'll link in some info on suggestions on how much memory you should give the zone.
First start by getting the ingress controller version by checking the image version on the deploy
kubectl -n <namespace> get deployment <deployment-name> | grep 'image:'
From there, you can retrieve the code for your version from the following URL. In the following, I will be using version 0.10.2.
https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.10.2
The nginx.conf template can be found at rootfs/etc/nginx/template/nginx.tmpl in the code or /etc/nginx/template/nginx.tmpl on a pod. This can be grepped for the line of interest. I the example case, we find the following line in the nginx.tmpl
vhost_traffic_status_zone shared:vhost_traffic_status:{{ $cfg.VtsStatusZoneSize }};
This gives us the config variable to look up in the code. Our next grep for VtsStatusZoneSize leads us to the lines in internal/ingress/controller/config/config.go
// Description: Sets parameters for a shared memory zone that will keep states for various keys. The cache is shared between all worker processe
// https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_zone
// Default value is 10m
VtsStatusZoneSize string `json:"vts-status-zone-size,omitempty"
This gives us the key "vts-status-zone-size" to be added to the configmap "ingress-nginx-ingress-controller". The current value can be found in the rendered nginx.conf template on a pod at /etc/nginx/nginx.conf.
When it comes to what size you may want to set the zone, there are the docs here that suggest setting it to 2*usedSize:
If the message("ngx_slab_alloc() failed: no memory in vhost_traffic_status_zone") printed in error_log, increase to more than (usedSize * 2).
https://github.com/vozlt/nginx-module-vts#vhost_traffic_status_zone
"usedSize" can be found by hitting the stats page for nginx or through the JSON endpoint. Here is the request to get the JSON version of the stats and if you have jq the path to the value: curl http://localhost:18080/nginx_status/format/json 2> /dev/null | jq .sharedZones.usedSize
Hope this helps.
I downloaded the torrent file of ubuntu 17.10 from here:
https://www.ubuntu.com/download/alternative-downloads
Here is what inside:
TorrentInfo{Created By: null
Main tracker: http://torrent.ubuntu.com:6969/announce
Comment: Ubuntu CD releases.ubuntu.com
Info_hash: f07e0b0584745b7bcb35e98097488d34e68623d0
Name: ubuntu-17.10.1-desktop-amd64.iso
Piece Length: 524288
Pieces: 2866
Total Size: 1502576640
Is Single File Torrent: true
File List:
Tracker List:
http://torrent.ubuntu.com:6969/announce
http://ipv6.torrent.ubuntu.com:6969/announce
What I have tried:
I sent: (Only torrent info-hash)
http://torrent.ubuntu.com:6969/announce?info_hash=%f0%7e%0b%05%84%74%5b%7b%cb%35%e9%80%97%48%8d%34%e6%86%23%d0
and received:
you sent me garbage - id not of length 20
I sent: (torrent info-hash and my peer-id)
http://torrent.ubuntu.com:6969/announce?info_hash=%f0%7e%0b%05%84%74%5b%7b%cb%35%e9%80%97%48%8d%34%e6%86%23%d0&peer_id=%2D%41%5A%35%37%35%30%2D%54%70%6B%58%74%74%5A%4C%66%70%53%48
and received:
you sent me garbage - invalid literal for long() with base 10: ''
What am I missing? The spec doesn't specify any example.
Spec:
https://wiki.theory.org/index.php/BitTorrentSpecification#Tracker_HTTP.2FHTTPS_Protocol
The announce misses the obligatory keys port, uploaded, downloaded and left.
These keys plus info_hash and peer_id, MUST be in every announce.
Further, while the event key isn't obligatory in every announce,
the first announce to the tracker MUST include 'event=started'.
Trying:
http://torrent.ubuntu.com:6969/announce?info_hash=%f0%7e%0b%05%84%74%5b%7b%cb%35%e9%80%97%48%8d%34%e6%86%23%d0&peer_id=%2D%41%5A%35%37%35%30%2D%54%70%6B%58%74%74%5A%4C%66%70%53%48&port=6881&uploaded=0&downloaded=0&left=1502576640&event=started
and the tracker responses with:
your client is outdated, please upgrade
oh well, more to fix...
From my answer here: Why does tracker server NOT understand my request? (Bittorrent protocol)
It is because the request string don't have compact=1 in it.
Most tracker require that nowadays. The legacy way is too ineffective.
So, adding compact=1 to the announce:
http://torrent.ubuntu.com:6969/announce?info_hash=%f0%7e%0b%05%84%74%5b%7b%cb%35%e9%80%97%48%8d%34%e6%86%23%d0&peer_id=%2D%41%5A%35%37%35%30%2D%54%70%6B%58%74%74%5A%4C%66%70%53%48&port=6881&uploaded=0&downloaded=0&left=1502576640&event=started&compact=1
and the tracker responses with:
d8:completei2134e10:incompletei100e8:intervali1800e5:peers300:[ binary data ... ]e
Success!
I have a problem to make IIS to allow download file with Bitrate Throottling. I set limit file to 100kb/s. There is no problem without bitrate limitation. But with limit I have a problem.
I'm using a code similar to described in this article:
Securing Large Downloads Using C# and IIS 7
I also tried to switch off IIS Bitrate Throottling and control bitrate "by hand" calculating with TimeSpan the bitrate and using Thread.Sleep(10) in a while...
But all my tries was useless, I don't get any exceptions.
to test download I use wget, this way:
wget -t 1 http://db.realestate.ru/yrl/RealEstateExportToYandex.xml
(you can try it with wget for windows)
this is a 240Mb text file, wget always stops, at random position of downloading, 5% - 60% and throws this error message:
Read error at byte ... (Connection reset by peer).
May be the problem is not with IIS, because in may localhost is working well, but not online on highly loaded server.
Solved with this parameters specified in wget command:
wget -t 1 --header="Keep-Alive: 30000" -nv http://db.realestate.ru/yrl/RealEstateExportToYandex.xml
Is there a command line utility where you can simply set up an HTTP request and have the trace simply output back to the console?
Also specifying the method simply would be a great feature instead of the method being a side effect.
I can get all the information I need with cURL but I can't figure out a way to just display it without dumping everything to files.
I'd like the output to show the sent headers the received headers and the body of the message.
There must be something out there but I haven't been able to google for it. Figured I should ask before going off and writing it myself.
I dislike answering my own question but c-smile's answer lead me down the right track:
Short answer shell script over cURL:
curl --dump-header - "$#"
The - [dash] meaning stdout is a convention I was unaware of but also works for wget and a number of other unix utilities. It is apparently not part of the shell but built into each utility. The wget equivalent is:
wget --save-headers -qO - "$#"
Did you try wget:
http://www.gnu.org/software/wget/manual/wget.html#Wgetrc-Commands ?
Like wget --save-headers ...
To include the HTTP headers in the output (as well as the server response), just use curl’s -i/--include option. For example:
curl -i "http://www.google.com/"
Here’s what man curl says about this setting:
-i/--include
(HTTP) Include the HTTP-header in the output. The HTTP-header
includes things like server-name, date of the document, HTTP-
version and more...
If this option is used twice, the second will again disable
header include.
Try http, e.g.
http -v example.org
Further into at https://httpie.org
It even includes a page to try online:
https://httpie.org/run
Telnet has for long been a well-known (though now forgotten, I guess) tool for looking at a web page. The general idea is to telnet to the http port, to type an http 1.1 GET command, and then to see the served page on the screen.
A good detailed explanation is http://support.microsoft.com/kb/279466
A Google search yields a whole bunch more.
Use telnet on port 80
For example:
telnet telehack.com 80
GET / HTTP/1.1
host: telehack.com
<CR>
<CR>
<CR> means Enter
When trying to run the Flex Builder 3 profiler on any I don't get the profiler dialog window and then after a few seconds I get "Socket timeout" in the console window. Any ideas why it can't connect?
I've got the latest debug version of Flash player and have tried shutting off my firewall.
I'm running it on XP from the local drive, ie. not through localhost.
Thanks,
Alex
It looks like the browser (Firefox in my case) has to be shutdown before the profiler is started. Step 1. in the livedocs even says this -- wish I had read it earlier. :)
http://livedocs.adobe.com/flex/3/html/help.html?content=profiler_3.html
Make sure that your firewall does not block port 9999, you can customize the port number too: Open Preferences->Flex->Profiler->Connections.
Check /etc/hosts (C:\Windows\System32\drivers\etc\hosts), and see if it contains a line:
127.0.0.1 localhost
In my case, it was somehow changed to ::1 localhost, and that's why it stopped working.
Thanks to Ziv for the (poorly formatted) answer.
While I try to run my Flex Profiler I got this error message:
In the flash application I got the following exception:
Error #2044: Unhandled securityError:. text=Error #2048: Security sandbox violation:
file:///C|%2Fwork%2Flabsense%2Fbranches%2Frel%5F1%5F2%5F5%5FEA%2Fsources%2Fui%2F.metadata%2F.plugins%2Fcom.adobe.flash.profiler%2FProfilerAgent.swf?host=localhost&port=9999
cannot load data from localhost:9999.
at ProfilerAgent()[C:\SVN\branches\3.2.0\modules\profiler3\as\ProfilerAgent.as:127]
And in the flex Profiler console (at the eclipse) I got : Socket timeout.
I am run on windows vista,
Flex builder: 3.2
Flash debugger: 10,0,22,87
Things that I have done to resolve this issue:
Switch the connection port of the profiler to 9998 (and back)
Remove and reinstall the flash debugger player.
Install flex builder 3.2 (instead of 3.0)
Delete all the enters in the mm.cfg file
Add enter to the mm.cfg:
PreloadSwf=C:\work\labsense\Sources\ui\.metadata\.plugins\com.adobe.flash.profiler\ProfilerAgent.swf?host=localhost&port=9999
or
PreloadSwf=C:\work\labsense\Sources\ui\.metadata\.plugins\com.adobe.flash.profiler\ProfilerAgent.swf?host=localhost&port=9998
or
PreloadSwf=C:/work/labsense/Sources/ui/.metadata/.plugins/com.adobe.flash.profiler/ProfilerAgent.swf?host=localhost&port=9999
or with spaces:
PreloadSwf=C: \ work \ labsense \ Sources \ ui \ .metadata \ .plugins \ com.adobe.flash.profiler \ ProfilerAgent.swf?host=localhost&port=9999
or
C:\work\labsense\Sources\ui\.metadata\.plugins\com.adobe.flash.profiler\ProfilerAgent.swf?
or add all or some of the enters:
TraceOutputFileName=C:\Users\zivo\AppData\Roaming\Macromedia\Flash Player\Logs\flashlog.txt
ErrorReportingEnable=1
MaxWarnings=0
TraceOutputFileEnable=1
ProfilingFileOutputEnable=1
Turn on and off the vista firewall
Add exception for port 9999 in the vista firewall
Try to run the profiler SWF separately
Same result.
Try one last thing:
Because I have expreins problem little bit similar before with the flash debugger, the resolution then was:
Right click on flash player (debugger),
choose “Debugger”,
choose “other machine”
add “127.0.0.1”
click ok
then, it solve the issue (but apparently he connect to the debugger with host 127.0.0.1 instead of localhost (which is a same)
I now add to the mm.cfg file, the follow entry:
PreloadSwf=C:/work/labsense/branches/rel_1_2_5_EA/sources/ui/.metadata/.plugins/com.adobe.flash.profiler/ProfilerAgent.swf?host=127.0.0.1&port=9999
Then, after saving, I run the profiler, and its work!!
And the reasons for all this was:
Some program change the file C:\Windows\System32\drivers\etc\hosts to:
# Copyright (c) 1993-2006 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
::1 localhost
127.0.0.1 iDBO # LMS GENERATED LINE
This means that localhost is not lead to 127.0.0.1!!!
Fixing is easy:
# ::1 localhost
# 127.0.0.1 iDBO # LMS GENERATED LINE
127.0.0.1 localhost
Instead (remark the problem and fix the problem
After trying all the other suggestions here, this post on Adobe's forum clued me in.
Adobe forum
When the Flash debug player launches, it looks for mm.cfg in %HOMEDRIVE%%HOMEPATH%. On this particular computer that path is not my home directory on C: but on the file server mapped to I:. So once I created I:\mm.cfg with the contents
PreloadSwf=C:\Users\ehedstrom\Documents\FLEXBU~1\.metadata\.plugins\com.adobe.flash.profiler\ProfilerAgent.swf?host=localhost&port=9999
everything magically started working!
Browser tabs, make sure you have latest debug as you said you did, also make sure that the port is correct, for some reason the port sometimes changes(1001 or 20957) from the default 9999, be sure that your mm.cfg has ProfilingFileOutputEnable=1 and that bittorrent isn't on.
hth