I'm searching for the syntax to write a PUT operation that upload a file with HTTPie. Please could you point me to the right syntax ? I could not find a way to do so on the official documentation
To achieve this with httpie, you need to do two things:
Set the HTTP method to PUT, which is trivial: $ http PUT […]
Pass the contents of the file, for which there are various ways:
Redirected input:
$ http PUT httpbin.org/put Content-Type:image/png < /images/photo.png
Request data from a filename (automatically sets the Content-Type header):
$ http PUT httpbin.org/put #/images/photo.png
Form file upload:
$ http --form PUT httpbin.org/put photo=#/images/photo.png
Related
In the response handling section
if below is the overall response
[
{
"empId":1001,
"empName":"abc"
},
{
"empId":1002,
"empName":"xyz"
}
]
I am able to get this response as
> {%
console.log(response.body);
%}
Is there any way to write this response to file on a local file system?
Also, we seem to have access only to client and response objects.
Can we also write control structures such as for loop, etc.?
You can redirect a response to a file. Use >> to create a new file with a suffix if it already exists and >>! to rewrite the file if it exists. You can specify an absolute path or relative to the current HTTP Request file. You can also use variables in paths, including environment variables and the following predefined variables:
{{$projectRoot}} points to the project root: .idea
{{$historyFolder}} points to .idea/httpRequests/
The following example HTTP request creates myFile.json in myFolder next to the HTTP Request file and redirects the response to it. If the file already exists, it creates myFile-1.json and so on.
POST https://httpbin.org/post
Content-Type: application/json
{
"id": 999,
"value": "content"
}
>> myFolder/myFile.json
ref:
https://www.jetbrains.com/help/idea/exploring-http-syntax.html#response-redirect
https://www.jetbrains.com/help/idea/http-response-handling-api-reference.html
Not possible yet, here's a link to a corresponding feature request: https://youtrack.jetbrains.com/issue/IDEA-239333. You can vote/comment it to receive updates.
Edit: this is now possible, see PhpStorm docs on that topic.
We're got a pre-existing rsyslog config file which is working for papertrail e.g.
/etc/rsyslog.d/20-papertrail.conf which has
*.* #logs4.papertrailapp.com:44407
However we've got a couple of NGINX websites on the server so would like to have it also monitor their error logs.
The paths to them are:
/var/log/nginx/www.website-one.com-error.log
/var/log/nginx/www.website-two.com-error.log
/var/log/nginx/www.website-three.com-error.log
However this /var/log/nginx also contains a bunch of .log files which we do not want to monitor e.g.
/var/log/nginx/error.log
/var/log/nginx/access.log
/var/log/nginx/error.log1
/var/log/nginx/nginx.log
In my head we need to add something like...
/var/log/nginx/*-error.log
And make sure they pipe to the papertrail url as well.
However I'm struggling to decipher the rsyslog documentation to figure out how to do this.
Thanks!
In rsyslog documentation it seems that you can use wildcards in files.
File
The file being monitored. So far, this must be an absolute name (no macros or templates). Note that wildcards are supported at the file name level (see WildCards below for more details).
WildCards
Before Version: 8.25.0
Wildcards are only supported in the filename part, not in directory names.
/var/log/*.log works.
/var/log/*/syslog.log does not work.
Since Version: 8.25.0
Wildcards are supported in filename and paths which means these samples will work:
/var/log/*.log works.
/var/log/*/syslog.log works.
/var/log/*/*.log works.
All matching files in all matching subfolders will work. Note that this may decrease performance in imfile depending on how many directories and files are being watched dynamically.
If you want to forward your vhosts logs you can change configuration directly in NGINX vhosts configuration, you should change/add access_log and error_log policies as explained here or use custom facilities to forward your logs (using rsyslog).
HOW TO DO IT USING RSYSLOG?
Create a new custom file in /etc/rsyslog.d/nginx_custom.conf:
module(load="imfile" PollingInterval="1") #needs to be done just once
# File 1
input(type="imfile"
File="/var/log/nginx/www.website-*.com-error.log"
Tag="websites"
Facility="local0")
local0.* #logs4.papertrailapp.com:44407
#Just to test that logs are forwarded, comment the line once you've tested it
local0.* /var/log/test.log
And restart rsyslog service
NOTE: Line local0.* /var/log/test.log is just to test that you can see forwarded logs into your local server, comment this line after you've tested that everything works.
I got a brand new empty installation of Elastic Search 6.1.1 running on port 9200 of my CentOS7 host. I have this "es.json" file with data sample ready to be inserted in the ES.
I also have a log file in which each line is a json chunk. I could use both to populate my ES database.
How do I insert this and other data to ES?
I didnt find good explanations on the documentation and on the internet. for some reason the docs does not make it very clear
Use the --data-binary flag in curl to bulk import from a JSON file.
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/{index}/{type}/_bulk?pretty' --data-binary #es.json
Data can be posted to one of the endpoints - /_bulk, /{index}/_bulk or {index}/{type}/_bulk. When {index} or {index}/{type} is provided, they will be used by default on bulk items that don’t provide them explicitly.
Content-Type: application/x-ndjson stands for Newline delimited JSON.
Before importing the JSON file, you might want to define mappings yourself or let Elasticsearch generate mappings dynamically during import. If you don't want Elasticsearch to generate mappings dynamically during import, refer to this doc to define mappings yourself.
References:
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/docs-bulk.html
https://www.elastic.co/guide/en/kibana/6.1/tutorial-load-dataset.html
You can use elasticsearch_loader for loading json files into elasticsearch (2.X, 5.X, 6.X).
You can download it with pip:
pip install elasticsearch-loader
And then you will be able to load json files into elasticsearch by issuing:
elasticsearch_loader --index incidents --type incident json file1.json file2.json
Disclaimer: I'm the author of elasticsearch_loader
I want to make a simple HTTP-Request in JMeter. I'm using some variables of a CSV I just created. In this CSV are names like "Müller" or "Böhm".
So when I run the test I notice that Jmeter convert "Müller" into "Müller".
I create my CSV in Notepad++ (UTF-8 without BOM).
Furthermore I change Jmeter.properties:
sampleresult.default.encoding=UTF-8
An other idea was to use the post-Benshellsampler:
prev.setDataEncoding("UTF-8");
and
request.setCharacterEncoding("UTF-8");
None of these worked for me.
Errorreport: http://pastebin.com/EydAjfdm
JMeter: 2.13 |
Plugins: Webdriver, Standard, Extra, ExtraLibs (1.3.1)
My expectation is that you need to change encoding of JVM. Add the following line:
log.info(System.getProperty("file.encoding") + " <--------------------");
to any Beanshell Test Element and look into jmeter.log file. If you see something other than UTF-8 - you have a problem which needs to be fixed
2015/12/15 19:05:37 INFO - jmeter.util.BeanShellTestElement: UTF-8 <--------------------
Remove all Beanshell tweaks (by the way, the one with request won't work)
Add the following line to system.properties file (lives under /bin folder of your JMeter installation)
file.encoding=UTF-8
Restart JMeter and that should be it.
See:
List of Java system properties in general and file.encoding bit in particular
Apache JMeter Properties Customization Guide - to learn more about different JMeter properties types and ways of working with them
I need to know about full downloading a resource from server. My server is configured with NginX web server, and I want to do something if and only if the resource downloaded completely by user.
If you are using Nginx to handle downloading your files (using XSendfile), you should add a specific access_log block to your download handling block in your Nginx configs (in your "server" block). It would be something like this:
location /download_music/ {
internal;
root /usr/share/nginx/MY_SITE/media/music;
access_log /var/log/nginx/download.MY_SITE.log download;
}
The word "download" at the end of the access_log line is actually a log format which you should add it to the nginx main config file (/etc/nginx/nginx.conf) in the "http" block. It could be something like this:
http {
...
log_format download '{ "remote_addr" : "$remote_addr", "time":"$time_local", "request":"$request", '
'"traffic":$body_bytes_sent, "x_forwarded_for":"$http_x_forwarded_for" }';
...
}
You can change this format to what format you want (you will use it in your script later). If you monitor this log file (using "tail -f /var/log/nginx/download.MY_SITE.log") you will see that any time a download is paused or finished, a line of log will be added to this file.
The next step is using rsyslog and the "imfile" and "omprog" modules. You should add these configs at the end of the config file of rsyslog (/etc/rsyslog.conf):
$ModLoad imfile
$InputFileName /var/log/nginx/download.MY_SITE.log
$InputFileTag nginx-access
$InputFileStateFile state-nginx-access
$InputFileSeverity info
$InputFileFacility local3
$InputFilePollInterval 1
$InputRunFileMonitor
$ModLoad omprog
$actionomprogbinary /usr/share/nginx/MY_SITE/scripts/download.py
$template LogZillaDbInsert,"%hostname:::lowercase%\9%pri%\9%programname%\9%msg%\n"
local3.* :omprog:;RSYSLOG_TraditionalFileFormat
Pay attention to this line:
/usr/share/nginx/MY_SITE/scripts/download.py
This is the address of the script which would be called every time a log entry is added to the log file and the whole log entry will be available in this script using (in Python code):
line = sys.stdin.readline()
Then you can parse the line and find whatever you have logged including the downloaded file size (on every pause/finish event). Now, you can simply include the file size in the download URL and retrieve it in this script and compare it with the downloaded bytes. If these two numbers are equal, it means that download has been finished successfully. Moreover, you can do any other thing you want in this script (for example, expire download link, increase download count on DB, ...)