I have a log file (zope/plone event.log) which using custom string (e.g "-----") as divider between events, how grok pattern for parsing this log file to logstash should be?
This is an example how the log look like:
------
2014-07-21T12:13:30 INFO ZServer HTTP server started at Mon Jul 21 12:13:30 2014
Hostname: localhost
Port: 8401
------
2014-07-21T12:13:44 WARNING SecurityInfo Conflicting security declarations for "setText"
------
2014-07-21T12:13:44 WARNING SecurityInfo Class "ATTopic" had conflicting security declarations
------
2014-07-21T12:13:47 INFO DocFinderTab Applied patch version 1.0.5.
You should start with the multiline codec or filter to create a single event for processing.
EDIT:
The doc gives this example:
filter {
multiline {
pattern => "pattern, a regexp"
negate => boolean
what => "previous" or "next"
}
}
And describes what 'negate' and 'what' do. Hopefully 'pattern' make sense.
So, how about "every line that doesn't start with a date belongs with the prior line"? That might be something like this:
filter {
multiline {
negate => 'true'
pattern => "^%{TIMESTAMP_ISO8601} "
what => 'previous'
}
}
You'd be left with the "----" at the end of each line. Since you don't need them as delimiters, you can get rid of them (before the multiline filter stanza):
if message =~ /^-+$/ {
drop{}
}
Related
I am trying to load a JSON file and use the values to perform some actions based on my tests. I tried to load the json value which I think I got right, but when trying to log the output, I got error message:
Resolving variable '${qa["REQUEST_ID"]}' failed: TypeError: list indices must be integers or slices, not str
Not exactly sure what this means since I am new to Robot Framework. This is what I did to load and log the values:
${file} Get File ${CURDIR}/RequestIDs.json
${qa} Evaluate json.loads('''${file}''') json
Log To Console ${qa["REQUEST_ID"]}
Json file looks something like:
[
{
"REQUEST_ID" : 10513
},
{
"REQUEST_ID" : 48156
},
{
"REQUEST_ID" : 455131
}
]
So basically I want to get the "REQUEST_ID" value and type that in a text field.
Look at the structure of your json - it's a list of dictionaries; so you have to first specify which list member you want, and then its REQUEST_ID field:
Log To Console ${qa[0]["REQUEST_ID"]
# print the value from all present dictionaries in the list:
FOR ${member} IN #{qa}
Log To Console ${member["REQUEST_ID"]
END
I installed Sublime Text 3. I installed Package Control, R-Box, SendText, SendCode, sublimeREPL, and 1337 Color Scheme. To test that REPL R is working I ran the code:
getwd()
The output was
R:/R_WD.
That's the correct location.
What I want to do is send code from one window to REPL R and have it evaluated. It originally would send code to the REPL R window and I could run the code but I wanted to be able to have it run automatically. I tried looking on forums for ideas and changed some settings. Now it will not send code to REPL R.
Here are my settings:
Preferences --> Package Settings --> R-Box --> Settings:
{
// enable auto completions
"auto_completions": true,
// show popup hints
"show_popup_hints": true,
// path to Rscript, for example
// (mac, linux): "/usr/local/bin/Rscript"
// (windows): "C:\\Program Files\\R\\R-x.y.z\\bin\\Rscript.exe"
"rscript_binary": null,
// additional paths to PATH variable
"additional_paths": []
}
User:
{
"prog": "Cmder",
}
Preferences --> Package Settings --> SendText --> Settings---> Default:
{
// Uncomment the program you want send text to:
"program": "Terminal.app",
// "program": "iTerm",
// "program": "tmux",
// "program": "screen",
"paths":
{
// It might be necessary to explicitly set path (usually /usr/bin
// or /usr/local/bin) to tmux and screen. Uncomment below and specify
// the correct path:
// "tmux": "/usr/local/bin/tmux",
// "screen": "/usr/local/bin/screen"
}
}
Preferences --> Package Settings --> SendText --> Settings---User:
{
"prog": "Cmder",
}
Preferences --> Package Settings --> SendCode--> Settings:
{
"prog": "Cmder",
}
Preferences --> Package Settings -->SublimeREPL--> Settings---Default:
{
// default_extend_env are used to augment any environment variables
// that should be visible for all subprocess repls launched within
// SublimeREPL. This is a very good place to add PATH extension
// once "PATH": "{PATH}:/home/username/mylocalinstalls/bin" or whatever
"default_extend_env": {},
// Specify whether to move repls to a different Sublime Text group (frame)
// immediately on opening. Setting this to true will simply move it to
// the 'next' group from the one that was in focus when it was opened
// (one down with row layout, one to the right with column and grid
// layout). Alternatively, you can set this to the index of the group in
// which you want all repls to be opened (index 0 being the top-left group).
// Activating this option will NOT automatically change your layout/create
// a new group if it isn't open.
"open_repl_in_group": true,
// Persistent history is stored per REPL external_id, it means that all python
// REPLS will share history. If you wish you can disable history altogether
"persistent_history_enabled": true,
// By default SublimeREPL leaves REPL view open once the underlying subprocess
// dies or closes connection. This is useful when the process dies for an unexpected
// reason as it allows you to inspect it output. If you want. Setting this
// to true will cause SublimreREPL to close view once the process died.
"view_auto_close": false,
// On POSIX system SublimeText launched from GUI does not inherit
// a proper environment. Often leading to problems with finding interpreters
// or not using the ones affected by changes in ~/.profile / *rc files
// This command is used as a workaround, it's launched before any subprocess
// repl starts and it's output is parsed as an environment
"getenv_command": ["/bin/bash", "--login", "-c", "env"],
// Some terminals output ascii color codes which are not currently supported
// enable this option to filter them out.
"filter_ascii_color_codes": true,
// Where to look for python virtualenvs
"python_virtualenv_paths": [
"~/.virtualenvs", // virtualenvwrapper
"~/.venv" // venv.bash https://github.com/wuub/venv
],
// Use arrows for history navigation instead of Alt+[P|N]/Ctrl+[P|N]
"history_arrows": true,
// standard sublime view settings that will be overwritten on each repl view
// this has to be customized as a whole dictionary
"repl_view_settings": {
"translate_tabs_to_spaces": false,
"auto_indent": false,
"smart_indent": false,
"spell_check": false,
"indent_subsequent_lines": false,
"detect_indentation": false,
"auto_complete": true,
"line_numbers": false,
"gutter": false
},
// this settings exposes additional variables in repl config files, especially
// those related to sublime projects that are not available through standard API
// WARNING: this will switch your build system back to Automatic each time a REPL
// is started so beware!
"use_build_system_hack": false,
// IP address used to setup autocomplete server in sublimerepl.
// changing this is usefull when you want to exclude one address
// from proxychains/tsocks routing
"autocomplete_server_ip": "127.0.0.1",
// Mapping is used, when external_id of REPL does not match
// source.[xxx] scope of syntax definition used to highlight
// files from which text is being transfered. For example octave
// repls use source.matlab syntax files and w/o this mapping text transfer
// will not work
"external_id_mapping": {
"octave": "matlab"
},
// If set to true, SublimeREPL will try to append evaluated code to repl
// output before evaluation (e.g. Ctrl+, f)
"show_transferred_text": true,
// If set to true repl view (tab) that receives text for evaluation will
// be brought to front after text transfer. Note: This will not fire if repl
view
// is in the same tab group as the view from which the code is sent.
"focus_view_on_transfer": true
}
Preferences --> Package Settings -->SublimeREPL-->Settings--- User:
{
"default_extend_env": {"PATH": "{PATH};R:\\R_WD\\R\\R-3.4.2\\bin\\x64"},
"show_transferred_text": true
}
I noticed that the comments said to uncomment or add code but I can't seem to edit the code in those locations.
Any help is appreciated. If there is any clarification needed, I'm happy to provide more info.
This is how I changed the code to get it to work:
I changed "Preferences --> Package Settings --> SendText --> Settings---User:" from:
{
"prog": "Cmder",
}
to:
{"r" : {
"prog": "cmder",
}
{ "rscript_binary" : "R:\\R_WD\\R\\R-3.4.2\\bin\\Rscript.exe"
}
}
All the other user settings code that was:
{
"prog": "Cmder",
}
I changed to:
{"r" : {
"prog": "cmder",
}
}
I can now send the code to be automatically evaluated with the default option.
I have a Spring Cloud Contract DSL that looks like this:
package contracts.someconsumer.messaging
import org.springframework.cloud.contract.spec.Contract
Contract.make {
label 'my_label'
// input to the contract
input {
// the contract will be triggered by a method
triggeredBy('someMethodThatSendsMessage()')
}
// output message of the contract
outputMessage {
// destination to which the output message will be sent
sentTo 'Consumer.contractTest.VirtualTopic.some_destination'
// the body of the output message
body([
id: value(consumer('11111111-2222-3333-4444-555555555555'),producer(regex(uuid()))),
correlationId: value(producer(regex(uuid()))),
service: 'MY_SERVICE',
payload:
[
email: 'test#example.com'
]
])
}
}
Without the "payload" part everything works great. With the payload, I encounter this exception:
com.jayway.jsonpath.InvalidPathException: Filter: [?] can not be applied to primitives. Current context is: {"email":"test#example.com","legalName":"ACME Inc"}
at com.jayway.jsonpath.internal.path.PredicatePathToken.evaluate(PredicatePathToken.java:66) ~[json-path-2.2.0.jar:2.2.0]
at com.jayway.jsonpath.internal.path.PathToken.handleObjectProperty(PathToken.java:81) ~[json-path-2.2.0.jar:2.2.0]
at com.jayway.jsonpath.internal.path.PropertyPathToken.evaluate(PropertyPathToken.java:79) ~[json-path-2.2.0.jar:2.2.0]
at com.jayway.jsonpath.internal.path.RootPathToken.evaluate(RootPathToken.java:62) ~[json-path-2.2.0.jar:2.2.0]
The relevant line from the generated test:
assertThatJson(parsedJson).field("['payload']").field("['email']").isEqualTo("test#example.com");
Just a little more info, this is what the serialized message looks like:
2017-09-21 08:32:03.721 INFO 10716 --- [ main] c.v.sccdemo.producer.InviteServiceImpl : Event: {"id":"e63de44e-6e1a-4c4e-b98b-3c49a49efc9c","destination":"VirtualTopic.some_destination","correlationId":"8efb9740-5651-4068-8a6e-574ae7759552","service":"MY_SERVICE","payload":"{\"email\":\"test#example.com\",\"legalName\":\"ACME Inc\"}","timestamp":1505997123576,"version":"v1"}
Am I doing something wrong in the DSL? Is the 'payload' part of the body expressed correctly?
The payload looks wrong... Notice that it's considering payload as a String value instead of a Map. I guess it's enough to change the payload to the proper one and things should work again!
I have this site: http://embed.plnkr.co/Bs5iDqtXSSnvye2ORI6k/preview
Code:
var app = angular.module('plunker', []);
var a = new Array(1000);
for (var i = 0; i< 1000; i++) {
a[i] = 'Name' + i;
}
app.controller('MainCtrl', function($scope, $interval) {
$scope.names = a;
$scope.start = function () {
$interval(function () {
$scope.names.pop();
}, 50);
}
});
And the following spec:
'use strict';
describe('Name list', function () {
it('should get the text of the last name', function () {
browser.driver.get('http://embed.plnkr.co/Bs5iDqtXSSnvye2ORI6k/preview');
browser.switchTo().frame(browser.driver.findElement(protractor.By.tagName('iframe')));
element(by.buttonText('start')).click();
expect(element.all(by.tagName('span)).last().getText()).toBe('Name999');
});
});
And this config:
'use strict';
// An example configuration file.
exports.config = {
baseUrl: 'http://localhost:3000',
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['stale.spec.js']
};
And when I run Protractor I get the following error:
StaleElementReferenceError: stale element reference: element is not
attached to the page document (Session info: chrome=43.0.2357.81)
(Driver info: chromedriver=2.15.322455
(ae8db840dac8d0c453355d3d922c91adfb61df8f),platform=Mac OS X 10.10.3
x86_64) (WARNING: The server did not provide any stacktrace
information) Command duration or timeout: 9 milliseconds For
documentation on this error, please visit:
http://seleniumhq.org/exceptions/stale_element_reference.html Build
info: version: '2.45.0', revision: '5017cb8', time: '2015-02-26
23:59:50' System info: host: 'ITs-MacBook-Pro.local', ip:
'129.192.20.150', os.name: 'Mac OS X', os.arch: 'x86_64', os.version:
'10.10.3', java.version: '1.8.0_31' Driver info:
org.openqa.selenium.chrome.ChromeDriver Capabilities
[{applicationCacheEnabled=false, rotatable=false,
mobileEmulationEnabled=false,
chrome={userDataDir=/var/folders/rr/63848xd90yscgwpkfn8srbyh0000gq/T/.org.chromium.Chromium.rarNyX},
takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=true,
version=43.0.2357.81, platform=MAC, browserConnectionEnabled=false,
nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true,
webStorageEnabled=true, browserName=chrome, takesScreenshot=true,
javascriptEnabled=true, cssSelectorsEnabled=true}] Session ID:
235ec977a69d98c7f5b75a329e8111b2
This means that the element I try to interact with (getting the text of the element), isn't attached to the DOM anymore. This example is really my spec simplyfied. What really happens in my real spec is I try to get the text of the last element of a list of elements (generated by ng-repeat). What also happens is that the model updates, by removing the last element of the array representing the list of element. This example above is just something to reproduce the error (every time).
If I comment out this line: element(by.buttonText('start')).click(); the spec is successful.
I struggled a lot with this and tried to figure out why this would happen. I first thought that the element finder which points to the last element of the list was created long before the interaction was done, so there was no surprise to me that the element could be detached from the DOM in that period of time between the creation of the element finder and the interaction.
What I've later found out is that the element is found just before the interaction is done, every time you interact with something. So pointing to the last element should actually point to the last element of the time interacting with the element.
Using browser.pause() I was able to see what WebDriver really does, and this is two tasks where in between the error is thrown:
(pending) Task::414<WebDriver.findElements(By.tagName("span"))>
| | | | | | Task: WebDriver.findElements(By.tagName("span"))
| | | | | | at Array.forEach (native)
Here in between, the DOM is updated according to the model, and the last element of the list is detached.
(pending) Task::1170<WebElement.getText()>
| | | | | | Task: WebElement.getText()
| | | | | | at Array.forEach (native)
The DOM is updated in this small hole of execution. Currently the model updates every 50 ms, and this is sure to throw a Stale Element Reference error. But if I increase the interval to, say 1000 ms, then the chance to get the error is much less. So it depends on how fast your computer runs if you get this error.
The fix is up to you, but with this information it should be a bit clearer what to do, I hope.
The browser is running asynchronously from your protractor tests. This example really highlights that nicely (its a problem for many protractor tests, but not usually so obvious). This is compounded by what looks like a single line:
expect(element.all(by.tagName('span')).last().getText()).toBe('Name999');
actually requires several round-trips to the browser (there are a lot of promises being returned and resolved: element.all, last, getText). For most web pages that are "passive" once they've stabilized, this isn't a problem, but for any web page that dynamically changes without user inputs, testing with protractor can be painful.
To make a lookup and test "atomic" in the browser, thus side-steping this issue, you can define a function to run in the browser (my CSS DOM-fu is weak, so hopefully someone can tweak this to make it less terrible):
expect(browser.executeScript(function() {
var spans = document.getElementsByTagName('span');
var lastSpan = spans[spans.length - 1]; // XXX handle empty spans
return lastSpan.innerText;
}).toBe('Name999');
Beware that the "function" is serialized and executed on the browser, so no variables from an enclosing scope will work. Also, this approach loses any browser-compatibility magic hiding in protractor or webdriver (e.g., I wouldn't be surprised if getText() wasn't simply a innerText accessor).
Also, note that you still have a race condition. Between the 'click' to get things started and this code to actually inspect the DOM, it may or may not have been mutated (and "Name999" might be gone).
I am trying to write a custom logstash filter. I have followed the documentation here.
My filter (at the minute) looks like:
# Call this file 'foo.rb' (in logstash/filters, as above)
require "logstash/filters/base"
require "logstash/namespace"
class LogStash::Filters::NLP < LogStash::Filters::Base
# Setting the config_name here is required. This is how you
# configure this filter from your logstash config.
#
# filter {
# foo { ... }
# }
config_name "nlp"
# New plugins should start life at milestone 1.
milestone 1
# Replace the message with this value.
config :message, :validate => :string
public
def register
# nothing to do
java_import 'uk.co.jaywayco.Parser'
end # def register
public
def filter(event)
# return nothing unless there's an actual filter event
return unless filter?(event)
if #message
# Replace the event message with our message as configured in the
# config file.
event["message"] = #message
end
# filter_matched should go in the last line of our successful code
filter_matched(event)
end # def filter
end # class LogStash::Filters::NLP
Ive created a very simple logstash config that looks like this:
input {
stdin { type => "nlp" }
}
filter {
if [type] == "nlp" {
nlp {
message => "Hello world! - Youve been NLP'd"
}
}
}
output {
stdout { }
}
When i run logstash with the following command:
bin/logstash -f /etc/logstash/conf.d/test.conf
I get the following error:
The error reported is:
enter code herecannot load Java class uk.co.jaywayco.Parser
The java class uk.co.jaywayco.Parser is in a jar file located in the /opt/logstash/lib/logstash/filters folder. Which is where all the filter scripts reside.
Why wont logstash load my java class?
Ok so thanks to the logstash IRC channel
The solution was to add require 'logstash/filters/myjarfile.jar' wher the other requires are