Is it possible in telegraf using a processor to drop a tag from a measurement?
Using the cisco_telemetry plugin that takes in series and within one of the measurements not the whole plugin I want to only keep one tag.
I tried using the tag_limit processor but it didn't work. The current measurement "Cisco-IOS-XR-procfind-oper:proc-distribution/nodes/node/process/pid/filter-type" has two tags "pid" and "proc_name" each contain around 10000 values. I only want to keep "proc_name" and drop "pid" from this measurement. Should the tag_limit processor work for this? Version 1.23
[[processors.tag_limit]]
namepass = ["Cisco-IOS-XR-procfind-oper:proc-distribution/nodes/node/process/pid/filter-type"]
## Maximum number of tags to preserve
limit = 1
## List of tags to preferentially preserve
keep = ["proc_name"]
within one of the measurements
I would probably use a starlark processor then. Use namepass as you have done, and then remove the specific tag.
[[processors.starlark]]
namepass = ["Cisco-IOS-XR-procfind-oper:proc-distribution/nodes/node/process/pid/filter-type"]
source = '''
def apply(metric):
metric.tags.pop("pid")
return metric
'''
For users looking to do this to an entire measurement, they can drop tags from a measurement with metric modifiers. Specifically, you are looking for tagexclude, which will remove tags from a measurement matching those patterns. This way, you do not even need to use a processor and can add this directly to the end of your input:
[[inputs.cisco_telemetry]]
<connection details>
tagexclude = ["pid"]
Related
I have embedded data that I have imported into Qualtrics use a web service block. The data comes from a .json file and reads something like 0.male, 1.male, 2.male, etc.
I have been trying to read this into my survey using the Qualtrics.SurveyEngine.getEmbeddedData method but without luck.
I'm trying to do something that takes the form.
let n = 2
Qualtrics.SurveyEngine.getEmbeddedData(n + ".male")
but this has been returning a NULL result. Is it possible to read embedded data that starts with a number?
Also see:
https://community.qualtrics.com/XMcommunity/discussion/15991/read-in-embedded-variables-using-a-loop#latest
The issue isn't the number, it is the dot. getEmbeddedData() doesn't work when the name contains a dot. See https://stackoverflow.com/a/51802695/4434072 for possible alternatives.
I have a table of records that has two logical flags, holdn and holdl. I want to loop through this table with 3 different criteria.
Either flag is TRUE - We want to see everything that is on hold
Flag holdl is TRUE - We only want see items that are on hold for this one reason
Flag holdn is TRUE - We only want to see items that are on hold for this other reason.
I cannot figure out how to dynamically change the for each loop based on this. What I have tried so far is to set the value of a variable based on these conditions and then use the content of the variable as one of the where parameters. This does not work as Progress complains that there is a data mismatch. The variable is a string, the flags are logical, so that does make sense. See sample code below. This is a snippet of the actual code with the the table name changed. The which-hold, order-from, etc variables are defined and set in a different module which calls this one.
DEFINE VARIABLE which-hold# AS CHARACTER FORMAT "x(30)" NO-UNDO.
CASE which-hold:
WHEN "B" THEN which-hold# = "(widget.holdn or widget.holdl)".
WHEN "L" THEN which-hold# = "widget.holdl".
WHEN "N" THEN which-hold# = "widget.holdn".
END CASE.
for each widget where which-hold# and
widget.order-no >= order-from and widget.order-no <= order-thru and
widget.line-no >= line-from and widget.line-no <= line-thru and
widget.joint-no >= joint-from and widget.joint-no <= joint-thru
no-lock:
A bunch of code to make a nice report with the retrieved records...
end.
Self taught Progress programmer here, who has inherited a huge, poorly documented application. Please be gentle.
If you would prefer not to deal with handles a semi-dynamic approach is also possible:
define variable i as integer no-undo.
define query q for customer.
do while true:
update i.
case i:
when 0 then quit.
when 1 then open query q for each customer no-lock where custNum >= 1000.
when 2 then open query q for each customer no-lock where state = "nh".
otherwise open query q for each customer no-lock where name begins "u".
end.
do while true with frame a:
get next q.
if not available customer then leave.
display custNum name state with frame a 10 down.
down with frame a.
end.
close query q.
end.
What you want is actually a dynamic query. I'll get to it at the end, but first I'd like to explain why you won't be able to try and substitute the field name in the which-hold# variable: because the query is evaluated at compile time. And this is what it reads (supposing which-hold# has a value of widget.holdn
FOR EACH widget where "widget-holdn" (...)
And that does not evaluate to TRUE or FALSE. So what, you ask? Well, that is the key here. Every condition needs to evaluate to true or false, so you'd be more in luck if you try
for each widget where (if widget-hold# = 'widget.holdn' then widget.holdn = true else TRUE) (...)
Again, notice the condition will exist if widget-hold# has the value I want, otherwise it doesn't filter on this at all.
So you can just code the way I showed (for each of the conditions you have) and it should work fine.
BUT let me suggest a dynamic query instead.
You need to have:
DEFINE VARIABLE hQuery AS HANDLE NO-UNDO.
CREATE QUERY hQuery.
hQuery:SET-BUFFERS(BUFFER widget:HANDLE).
hQuery:QUERY-PREPARE('<THIS IS THE CORE>').
hQuery:QUERY-OPEN().
DO WHILE hQuery:GET-NEXT():
A bunch of code to make a nice report with the retrieved records...
END.
So in the core you have a string that corresponds to your for each the way you want it to look. So it should be for example (store this in a variable, or assemble it inside the query prepare, it doesn't matter):
'FOR EACH widget NO-LOCK WHERE ' +
(if which-hold = 'B' then 'widget.holdn = true and widget.holdl = true'
else if which-hold = 'L' then 'widget-holdl = true'
else /* N */ 'widget-holdn = true').
Remember I said your query is evaluated at compile time? Well, just so you know, dynamic queries on the other end are evaluated at run time, so be prepared for errors to pop up only when you run. Another thing I should mention is dynamic queries are slower than static ones, so please evaluate and choose your poison :)
This should be what you need. Please let me know if it's helpful or any questions remain.
Is it possible to extract files only for 3 days, without extracting all the files.
DROP VIEW IF EXISTS dbo.Read;
CREATE VIEW IF NOT EXISTS dbo.Read AS
EXTRACT
Statements
FROM
"adl://Test/{date:yyyy}/{date:M}/{date:d}/Testfile.csv"
USING Extractors.Csv(silent:true,quoting : true, nullEscape : "/N");
#res =
SELECT * FROM dbo.Read
WHERE date BETWEEN DateTime.Parse("2015/07/01") AND DateTime.Parse("2015/07/03");
OUTPUT #res
TO "adl://test/Testing/loop.csv"
USING Outputters.Csv();
Partition elimination already ensures for your query that only files matching predicates will actually be read (you can confirm that in the job graph).
See also my previous answer for How to implement Loops in U-SQL
If you have remaining concerns about performance, the job graph can also help you nail down where they originate.
You can use the pattern identifiers in the fileset specification in parts of the path or even parts of the name (see https://msdn.microsoft.com/en-us/library/azure/mt771650.aspx). You can do lists of files, so if you only have one file in each directory you can do;
EXTRACT ...
FROM "adl://Test/2015/07/1/Testfile.csv"
, "adl://Test/2015/07/2/Testfile.csv"
USING ...;
If there is more than one file in each directory you can do individual extracts for each day and then union the result. Something like:
#a = EXTRACT ....
FROM "adl://Test/2015/07/1/{*}.csv"
USING ...;
#b = EXTRACT ....
FROM "adl://Test/2015/07/2/{*}.csv"
USING ...;
#fullset = SELECT * FROM #a UNION SELECT * FROM #b;
Unfortunately I believe there is no list of filesets at the moment allowing you to do above case in one EXTRACT statement.
I have the data as below manner.
<Status>Active Leave Terminated</Status>
<date>05/06/2014 09/10/2014 01/10/2015</date>
I want to get the data as in the below manner.
<status>Active</Status>
<date>05/06/2014</date>
<status>Leave</Status>
<date>09/10/2014</date>
<status>Terminated</Status>
<date>01/10/2015</date>
please help me on the query, to retrieve the data as specified above.
Well, you have a string and want to split it at the whitestapces. That's what tokenize() is for and \s is a whitespace. To get the corresponding date you can get the current position in the for loop using at. Together it looks something like this (note that I assume that the input data is the current context item):
let $dates := tokenize(date, "\s+")
for $status at $pos in tokenize(Status, "\s+")
return (
<status>{$status}</status>,
<date>{$dates[$pos]}</date>
)
You did not indicate whether your data is on the file system or already loaded into MarkLogic. It's also not clear if this is something you need to do once on a small set of data or on an on-going basis with a lot of data.
If it's on the file system, you can transform it as it is being loaded. For instance, MarkLogic Content Pump can apply a transformation during load.
If you have already loaded the content and you want to transform it in place, you can use Corb2.
If you have a small amount of data, then you can just loop across it using Query Console.
Regardless of how you apply the transformation code, dirkk's answer shows how you need to change it. If you are updating content already in your database, you'll xdmp:node-delete() the original Status and date elements and xdmp:node-insert-child() the new ones.
I'm attempting to scrape a page that has about 10 columns using Ruby and Nokogiri, with most of the columns being pretty straightforward by having unique class names. However, some of them have class ids that seem to have long number strings appended to what would be the standard class name.
For example, gametimes are all picked up with .eventLine-time, team names with .team-name, but this particular one has, for example:
<div class="eventLine-book-value" id="eventLineOpener-118079-19-1522-1">-3 -120</div>
.eventLine-book-value is not specific to this column, so it's not useful. The 13 digits are different for every game, and trying something like:
def nodes_by_selector(filename,selector)
file = open(filename)
doc = Nokogiri::HTML(file)
doc.css(^selector)
end
Has left me with errors. I've seen ^ and ~ be used in other languages, but I'm new to this and I have tried searching for ways to pick up all data under id=eventLineOpener-XXXX to no avail.
To pick up all data under id=eventLineOpener-XXXX, you need to pass 'div[id*=eventLineOpener]' as the selector:
def nodes_by_selector(filename,selector)
file = open(filename)
doc = Nokogiri::HTML(file)
doc.css(selector) #doc.css('div[id*=eventLineOpener]')
end
The above method will return you an array of Nokogiri::XML::Element objects having id=eventLineOpener-XXXX.
Further, to extract the content of each of these Nokogiri::XML::Element objects, you need to iterate over each of these objects and use the text method on those objects. For example:
doc.css('div[id*=eventLineOpener]')[0].text