Is there any way to pass the postfix' queue id (not the Message-ID: header) to a procmail command?
The reason behind is to have a procmail logfile containing the queue id for detailed logging about a handled message.
Procmail can extract anything you can articulate a regex for. Without access to your local headers, this is speculative and somewhat ad lib but hopefully it should at least give you an idea.
# Put a newline in a string for the log message
NL="
"
:0
* ^Received: from .* \
by yourserver\.example\.com \(Postfix)\
with [A-Z]+ id \/[A-F0-9]+
{ LOG="Postfix id $MATCH$NL" }
The special token \/ causes the matching text after it to be captured into the variable $MATCH.
Related
We are using fluent-bit plugin to tail from a file and send to an HTTP endpoint.
The sample log looks like the following.
tenant 1 testing 100
The configuration for input looks like the following.
[INPUT]
Name tail
Path /var/log/input/**/*.log
Tag tenant
Path_Key filename
We then use a lua filter to add a key based on the filepath. This works as expected.
[FILTER]
Name lua
Match *
script /etc/td-agent-bit/test.lua
call extract_id
At this point, we try to filter the message and rewrite the tag based on the tenantid.
[FILTER]
Name rewrite_tag
Match *
Rule $tenantid ^([a-z]+)-([0-9]+)$ from.$tenantid false
Emitter_Name re_emitted
With a stdout, like below,
[OUTPUT]
Name stdout
Match *
we verified the message to be like the following.
tenant: [1630073320.394812583, {"log"=>"tenant 1 testing 100", "tenantid"=>"tenant1", "filename"=>"/var/log/input/tenant1/file1.log"}]
It looks like the rewrite_tag plugin is not able to work and change the tag as expected. Is there a problem with the regex pattern ? Any help on this will be hugely appreciated.
I believe your regex needs a very slight tweak to remove the dash in the match pattern. At the moment, it's looking for abc-123, but your tenant id format is abc123.
By removing the dash from your regex, the example tenantid field should match:
^([a-z]+)-([0-9]+)$ will match "tenant-1"
^([a-z]+)([0-9]+)$ will match "tenant1"
A URI is composed of several parts: scheme:[//[user[:password]#]host[:port]][/path][?query][#fragment]. Often, I find myself wanting to refer to the entire part of the URI to the right of the host and port - the part that would be considered the URI in an HTTP request:
GET /path?query#fragment
Host: example.com
As a short-hand, I normally call this the "path", but that's not quite accurate, as the path is only part of it. This is essentially the inverse of What do you call the entire first part of a URL?
Is there an agreed-upon name for this?
Within a full HTTP URI, there doesn’t seem to be a term that denotes everything coming after the authority.
If you only have the part in question as a URI reference (e.g., in a HTTP GET request), it’s called a relative reference:
relative-ref = relative-part [ "?" query ] [ "#" fragment ]
But this term also includes network-path references (often called protocol-relative URIs), e.g. //example.com/path?query#fragment. To exclude this case, you could use the terms for the other two cases:
absolute-path reference (begins with a single /, e.g. /path?query#fragment)
relative-path reference (doesn’t begin with a /, e.g., path?query#fragment)¹
¹ If the first path segment contains a :, you have to begin the relative-path reference with ./ (e.g., ./pa:th?query#fragment).
RFC 7230 says:
request-line = method SP request-target SP HTTP-version CRLF
I personally prefer to use the terms
origin for scheme and authority (where) and
resource for path, query string and fragment (what).
I'm not aware of any term for that portion of a URI.
RFC3986 says this
foo://example.com:8042/over/there?name=ferret#nose
\_/ \______________/\_________/ \_________/ \__/
| | | | |
scheme authority path query fragment
| _____________________|__
/ \ / \
urn:example:animal:ferret:nose
I need to reuse value which is generated for my previous request.
For example, at first request, I make a POST to the URL /api/products/{UUID} and get HTTP response with code 201 (Created) with an empty body.
And at second request I want to get that product by request GET /api/products/{UUID}, where UUID should be from the first request.
So, the question is how to store that UUID between requests and reuse it?
You can use the Request Sent Dynamic values https://paw.cloud/extensions?extension_type=dynamic_value&q=request+send these will get the value used last time you sent a requst for a given request.
In your case you will want to combine the URLSentValue with the RegExMatch (https://paw.cloud/extensions/RegExMatch) to first get the url as it was last sent for a request and then extract the UUID from the url.
e.g
REQUEST A)
REQUEST B)
The problem is in your first requests answer. Just dont return "[...] an empty body."
If you are talking about a REST design, you will return the UUID in the first request and the client will use it in his second call: GET /api/products/{UUID}
The basic idea behind REST is, that the server doesn't store any informations about previous requests and is "stateless".
I would also adjust your first query. In general the server should generate the UUID and return it (maybe you have reasons to break that, then please excuse me). Your server has (at least sometimes) a better random generator and you can avoid conflicts. So you would usually design it like this:
CLIENT: POST /api/products/ -> Server returns: 201 {product_id: UUID(1234...)}
Client: GET /api/products/{UUID} -> Server returns: 200 {product_detail1: ..., product_detail2: ...}
If your client "loses" the informations and you want him to be later able to get his products, you would usually implement an API endpoint like this:
Client: GET /api/products/ -> Server returns: 200 [{id:UUID(1234...), title:...}, {id:UUID(5678...),, title:...}]
Given something like this, presuming the {UUID} is your replacement "variable":
It is probably so simple it escaped you. All you need to do is create a text file, say UUID.txt:
(with sample data say "12345678U910" as text in the file)
Then all you need to do is replace the {UUID} in the URL with a dynamic token for a file. Delete the {UUID} portion, then right click in the URL line where it was and select
Add Dynamic Value -> File -> File Content :
You will get a drag-n-drop reception widget:
Either press the "Choose File..." or drop the file into the receiver widget:
Don't worry that the dynamic variable token (blue thing in URL) doesn't change yet... Then click elsewhere to let the drop receiver go away and you will have exactly what you want, a variable you can use across URLs or anywhere else for that matter (header fields, form fields, body, etc):
Paw is a great tool that goes asymptotic to awesome when you explore the dynamic value capability. The most powerful yet I have found is the regular expression parsing that can parse raw reply HTML and capture anything you want for the next request... For example, if you UUID came from some user input and was ingested into the server, then returned in a html reply, you could capture that from the reply HTML and re-inject it to the URL, or any field or even add it to the cookies using the Dynamic Value capabilities of Paw.
#chickahoona's answer touches on the more normal way of doing it, with the first request posting to an endpoint without a UUID and the server returning it. With that in place then you can use the RegExpMatch extension to extract the value from the servers's response and use it in subsequent requests.
Alternately, if you must generate the UUID on the client side, then again the RegExpMatch extension can help, simply choose the create request's url for the source and provide a regexp that will strip the UUID off the end of it, such as /([^/]+)$.
A third option I'll throw out to you, put the UUID in an environment variable and just have all of your requests reference it from there.
I know that URI supports the following syntax:
http://[user]:[password]#[domain.tld]
When there is no password or if the password is empty, is there a colon?
In other words, should I accept this:
http://[user]:#[domain.tld]
Or this:
http://[user]#[domain.tld]
Or are they both valid?
The current URI standard (STD 66) is RFC 3986, and the relevant section is 3.2.1. User Information.
There it’s defined that the userinfo subcomponent (which gets followed by #) can contain any combination of
the character :,
percent-encoded characters, and
characters from the sets unreserved and sub-delims.
So this means that both of your examples are valid.
However, note that the format user:password is deprecated. Anyway, they give recommendations how applications should handle such URIs, i.e., everything after the first : character should not be displayed by applications, unless
the data after the colon is the empty string (indicating no password).
So according to this recommendation, the userinfo subcomponent user: indicates that there is the username "user" and no password.
This is more like convenience and both are valid. I would go with http://[user]#[domain.tld] (and prompt for a password.) because it's simple and not ambiguous. It does not give any chance for user to think if he has to add anything after :
What I'm trying to do for a PoC and is to add a href to web pages coming from a dynamic backend server. Adding the href is easy using "subs_filter", but I need to use information embedded within the response to construct the href.
Is it possible to use LUA to process the response from proxy_pass, modify it and return to requester (client)?
Any and all suggestions welcome.
Below is the code I'm looking at, now I understand Lua better and how nginx uses it I see that 'body_filter' is the correct way to. However the code seems simple enough but i can't get the regex to work.
Further background, I'm trying to parse the returned proxy_pass response, parse it for a start and end time, then construct a JS script url placed into the head.
Example response that I want to regex against.
Informações Adicionais
Horário de início: 08H50
Horário de término: 09H14
The code from within the 'location {}'
body_filter_by_lua '
-- my regex which is validate but doesn't seem to be within LUAJIT
--local from, to, err = ngx.re.find(ngx.arg[1], "(.início: *\d{2}H\d{2})", "jo")
local from, to, err = ngx.re.find(ngx.arg[1], "início", "jo")
replacestr = string.sub(ngx.arg[1], to, 5)
replaceme = "<script></script></head>"
ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replaceme)
';
Changing "início" to "head" for example works, so I'm assuming it is the accented char but I'm unable to find confirmation of this.
Changing "início" to "\d{2}H\d{2}" fails, with "body_filter_by_lua:5: invalid escape sequence near '"'"
I discovered what I mentioned in the comments regarding 'nix.header.content_length' and importantly nginx and lua require double escaping see: lua-nginx-module special pcre sequences for more details.
The accented chars needed the flag 'u' adding to 'jo' of the 'ngx.re.find'
user "body_filter_by_lua_file"
Equivalent to body_filter_by_lua, except that the file specified by contains the Lua code, or, as from the v0.5.0rc32 release, the Lua/LuaJIT bytecode to be executed.
When a relative path like foo/bar.lua is given, they will be turned into the absolute path relative to the server prefix path determined by the -p PATH command-line option while starting the Nginx server.
This directive was first introduced in the v0.5.0rc32 release.