I am having a strange problem.
My SNS -> SQS subscription with filter policy (anything-but) has been working fine.
However when I added a new attribute, initially as a String (JSON Object stringified) things stopped working, and seemed due to this new attribute as if I remove it, things work again. From what I can see in the Cloudwatch Metrics it is due to NumberOfNotificationsFilteredOut-InvalidAttributes.
I tried to change it to send as a Binary type, but the same problem, the filtering again fails on NumberOfNotificationsFilteredOut-InvalidAttributes.
I seemed to be able to work around it, by Base64 encoding the value myself and sending as a String type.
What might be happening here? From what I understand, a filter policy should ignore attributes it doesn't care about, and always ignore binary attributes. However this doesn't seem to be my experience.
Amazon SNS ignores message attributes with the Binary data type.
https://docs.aws.amazon.com/sns/latest/dg/subscription-filter-policy-constraints.html
Related
I'm using xtdb in a testing environment with a RocksDB backend. All was well until yesterday, when the system stopped ingesting new data. It tells me that this is because of "missing docs", and gives me the id of the allegedly missing doc, but since it is missing, that doesn't tell me much. I have a specific format for my xt/ids (basically type+guid) and this doesn't match that format, so I don't think this id is one of mine. Calling history on the entity id just gives me an empty vector. I understand the block on updates for consistency reasons, but how to diagnose and recover from this situation (short of trashing the database and starting again)? This would obviously be a massive worry were it to happen in production.
In the general case this "missing docs" error indicates a corrupted document store and the only proper resolution is to manually restore/recover based on a backup of the document store. This almost certainly implies some level of data loss.
However, there was a known bug in the transaction function logic prior to 1.22.0 which could intermittently produce this error (but without any genuine data loss), see https://github.com/xtdb/xtdb/commit/1c30550fb14bd6d09027ff902cb00021bd6e57c4
However, if you weren't using transaction functions then there may be another unknown explanation.
I've created a Web API project in ASP.Net, and am having some trouble getting the authentication working.
The API is expecting a token to be submitted in the Authorization header in each request. The code that checks to see if the header is set checks if the
HttpRequestMessage.Headers.Authorization
property is null. The first few times I tested this, I discovered that this property was always null, but the strange part is that if you checked the HttpRequestMessage.Headers enumerable, the Authorization header WAS set correctly (also if you did HttpRequestMessage.Headers.ToString(), it would appear there too).
Stranger still, I found that if I removed some of the attributes that are sent in the token, I could get it to work as expected. So it was as though the Authorization property wasn't being set if the header value's character length was too long. Unfortunately, even when manually removing some of the text from the token, it would then proceed to fail on a digest check, as it should!
I can't find any documentation that mentions this, so I was wondering if anyone else has come across it? I don't think the header is too long for IIS, because the header value appears in HttpRequestMessage.Headers.ToString(), so it IS being received, but for some reason it's not being assigned to the Authorization property.
Unfortunately I can't re-write the code that checks this property (this seems the easy solution) because it's apart of the Thinktecture library (ie not written by ourselves).
If you are passing the parameters on a GET, you will be limited to 2100 characters. The RFC spec will be different between implementations. Most of the browsers limit you to 2083 characters. You can definitely get away with 1000 characters.
Microsoft
Pretty much everybody else
If you are passing the parameters on a POST, you should have virtually unlimited lengths.
I have an object:
Account
{
Id,
Name,
CurrentBalance
}
Id is an immutable key, Name is a mutable string, and CurrentBalance is calculated from all of the transactions associated with the account.
I am stuck on the fact that GET \Accounts\{Id} will not be idempotent because changes to a transaction will cause a change in CurrentBalance. Should I remove this field from the object and make a request like
POST \Accounts\{Id}\CurrentBalance
But now I have to make multiple calls to the server to get the CurrentBalance of all objects:
GET \Accounts
POST \Accounts\{Id1}\CurrentBalance
POST \Accounts\{Id2}\CurrentBalance
POST \Accounts\{Id3}\CurrentBalance
....
I guess I am just looking to see if there is already a standard way to handle this that I am missing?
UPDATE
Part 2 if the original object is ok via GET. My only way to update the Account.Name is via a PATCH as I cannot allow an update to CurrentBalance, correct?
NOTE
I realize I could put this on the client to have to get all transactions and calculate it, but I would prefer to do this on the server for multiple reasons
Idempotency does not mean that you must always get the same response back.
Consider the resource /TodaysWeather. It would be pretty useless if it always returned the same value.
Idempotency simply states that if a client makes the same request multiple times instead of just once, the impact on the system (from the client's perspective) will be the same.
I just re-read the HTTP specs and realized that if I want to be truly RESTful I have to make multiple calls because GET has to be safe.
In particular, the convention has been established that the GET and
HEAD methods SHOULD NOT have the significance of taking an action
other than retrieval.
I am not deleting this question because I think it could help others in the future, but if the majority disagree I will delete it
If it's important that you're able to PUT some data, then immediately retrieve the same data via GET, then you could simply treat it as a different resource entirely, e.g.:
# Change an account name
PUT \Accounts\{id}
# Get accounts/names/balances
GET \AccountDetails
# Get balance of an account
GET \AccountDetails\{id}\CurrentBalance
However, there's really no good reason to go through the trouble of doing that. Your PUT is idempotent as long as making the same request multiple times doesn't change the state of the system. Not changing the system's state if some spurious value is submitted is the correct behavior. In fact, if someone does try a PUT including CurrentBalance, you might want to return a 400 (Bad Request) status explaining that CurrentBalance can't be updated.
I want to make a small plugin for awesome WM that will show a number of unread messages pending in Thunderbird. I want to fetch the number of messages accessing directly Thunderbird sqlite base. The question is: which base, table and fields should I query?
There are at least 15 bases under ~/.thunderbird/profile/, including ./global-messages-db.sqlite. In this base I tried messageAttributes table, but with no big success. I could not find development documentation describing the attributes...
Any help here?
You will find what you need in the global-messages-db.sqlite file. If you watch the messages table, you will find a column jsonAttributes. In there you will find a JSON array, from attribute ids and their value. The key 58 is the read-status of a message. So if you find something like {"58": false} in this column the message is still unread. But this database won't be updated immediately when a new message is received. (It might even be updated only, when you close Thunderbird -- I am not sure about that.)
So as you see finding unread messages that way will be a bit of the hard way to go. I would recommend you to better create a plugin, that is directly checking the server via IMAP or POP3.
For IMAP servers there already exist an awesome-plugin inside the Delightful Extensions. I don't know of any POP3 plugin, and as it seems POP3 libs for lua are also rare to find.
I've got a site that uses an order entry form and sends a rather decently sized POST request when the form is submitted.
However, when a particular value is passed in one of our form variables (OrderDetail), every time without fail, it gets an error page in the browser and a 504 error via Fiddler.
Here are a couple examples of tests I ran last night sending POST requests through Fiddler. When the "OrderDetail=" value is changed to the below it will either submit successfully or return a 504 error after a few seconds:
These ones FAIL:
&OrderDetail=Deliver+Writ+of+Execution%3B+and+Application+for+Earnings+Withholding+Order+to+Los+Angeles+County+Sheriff+DASH+Court+Services+Division+per+instructions
&OrderDetail=Deliver+Execution+Earnings+Withholding+Order+to+Los+Angeles+County+Sheriff+DASH+Court+Services+Division+per+instructions
&OrderDetail=Deliver+Writ+of+Execution%3B+and+Application+for+Earnings+Withholding+Order+to+Los+Angeles+County+Sheriff
&OrderDetail=Deliver+Writ+of+Execution%3B+Application+for+Earnings+Withholding+Order+to+Los+Angeles+County+Sheriff
&OrderDetail=Writ+of+Withholding+Execution+Order+Los+Angeles+County+Sheriff
&OrderDetail=writ+Execution+adsfsdfsdfsd+Order+County
&OrderDetail=wd+Execution+adsfsdfsdfsd+Order+Count
This got me thinking that perhaps it has to do with the words "Exec" ('Exec' and 'Execution' throw errors, 'Exe' does not) and "Count" ('County' and 'Count' throw errors, 'Cont' does not)
However, I haven't seen anything this specific mentioned in google searches regarding the 504 error.
Regarding the Coldfusion code around this, there is nothing fancy for this page. Just a standard form post. I added a cfmail test in the Application file and on these failures it is never ran, so this seems to be between the browser and IIS. We're on a shared server, so I can't see too much there, though.
Oddly enough, when the &OrderDetail= param is changed to one of these values (very similar to the above), the result is success:
&OrderDetail=wd+Execution+adsfsdfsdfsd+Order+Coun
&OrderDetail=wd+Execution+adsfsdfsdfsd+Order+Conty
&OrderDetail=Writ+of+Withholding+Order+Execution+Los+Angeles+County+Sheriff
&OrderDetail=Writ+of+Withholding+ExecutionOrder+Los+Angeles+County+Sheriff
In the 3rd one, I put 'Order' BEFORE 'Execution' and it works..
The total length of this POST request is about 4720 characters. I've increased the length of this one field to 5-6 times its length and they passed, so it almost seems tied to the value of the "&OrderDetail" param in the POST.
Any ideas on why this specific data could be an issue for a web server? I've never seen this before and it doesn't continue to be a problem for nearly any other request going through.
One interesting note as well: In the POST request, this variable is pretty close to the start of the param list. If I delete everything after it, it goes with no problem. Although I haven't been able to nail down what in the subsequent lines could be causing it. I can post the entire request if it will help.
More importantly though, I just want to know what could qualify as "reserved" or "illegal" for FORM data. Everything appears to be escaped properly so I'm not sure what else can be done here except for some pre-processing javascript to further escape any such words.
Thanks!
Given that EXEC and COUNT are causing the error, whilst putting ORDER before EXEC is preventing the error, this sounds like something is making a flawed attempt at protecting from SQL injection attacks.
If you have any software in place that claims to do that, I would see if (temporarily) disabling it stops the problem from occurring.
(This software might be at the firewall level, so you may need to talk to your sys admins.)
Importantly, I would also check your codebase for where OrderDetail is used, and make sure that it is using cfqueryparam whenever it is used inside a query - and the same goes for all other user-supplied data.