I am new to backend service and I was tracing API calls from a banking website. Normally, I have seen parameters in POST requests being encoded with base64 encoding. However, I came across a type of request where the date was encrypted with a type of encoding that I am unable to figure out.
For date: 19/12/2018 the encoding is: U2FsdGVkX1/o1qw9zIiZBHLAbGck6j15wwUZ/z/zLqw=
and for date: 18/12/2019 the encoded string is: U2FsdGVkX1/mo5+FfuqqqbUtCsdFObB8eKvyosc4b8E=
I am aware of only base64 encoding, but since I am unable to decode this with base64, it seems this is using something else. Appreciate if anyone can help and share some knowledge about the different ways in parameters can be sent to backend in a secret manner.
The encoding/encryption seems to be happening from the frontend side and I feel this could also be an encrypted string with seed sent in a separate parameter. Appreciate if someone can atleast share a list of possible algorithms that I can look into to understand the request sent and create my own requests.
Related
I would decode this message sent to our national tax office. I have no clue how to start it. Seems it is a SAML encoding, I do not know much about SAML.
But in the content either xml or json is.
I tried this website to decode, but it did not do it.
https://www.samltool.com/decode.php
How to decode?
The HTTP-Redirect binding uses the DEFLATE encoding. The SAML message has been sent using the HTTP-Post binding. This means that the form data will include either a SAMLRequest or SAMLResponse form variable. This form variable must first be URL-decoded and then base-64 decoded to get the resultant XML.
I am using ruby to send a SOAP request to a very enterprisey bla bla service, so unfortunately I can not attach any samples, there's nobody to send any server-side logs, nobody knows whats wrong on the provider side or how the actual HTTP requests need to look like (except a single XML example I got, but no HTTP headers), the docs are very Microsoft-centric with C# examples and whatnot ("instantiate AbstractFactoryFactory..." and whatnot), long live enterprise software.
But the bottom line is, eventually I took one of their own XMLs from their logs and sent it via HTTP to the endpoint from the WSDL and sent it to their host using the Savon gem raw XML option and got a HTTP 500 error from their host and a bunch of non-ascii binary data inside - literally, no ASCII characters are in the body.
I guessed that maybe Savon does some bad magic or that the XML option is not working as expected and I tried sending the same request via Faraday, but got the same thing,
the HTTP response headers says it's a HTTP response, XML encoded, from an ASP.NET host:
"content-type"=>"text/xml; charset=utf-8",
"server"=>"Microsoft-IIS/7.5",
"x-aspnet-version"=>"2.0.50727",
"x-powered-by"=>"ASP.NET",
but again, a 440 bytes worth of binaries in the response:
method=:post,
body=
"\x1F\x8B\b\x00\x00...
etc.
Am I missing some weird aspect of the SOAP specification and I need to do something to decode this data or has their server gone bonkers from my XML, HTTP headers or something else and I need to ping the provider?
Update 1
I noticed that their original XML had UTF-16 encoding set, so I tried encoding the raw string to UTF-16, then had Savon spew errors at me about bad data, then I updated encoding in the Savon client config. But I still get HTTP 500 error and binaries as response and if I try to log anything Savon reports a bug:
Encoding::CompatibilityError: incompatible encoding regexp match (US-ASCII regexp with UTF-16 string)
from /home/bbozo/.rvm/gems/ruby-2.2.4/gems/savon-2.11.1/lib/savon/log_message.rb:13:in `to_s'
Faraday basically reported the same behavior, an binary blob.
Update 2
I tried piping the encoding to every known encoding, and got nothing, even though the HTTP headers imply the encoding is UTF-8, it obviously isn't
Encoding.name_list.map{ |e_in| [ e_in, ( response.body.dup.force_encoding(e_in).encode('utf-8') rescue 'incompatible' ) ] }
There is nothing that would indicate the encoding in the WSDL files, the API spec doesn't even mention encoding except that the request XMLs need to be UTF-8 encoding, I tried encoding the body, changing the XML encoding definition, HTTP headers, but still I get the same binary blob, with the same heading (\x1F\x8B\b\x00\x00) - so it's not some weird encryption either.
Compression maybe?
I tried with https for good measure and nothing.
Question
Am I missing some weird aspect of the SOAP specification and I need to do something to decode this data or has their server gone bonkers from my XML, HTTP headers or something else and I need to ping the provider?
The response body was compressed! In the end I just gunzipped it and there it was,
How to decompress Gzip string in ruby?
I'm looking at HTTP traffic, and seeing a POST request that is "application/x-www-form-urlencoded" with strange characters in it:
t=%d8%94%b2%bc+%cb%ae%c9%ab
Various URL decodes that I tried running on this complain that it's not UTF-8 encoded.
I know that in theory a web request can be encoded in any given encoding scheme, but if so, how can I tell which scheme is being used (in order for me to decode it properly)?
I see no hint in the request itself.
This is not a language-specific question (more of a question about the HTTP protocol, I think.)
Any help will be appreciated!
Your question actually contains the answer (form-urlencoded). The request is URL encoded. See this link for more information.
The strange characters decode to non-ASCII characters except for the '+'. There are decoders on the Web where you can cut and paste your text to see what it really is.
The link mentioned by Craig is correct. It decodes to Ø”²¼+ˮɫ
In an ASP.Net web app, which runs on HTTPS and has RequireClientCertificate set in web.config, I need to receive the client certificate of the user and digital signature of the request on the server. The certificate is found in HttpContext.Request.ClientCertificate, but I can't find the signed data. The post params are automatically decoded and decrypted, but I need the signature too. Does anyone know where is it found or is it possible to get it?
One more question, when the browser asks for your certificate and hands it to the server, does it encrypt the whole HttpRequest with your private key or just a part of it (for example post params)?
Thanks for any help
One more question, when the browser asks for your certificate and
hands it to the server, does it encrypt the whole HttpRequest with
your private key or just a part of it (for example post params)?
Firstly, it doesn't make sense to "encrypt with a private key": you sign with a private key. While some algorithms (e.g. RSA) use very similar procedures to encrypt and sign, "encrypt" means "hiding" something: you're not hiding anything if anyone with the public key can decipher it.
Secondly, SSL/TLS uses symmetric keys (negotiated during the handshake) for encryption, not the keys in the certificates. The whole HTTP request will indeed be encrypted in this case.
The certificate is found in HttpContext.Request.ClientCertificate, but
I can't find the signed data. The post params are automatically
decoded and decrypted, but I need the signature too. Does anyone know
where is it found or is it possible to get it?
What's signed when using a client certificate is the handshake messages, not the HTTP request. Once the appropriate verification has been made by your SSL/TLS stack, it's unlikely to be of any use, either technically or administratively. (This is why it is generally not accessible.)
I have a program that is supposed to interact with a web server and retrieve a file containing structured data using http and cgi. I have a couple questions:
The cgi script on the server needs to specify a body right? What should the content-type be?
Should I be using POST or GET?
Could anyone tell me a good resource for reading about HTTP?
If you just want to retrieve the resource, I’d use GET. And with GET you don’t need a Content-Type since a GET request has no body. And as of HTTP, I’d suggest you to read the HTTP 1.1 specification.
The content-type specified by the server will depend on what type of data you plan to return. As Jim said if it's JSON you can use 'application/json'. The obvious payload for the request would be whatever data you're sending to the client.
From the servers prospective it shouldn't matter that much. In general if you're not expecting a lot of information from the client I'd set up the server to respond to GET requests as opposed to POST requests. An advantage I like is simply being able to specify what I want in the url (this can't be done if it's expecting a POST request).
I would point you to the rfc for HTTP...probably the best source for information..maybe not the most user friendly way to get your answers but it should have all the answers you need. link text
For (1) the Content-Type depends on the structured data. If it's XML you can use application/xml, JSON can be application/json, etc. Content-Type is set by the server. Your client would ask for that type of content using the Accept header. (Try to use existing data format standards and content types if you can.)
For (2) GET is best (you aren't sending up any data to the server).
I found RESTful Web Services by Richardson and Ruby a very interesting introduction to HTTP. It takes a very strict, but very helpful, view of HTTP.