I have seen multiple sources stating that compressing before encrypting is the best way to increase security.
But what I am curious is that if you compress, encrypt and than compress again, does it provide better security, or does it not matter?
I do not care about efficiency, just the security.
Compression after encryption is pointless, since it won't compress. Encrypted data is indistinguishable from random data to the compressor, so the result will be very slightly expanded instead of compressed.
Related
With plain HTTP, cookieless domains are an optimization to avoid unnecessarily sending cookie headers for page resources.
However, the SPDY protocol compresses HTTP headers and in some cases eliminates unnecessary headers. My question then is, does SPDY make cookieless domains irrelevant?
Furthermore, should the page source and all of its resources be hosted at the same domain in order to optimize a SPDY implementation?
Does SPDY make cookieless domains irrelevant?
Sort of, mostly... But not entirely.
First off, there are at least two good reasons for using "cookieless domains": one is to avoid the extra headers and reduce the size of the request, second is to avoid leaking any private or secure information about the user. Each is valid independent of each other. So with that, clearly there is still a reason to have a "cookieless domain" under HTTP 2.0 for security and privacy.
Further, compression is not a magic bullet either. Establishing a compression / decompression context is not free, and depending on the used compression scheme, allocated buffer sizes, etc, a large cookie could completely destroy the performance of the compressor. Up to spdy/v3, a gzip compressor (sliding window) was used, and given a large enough cookie, you would have a negative impact on performance of the compressor (degree varies by browser, based on implementation). In spdy/v4, the gzip compressor is out and an entirely new algorithm is being implemented from scratch -- since v4 is not out yet, it's too early to speculate about the specifics of performance. Having said that, in most cases, you should be fine.. I'm just highlighting the edge cases.
Furthermore, should the page source and all of its resources be hosted at the same domain in order to optimize a SPDY implementation?
Yes, to the extent possible - that'll give you best performance. There are caveats here as well: high packet loss to origin server, or high BDP product without window scaling. But chances are, if you're using a reasonable hosting provider with good connectivity, neither of these should be an issue.
I am wondering if I could use XML as the application-level protocol. The XML will just sit on-top of the TCP protocol and the 2 applications will just need to know how to parse XML.
But the problem is, XML is not compact enough. Should I use some algorithm to compact the XML payload before send it with TCP? If I do compact it, then the compressing/decompressing cost will be involved on both ends. What a dilemma. Any good suggestions? Or I take the wrong approach?
Many thanks.
Define "compact enough", have you measured it to be too slow for your application? Avoid premature optimisation.
As with any protocol, there are trade-offs in various directions. XML buys you a well-known cross-platform format with libraries for just about any language, that's capable of representing all kinds of structured data. XMPP opts for this, and uses optional compression for bandwidth-constrained setups. Experiments in the XMPP world with alternate representations have rarely proven worth the effort.
A notch down from XML, but still providing many of the advantages is JSON. While it lacks namespacing it's fairly simple, and libraries are near as common as XML ones. Still, JSON is text-based and may still be verbose for some situations.
The last sensible choice would be a binary protocol. This has advantages in that you can tailor and optimise it specifically to your application. The disadvantages are that you have to write the parsing and serialization yourself, although there are tools to automate this such as Google's Protocol Buffers project.
Ultimately all of these are suitable in different places, and the choice is up to the application developer for which one they should use for a given project.
I'd say that you can use straight XML for your messages and don't bother with compression or anything (XML isn't exactly blazingly fast) if you need a simple way of specifying how your messages look. If you need something faster and more compact, take a look at Google Protocol Buffers or Thrift (I personally prefer Protocol buffers, but everyone is not like me..).
Using XML can for instance be a good way if you need to make your interface be easy to interoperate with lots of different clients for instance (such as Web Services). If on the other hand you are always in charge of both ends of the communication, you can always do something quick to get going and then optimize later.
I was wondering if there is any difference in performance (or any other important factor) between a file sent to the browser from our server by this method :
For i = 1 To fileSize \ chunk
If Not Response.IsClientConnected Then Exit For
Response.BinaryWrite stream.Read(chunk)
Response.Flush
Next
VS
the old plain file access method that the IIS comes with.
We are working on a file manager handler for security reasons and would like to know what is the performance hit.
Unless you are dealing with a fairly large file, there shouldn't be a noticeable difference. Since you are creating the chunks manually and the flushing the buffer, you are going to have more packet traffic to the client (the payload of the packet or the last packet will be only partially full). However, as I said, this probably won't be noticeable unless you have a large file and even then it's not likely to show up.
Both methods need to push binary data to the browser.
would like to know what is the performance hit.
Like always in such cases: measure. Try to optimize settings on IIS and measure again until you get the most optimal solution.
In my experience chunking stuff down using script is significantly slower than IIS's highly optimised static file handling. How much slower depends on too many factors I've seen up to 10 times when other poor choices have been made (like using 4096 bytes as the buffer size).
Some things may have improved on IIS7 but if you can fetch the file as static content I would definitely go with that.
I'm wondering if anyone has a clue on whether the gzip compression is as much useful on mobile devices than it is on a desktop computer.
Will the phone use more battery?
Or will it save some because of the bandwidth saving?
Will the page page load faster or is the uncompress process slow on those limited devices?
Does the compressed data actually reach the end-user or is it uncompressed somewhere by the 3G provider? (this may be a stupid question, sorry).
Thank you.
Not a stupid question at all.
The correct trade-of is in favor of GZip.
It turns out that the Lempel-Ziv decompression is fairly cheap (much unlike the compression), while bandwidth is usually quite expensive, esspecially for roaming consumers, and also takes much battery power and transfer time.
It always depends on where your bottleneck is.
If it is a very weak cpu, anything
that puts a bigger burden on it is
bad.
If is your network connection,
compressed data transfer is a huge
performance boost.
The strain on the battery should be negilible in any case
With today's mobile devices, cpu power is certainly weaker that that of a desktop pc, but usually strong enough for gzip compression and decompression. In most cases, the bottleneck will be the network connection, so gzip compression is certainly useful. There will be rare cases though, where the opposite is true.
You just need to use a little common sense to see if my answer applies to your special case ;-)
One question you may also want to investigate is whether or not the mobile browsers you are considering even support compression. For example, I just checked the request headers sent by my BlackBerry Storm and it does not send any "Accept-Encoding" headers -- which means the server should not send back a compressed response.
There is an interesting problem at hand. I have a role-playing MMOG running through a client application (not a browser) which sends the actions of my player to a server which keeps all the players in sync by sending packets back.
Now, the game uses a top layer protocol over TCP/IP to send the data. However, wireshark does not know what protocol is being used and shows everything beyond the TCP header as a dump.
Further, this dump does not have any plain text strings. Although the game has a chat feature, the chat string being sent is not seen in this dump as plain text anywhere.
My task is to reverse engineer the protocol a little to find some very basic stuff about the data contained in the packets.
Does anybody know why is the chat string not visible as plain text and whether it is likely that a standard top level protocol is being used?
Also, are there any tools which can help to get the data from the dump?
If it's encrypted you do have a chance (in fact, you have a 100% chance if you handle it right): the key must reside somewhere on your computer. Just pop open your favorite debugger, watch for a bit (err, a hundred bytes or so I'd hope) of data to come in from a socket, set a watchpoint on that data, and look at the stack traces of things that access it. If you're really lucky, you might even see it get decrypted in place. If not, you'll probably pick up on the fact that they're using a standard encryption algorithm (they'd be fools not to from a theoretical security standpoint) either by looking at stack traces (if you're lucky) or by using one of the IV / S-box profilers out there (avoid the academic ones, most of them don't work without a lot of trouble). Many encryption algorithms use blocks of "standard data" that can be detected (these are the IVs / S-boxes), these are what you look for in the absence of other information. Whatever you find, google it, and try to override their encryption library to dump the data that's being encrypted/decrypted. From these dumps, it should be relatively easy to see what's going on.
REing an encrypted session can be a lot of fun, but it requires skill with your debugger and lots of reading. It can be frustrating but you won't be sorry if you spend the time to learn how to do it :)
Best guess: encryption, or compression.
Even telnet supports compression over the wire, even though the whole protocol is entirely text based (well, very nearly).
You could try running the data stream through some common compression utilities, but I doubt that'd do much for you, since in all likelihood they don't transmit compression headers, there's simply some predefined values enforced.
If it's infact encryption, then you're pretty much screwed (without much, much more effort that I'm not even going to start to get into).
It's most likely either compressed or encrypted.
If it's encrypted you won't have a chance.
If it's compressed you'll have to somehow figure out which parts of the data are compressed, where the compressed parts start and what the compression algorithm is. If your lucky there will be standard headers that you can identify, although they are probably stripped out to save space.
None of this is simple. Reverse engineering is hard. There aren't any standard tools to help you, you'll just have to investigate and try things until you figure it out. My advice would be to ask the developers for a protocol spec and see if they are willing to help support what you are trying to do.