I am trying to open a file used by a reading application on my Kindle HD, and I am trying to export the ton of highlights that i have within the app.
When opening the .mrstd file with sublime text it looks like :
"504b 0304 1400 0808 0800 a46a 8e45 0000
0000 0000 0000 0000 0000 1d00 0000 636f
6d2e 666c 7965 7273 6f66 742e 6d6f 6f6e
7265 6164 6572 2f6c 6962 2f03 0050 4b07
0800 0000 0002 0000 0000 0000 0050 4b03" and on and on.
what is this & how do i crack it?
please,
thank you,
nicoara
TL;DR it's a zip file.
In linux, you can use the command "file" which will try to identify the file format.
I asked the same question you did, so I ran file a.mrstd and got a.mrstd: Zip archive data, at least v2.0 to extract
If you have windows instead, you can run linux commands using WSL, google for it.
Related
In Operating Systems Design and Implementation by Andrew S. Tanenbaum and Albert S. Woodhull, there's the following fragment:
In MINIX 3 every file has an 11-bit mode used for protection. Nine of these bits are the read-
write-execute bits for the owner, group, and others.
And then, a few lines after, they write:
The other two protection bits, 02000 [octal 200] and 04000 [octal 400], are the SETGID (set-group-id) and SETUID (set-
user-id) bits, respectively.
But Python shows that octal 400 is a 12-bit long mask:
>>> len(str(bin(0o4000))) - len('0b')
12
How can a 12-bit long mask be applied on a 11-bit field?
01000 is the "sticky" bit in Unix, and Minix didn't support it at the time the book was released. It didn't add support until 2010 (the book was released in 2005).
I used ethereal to capture some packets and I'm trying to make sense of the data. This is the output of 1 of the frames.
0x0000: 4500 003a 4564 4000 4006 df05 0a01 012b E..:Ed#.#......+
0x0010: 0a01 0128 bfcf 3a98 e938 b9c8 e8a0 761f ...(..:..8....v.
0x0020: 8018 05b4 ef33 0000 0101 080a 0005 ff31 .....3.........1
0x0030: 0005 2c31 6865 6c6c 6f0a ..,1hello.
The characters on the right is the ascii representation of the info on the left (Which is hex). The message being sent is "hello" and the rest of the information (or atleast some of it) is the header, right? What's the format of the header? I did some googling and found that this image: http://electronicdesign.com/site-files/electronicdesign.com/files/archive/electronicdesign.com/files/29/2099/figure_03.gif. Is this correct? According to the image, bytes 27-30 (761f 8018) are the destination IP but converting the hex to decimal doesn't end up being my ip. Am I missing something or am I completely wrong? Also how would I find the protocol number?
Are your source ip 10.1.1.43 and destination ip 10.1.1.40?
I believe the info shown is not the ethernet frame, it is an IP package. The first octet is already the IP header.
To get a quick reference to the IP and TCP headers, you can use wiki
http://en.wikipedia.org/wiki/IPv4#Header
http://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure
But if you want more details and reliable information, you should check the RFC directly.
https://www.rfc-editor.org/rfc/rfc791#section-3.1
https://www.rfc-editor.org/rfc/rfc793#section-3.1
If I rearrange your data in a 4 octets per row format
4500 003a total length
4564 4000 fragment
4006 df05 header checksum
0a01 012b source IP
0a01 0128 destination IP
bfcf 3a98 ports
e938 b9c8 seq
e8a0 761f ack
8018 05b4 windows
ef33 0000 checksum
0101 080a
0005 ff31
0005 2c31
6865 6c6c
6f0a
The first octet is 4 so you should be using IPv4. On the 3rd row, the second octet is 6. This is the protocol number for TCP. From these two hints i guess this is IP package not ethernet package.
And about ethereal, Remy is right, Use Wireshare https://www.wireshark.org/
I am trying to read in a csv from the net. When I run
library(data.table)
rawfile<-fread("http://www.pjm.com/pub/account/genoutages/20140116_genout.csv",stringsAsFactors=FALSE,skip=2)
rawfile
I get
Empty data.table (0 rows) of 2 cols: File Posted on 16-Jan-2014 04:00,V2
But if I download the file to my computer first and then do
rawfile<-fread("c:\\temp\\20140116_genout.csv",stringsAsFactors=FALSE,skip=2)
rawfile
then I get the output that I expect.
I know this is easy to work around by I just wanted to see if this was a bug or if there was some nuance that was going over my head.
UPDATE Sep 2014 : Now fixed in v1.9.3
URLs now work on Windows. R's download.file() converts \r\n to \r\r\n on Windows. Now avoided by downloading in binary mode. Thanks to Steve Miller and Dean MacGregor for reporting, #492.
Thanks for reporting. It worked fine for me when I tried just now on Linux. See output below.
?fread confirms that URLs are supported: "A filename input is passed through path.expand for convenience and may be a URL starting http:// or file://."
Given the comments, seems like a Windows only problem then. Have raised a bug here :
#5288 fread reads URLs on Linux but not Windows?
Here's the output from it working on Linux :
fread("http://www.pjm.com/pub/account/genoutages/20140116_genout.csv",stringsAsFactors=FALSE,skip=2)
trying URL 'http://www.pjm.com/pub/account/genoutages/20140116_genout.csv'
Content type 'application/octet-stream' length 2767 bytes
opened URL
==================================================
downloaded 2767 bytes
Date Forecasted Generation Outage Western Region Other Regions
1: 1/17/2014 1543 374 1169
2: 1/18/2014 1520 374 1146
3: 1/19/2014 1520 374 1146
4: 1/20/2014 1520 374 1146
5: 1/21/2014 1520 374 1146
6: 1/22/2014 1520 374 1146
7: 1/23/2014 1506 374 1132
.. snip
I'm trying to decrypt a XOR encrypted file, after running the key length test using xortool I got this key: "fallen"..
# python xortool.py -c 00 /cygdrive/c/Users/Me/Desktop/ch3.bmp
The most probable key lengths:
1: 10.6%
3: 11.6%
6: 18.5%
9: 8.8%
12: 13.8%
15: 6.6%
18: 10.4%
24: 8.1%
30: 6.4%
36: 5.2%
Key-length can be 3*n
1 possible key(s) of length 6:
fallen
Whatever is there a way to decipher the file (a bmp file) and get the original one, using tools like openssl or gpg?? Do they have a XOR operation?
Neither OpenSSL nor GPG have such XOR functionality that I'm aware of, however writing a program to do it yourself should be trivial.
Given that you know that the file is a .bmp, you should be able to use this fact to decrypt the file quite easily, especially given that .bmp files have a well defined structure. For example, the first two bytes when decrypted should be 0x42, 0x4D (that's ASCII BM), and the following 4 bytes are the (big-endian) size of the entire file in bytes, so you should be able to get at least 6 bytes of the key immediately.
Since you already have xortool, just use xortool-xor from the xortool distribution:
python xortool/xortool-xor -s fallen /cygdrive/c/Users/Me/Desktop/ch3.bmp > decoded.bmp
Also note that xortool itself saves the decoded output in the xortool_out folder, so after using xortool to find the key, you could just do:
mv xortool_out/0_fallen decoded.bmp
I have used both, and I conclude that I can read html data from webpages with tcpflow but cannot do so with tcpdump. The best I get is some ugly ASCII text with lots of period symbols.
My understanding is that tcpdump doesn't reassemble packets, whereas tcpflow does. But if that was the key difference, wouldn't the packet data from tcpdump still be human readable - just in smaller chunks? Is the problem that tcpdump is limited to ASCII and most network traffic is encoded in UTF-8?
I'm a rookie on network analysis/programming so forgive me if I'm missing something obvious.
To get that encrypted data one should use tcpdump with option tcpdump –A (capital a). It transfers text without any headers and is used mainly for web pages. Hence we get response page easily.
I think you are getting confused between an application layer and transport layer packet.
I do not know about tcpflow but tcpdump capture the whole packet (including header and all other stuff) not just the data.The html data which you are mentioning would be in the data part of a tcp/udp/icmp packet whichever you are using and so it needs you to understand the structure of tcp/udp/icmp packet as well ...
I capture this packet on my machine and HTML data is clearly visible , you need to write script to get it from the output with a knowledge of packet structure.
16:15:17.968325 IP 172.31.9.84.80 > 172.31.9.177.56559: P 2575928381:2575928808(427)
ack 65285447 win 1716 <nop,nop,timestamp 21031622 15165633>
0x0000: 4500 01df 66fd 4000 4006 66d8 ac1f 0954 E...f.#.#.f....T
0x0010: ac1f 09b1 0050 dcef 9989 8c3d 03e4 2d47 ..... P.....=..-G
0x0020: 8018 06b4 1a2a 0000 0101 080a 0140 eac6 .....*.......#..
0x0030: 00e7 68c1 4854 5450 2f31 2e31 2032 3030 ..h.HTTP/1.1.200
0x0040: 204f 4b0d 0a44 6174 653a 2046 7269 2c20 .OK..Date:.Fri,.
0x0050: 3130 204a 756c 2032 3030 3920 3130 3a32 10.Jul.2009.10:2
0x0060: 303a 3136 2047 4d54 0d0a 5365 7276 6572 0:16.GMT..Server
0x0070: 3a20 4170 6163 6865 2f32 2e30 2e35 3220 :.Apache/2.0.52.
0x0080: 2852 6564 2048 6174 290d 0a4c 6173 742d (Red.Hat)..Last-
0x0090: 4d6f 6469 6669 6564 3a20 4672 692c 2031 Modified:.Fri,.1
0x00a0: 3020 4a75 6c20 3230 3039 2030 393a 3331 0.Jul.2009.09:31
0x00b0: 3a30 3420 474d 540d 0a45 5461 673a 2022 :04.GMT..ETag:."
0x00c0: 3266 6231 3333 2d37 372d 6131 3935 3436 2fb133-77-a19546
0x00d0: 3030 220d 0a41 6363 6570 742d 5261 6e67 00"..Accept-Rang
0x00e0: 6573 3a20 6279 7465 730d 0a43 6f6e 7465 es:.bytes..Conte
0x00f0: 6e74 2d4c 656e 6774 683a 2031 3139 0d0a nt-Length:.119..
0x0100: 4b65 6570 2d41 6c69 7665 3a20 7469 6d65 Keep-Alive:.time
0x0110: 6f75 743d 3135 2c20 6d61 783d 3130 300d out=15,.max=100.
0x0120: 0a43 6f6e 6e65 6374 696f 6e3a 204b 6565 .Connection:.Kee
0x0130: 702d 416c 6976 650d 0a43 6f6e 7465 6e74 p-Alive..Content
0x0140: 2d54 7970 653a 2074 6578 742f 6874 6d6c -Type:.text/html
0x0150: 3b20 6368 6172 7365 743d 4953 4f2d 3838 ;.charset=ISO-88
0x0160: 3539 2d31 0d0a 0d0a 3c68 746d 6c3e 0a0a 59-1....<html>..
0x0170: 3c62 6f64 793e 0a49 6e64 6961 6e20 696e <body>.Indian.in
0x0180: 7374 6974 7574 6520 6f66 2074 6563 686e stitute.of.techn
0x0190: 6f6c 6f67 7920 2e2e 2e2e 2e20 7468 6973 ology.......this
0x01a0: 2069 7320 7468 6520 7465 7374 2070 6167 .is.the.test.pag
0x01b0: 652e 0a3c 4120 6872 6566 3d22 6164 2e68 e..<A.href="ad.h
0x01c0: 746d 6c22 3e61 6263 643c 2f41 3e0a 0a3c tml">abcd</A>..<
0x01d0: 2f62 6f64 793e 0a3c 2f68 746d 6c3e 0a /body>.</html>.
The last 7-8 lines describe the html data.
use -s0 to capture whole frame and -X to print in above ASCII human readable format.
To get that encrypted data one should use TCPDUMP with option –A (capital a). It transfers text without any headers and is used mainly for web pages. Hence we get response page easily.
For eg:
I request index.html to 172.31.9.84 at port 80
Then I requested GET/index.html (an example page that contains only text “Indian institute of technology this is the test page”)
At this moment when I captured packets I got something as:
Request:
18:10:42.387426 IP 172.31.9.177.42943 > 172.31.9.84.80:
P 1:15(14) ack 1 win 46 <nop,nop,timestamp 6644101 353753635>
E..B.G#.#........T...P^R.Mb.L.....ke.....
.ea....#GET /index.html
Response:
18:10:42.388127 IP 172.31.9.84.80 > 172.31.9.177.42943: P 1:53(52)
ack 15 win 1448 <nop,nop,timestamp 353758368 6644101>
E..h..#.#.8m..T.. ..P..b.L.^R.[....d......
.....ea.<html>
<body>
Indian institute of technology this is the test page
</body>
</html>
When trying to get HTTP data using tcpdump, you will see clear text in ASCII if you apply the -A option. However, a HTTP data is almost always compressed with gzip mode or other. You can see it in headers:
Content-Encoding: gzip, compress, bzip2
Thus, tcpdump will output in your screen several bytes and they are the compressed data!!! You will need to use tshark or wireshark to see flat data. Then your problem isn't UTF8.
You can test tcpdump data display disabling deflate module in Apache web server.
Hope this help.