Are there any published extensions to PKCS#12? - encryption
PKCS#12 is a convenient way to lump together a private key with its corresponding X.509 certificate into a standardized single file format. However, the specification was published by RSALabs in 1999 and uses only RC4, RC2 and TripleDES for symmetric encryption. Are there any common semi-standard extensions to the scheme that add more encryption algorithms or other key derivation functions? OpenSSL is documented to implement support for AES and Camellia, but a search for a corresponding standard turns up blank, so this seems to be something implementation specific to OpenSSL. Has anyone documented the ASN.1 module and pseudo code for these extensions?
PKCS#12 uses building-blocks from other standards.
The recommended encryption-mode is based on password based encryption from PKCS#5 (PBES2). This has been extended with support for SHA-2 and AES in PKCS#5 v.2.1.
When OpenSSL uses AES it does it like this:
684 30 806: SEQUENCE {
688 30 802: SEQUENCE {
692 06 11: OBJECT IDENTIFIER
: pkcs-12-pkcs-8ShroudedKeyBag (1 2 840 113549 1 12 10 1 2)
705 A0 723: [0] {
709 30 719: SEQUENCE {
713 30 73: SEQUENCE {
715 06 9: OBJECT IDENTIFIER
: pkcs5PBES2 (1 2 840 113549 1 5 13)
726 30 60: SEQUENCE {
728 30 27: SEQUENCE {
730 06 9: OBJECT IDENTIFIER
: pkcs5PBKDF2 (1 2 840 113549 1
5 12)
741 30 14: SEQUENCE {
743 04 8: OCTET STRING
: BA 6B 5B B3 47 27 C9 73
753 02 2: INTEGER 2048
: }
: }
757 30 29: SEQUENCE {
759 06 9: OBJECT IDENTIFIER
: aes128-CBC (2 16 840 1 101 3 4 1 2)
770 04 16: OCTET STRING
: 0F 79 79 0A D3 EC C0 3E 20 B8 51 85 2F 2B 6C 29
: }
: }
: }
As far as I can read the source, OpenSSL encodes the password as ASCII rather than zero-terminated UTF-16 when using PKCS#5 PBES2.
Related
Decoding Thrift Object what are these extra bytes?
I'm working on writing a pure JS thrift decoder that doesn't depend on thrift definitions. I have been following this handy guide which has been my bible for the past few days: https://erikvanoosten.github.io/thrift-missing-specification/ I almost have my parser working, but there is a string type that throws a wrench into the program, and I don't quite understand what it's doing. Here is an excerpt of the hexdump, which I did my best to annotate: Correctly parsing: 000001a0 0a 32 30 32 31 2d 31 31 2d 32 34 16 02 00 18 07 |.2021-11-24.....| ........................blah blah blah............| | | Object End-| | | 0x18 & 0xF = 0x8 = Binary-| | The binary sequence is 0x7 characters long-| 000001b0 53 65 61 74 74 6c 65 18 02 55 53 18 02 55 53 18 |Seattle..US..US.| S E A T T L E |___| U S |___| U S Another string, 2 bytes long |------------| So far so good. But then I get to this point: There string I am trying to extract is "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4592.0 Safari/537.36 Edg/94.0.975.1" and is 134 bytes long. 000001c0 09 54 61 68 6f 65 2c 20 43 41 12 12 00 00 08 c8 |.Tahoe, CA......| Object ends here-| | | 0x8 & 0xF = 0x8 = Binary -| | 0xc8 bytes long (200)-| 000001d0 01 86 01 4d 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 |...Mozilla/5.0 (| | | | M o z i l l a ???? |--|-134, encoded as var-int 000001e0 4d 61 63 69 6e 74 6f 73 68 3b 20 49 6e 74 65 6c |Macintosh; Intel| As you can see, I have a byte sequence 0x08 0xC8 0x01 0x86 0x01 which contains the length of the string I'm looking for, is followed by the string I'm looking for but has 3 extra bytes that are unclear in purpose. The 0x01 is especially confusing as it neither a type identifier, nor seems to have a concrete value. What am I missing?
Thrift supports pluggable serialization schemes. In tree you have binary, compact and json. Out of tree anything goes. From the looks of it you are trying to decode compact protocol, so I'll answer accordingly. Everything sent and everything returned in a Thrift RPC call is packaged in a struct. Every field in a struct has a 1 byte type and a 2 byte field ID prefix. In compact protocol field ids, when possible, are delta encoded into the type and all ints are compressed down to just the bits needed to store them (and some flags). Because ints can now take up varying numbers of bytes we need to know when they end. Compact protocol encodes the int bits in 7 bits of a byte and sets the high order bit to 1 if the next byte continues the int. If the high order bit is 0 the int is complete. Thus the int 5 (101) would be encoded in one byte as 0000101. Compact knows this is the end of the int because the high order bit is 0. In your case, the int 134 (binary 10000110) will need 2 bytes to encode because it is more than 7 bits. The fist 7 bits are stored in byte 1 with the 0x80 bit set to flag "the int continues". The second and final byte encodes the last bit (00000001). What you thought was 134 was just the encoding of the first seven bits. The stray 1 was the final bit of the 134. I'd recommend you use the in tree source to do any needed protocol encoding/decoding. It's already written and tested: https://github.com/apache/thrift/blob/master/lib/nodejs/lib/thrift/compact_protocol.js
The byte sequence reads as follows 0x08: String type, the next 2 bytes define the elementId 0xC8 0x01: ElementId, encoded in 16 bits 0x86 0x01: String length, encoded as var int It turns out that if the type identifier does not contain bits defining the elementId, the elementId will be stored in the next 2 bytes.
TCPDF - Problem with Alphanumerical characters (wrong size)
I have a size problem when using TCPDF to generate QR code with only ALPHANUMERICAL characters. My objective: generate the longest URL (with a random part), but keeping the QR code at its lowest size, i.e. 21x21 modules (version1).
Documentation (QRcode.com) reports that using only alphanumerical characters set (thonky.com), URL can be 25 characters long with ERC set to L.
Using write2DBarCode with this 25 Alphanumerical URL leads to version1 (21x21mod) QR as expected
$pdf->write2DBarcode('HTTP://SITE-COM/123456789', 'QRCODE,L', 20, 20, 40, 40, $style, 'N');
but changing to this other URL, with also 25 Alphanumerical, I get a version 2 (25x25mod) QR code, whereas a version 1 could be done (Tested on Nayuki)
$pdf->write2DBarcode('HTTP://TXT-CH/AYAWEQYAF4A', 'QRCODE,L', 20, 70, 40, 40, $style, 'N');
I join the TCPDF Outputs of the 2 QR codes given as examples TCPDF Outputs
Thank you in advance for your help on this wonderful TCPDF library.
Short answer: The TCPDF software that you are using is suboptimal. It is generating a full 4-bit terminator even when a shorter one suffices. Please contact the software's authors to fix the problem. You can link them to this thread. So I cropped your image into two QR Code images, and submitted them to ZXing Decoder Online and KaarPoSoft QR Decode with debug output. ZXing, first barcode: Decode Succeeded Raw text HTTP://SITE-COM/123456789 Raw bytes 20 83 1a a6 5f 9f d5 b4 75 3e 8d 20 48 81 23 db 91 8a 80 Barcode format QR_CODE Parsed Result Type URI Parsed Result HTTP://SITE-COM/123456789 ZXing, second barcode: Decode Succeeded Raw text HTTP://TXT-CH/AYAWEQYAF4A Raw bytes 20 cb 1a a6 5f 9f d6 5e ae 82 ca 0f 21 e2 52 18 11 53 94 00 ec 11 ec 11 ec 11 ec 11 ec 11 ec 11 ec 11 Barcode format QR_CODE Parsed Result Type URI Parsed Result HTTP://TXT-CH/AYAWEQYAF4A KaarPoSoft, first barcode: Debug output skew_limit=7.21875 skew=0 left=31 right=427 top=27 bottom=423 size=397 matchVersion version=1 finder0=64 finder1=64 finder2=64 matchVersion version=1 timing0=1 timing1=1 alignment=1 matchVersion version=1 format_NW =14 0 format_NESW =14 1 format = 14 ecl = 1 mask = 6 matchVersion version=1 grades(F(V)TAF): 4444->4 findModuleSize matchVersion version=1 grade=4 findModuleSize version=1 grade=4 error_correction_level=1 mask=6 getCodewords mask=6 length=26 getCodewords = 32,131,26,166,95,159,213,180,117,62,141,32,72,129,35,219,145,138,128,62,191,105,157,147,176,164 setBlocks n_blocks_first=1 n_blocks_second=0 n_blocks=1 n_block_words_first=19 n_block_words_second=0 n_block_ec_words=7 total=26 setBlocks block 0 (26): 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25 RS calculateSyndroms: No errors correctErrors in = 32,131,26,166,95,159,213,180,117,62,141,32,72,129,35,219,145,138,128,62,191,105,157,147,176,164 correctErrors out = 32,131,26,166,95,159,213,180,117,62,141,32,72,129,35,219,145,138,128 error_grade=4 extractData bytes in (19) = 32,131,26,166,95,159,213,180,117,62,141,32,72,129,35,219,145,138,128 extractData mode = 2 extractAlphanum charcount = 16 extractData mode = 1 extractNumeric charcount = 9 extractData mode = 0 extractData data(25) = 72,84,84,80,58,47,47,83,73,84,69,45,67,79,77,47,49,50,51,52,53,54,55,56,57 KaarPoSoft, second barcode: Debug output skew_limit=7.015625 skew=1 left=21 right=417 top=30 bottom=425 size=396.5 findModuleSize matchVersion version=1 grade=0 matchVersion version=2 finder0=64 finder1=64 finder2=64 matchVersion version=2 timing0=1 timing1=1 alignment=1 matchVersion version=2 format_NW =14 0 format_NESW =14 1 format = 14 ecl = 1 mask = 6 matchVersion version=2 grades(F(V)TAF): 4444->4 findModuleSize matchVersion version=2 grade=4 findModuleSize version=2 grade=4 error_correction_level=1 mask=6 getCodewords mask=6 length=44 getCodewords = 32,203,26,166,95,159,214,94,174,130,202,15,33,226,82,24,17,83,148,0,236,17,236,17,236,17,236,17,236,17,236,17,236,17,87,194,99,197,7,184,131,204,163,52 setBlocks n_blocks_first=1 n_blocks_second=0 n_blocks=1 n_block_words_first=34 n_block_words_second=0 n_block_ec_words=10 total=44 setBlocks block 0 (44): 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43 RS calculateSyndroms: No errors correctErrors in = 32,203,26,166,95,159,214,94,174,130,202,15,33,226,82,24,17,83,148,0,236,17,236,17,236,17,236,17,236,17,236,17,236,17,87,194,99,197,7,184,131,204,163,52 correctErrors out = 32,203,26,166,95,159,214,94,174,130,202,15,33,226,82,24,17,83,148,0,236,17,236,17,236,17,236,17,236,17,236,17,236,17 error_grade=4 extractData bytes in (34) = 32,203,26,166,95,159,214,94,174,130,202,15,33,226,82,24,17,83,148,0,236,17,236,17,236,17,236,17,236,17,236,17,236,17 extractData mode = 2 extractAlphanum charcount = 25 extractData mode = 0 extractData data(25) = 72,84,84,80,58,47,47,84,88,84,45,67,72,47,65,89,65,87,69,81,89,65,70,52,65 Both QR Codes appear to have no problems regarding error correction or format violations. From the KaarPoSoft output, we can see the segments in the QR Codes. The first barcode has two segments: Alphanumeric mode, count = 16, text = "HTTP://SITE-COM/". Segment bit length = 4 (mode) + 9 (count) + 88 (data) = 101 bits. Numeric mode, count = 9, text = "123456789". Segment bit length = 4 (mode) + 10 (count) + 30 (data) = 44 bits. The second barcode has one segment: Alphanumeric mode, count = 25, text = "HTTP://TXT-CH/AYAWEQYAF4A". Segment bit length = 4 (mode) + 9 (count) + 138 (data) = 151 bits. Now, a QR Code at version 1 with low error correction level has capacity for 19 data codeword bytes, or 152 bits. The first barcode uses 101 + 44 = 145 bits = 19 bytes (rounded up), so it fits. The second barcode uses 151 bits = 19 bytes (rounded up), so it fits. So in theory, both lists of segments of text data should fit in version 1 low ECC. According to the QR spec, after the list of segments ends, these bits are appended: (TERM) Up to four "0" bits (but less if the data capacity is reached) for the terminator pseudo-mode. (BITPAD) Zero to seven "0" bits to fill up the last partial byte. (BYTEPAD) Alternating bytes of 0xEC and 0x11 until the data capacity is reached. Let's dissect what actually happened. Convert the ZXing hexadecimal byte output to binary, and annotate the fields. First barcode: 20 83 1a a6 5f 9f d5 b4 75 3e 8d 20 48 81 23 db 91 8a 80 0010 000010000 [88 bits] 0001 0000001001 [30 bits] 0000 000 (Total length = 152 bits) ^Mode ^Count ^Data ^Mode ^Count ^Data ^TERM ^BITPAD Second barcode: 20 cb 1a a6 5f 9f d6 5e ae 82 ca 0f 21 e2 52 18 11 53 94 00 ec 11 ec 11 ec 11 ec 11 ec 11 ec 11 ec 11 0010 000011001 [138 bits] | 0000 00000 11101100 00010001 [...] (Total length = 272 bits) ^Mode ^Count ^Data | ^TERM ^BITPAD ^BYTEPAD Note that in the second barcode, at the position | just before the TERMinator, there are 151 bits on the left side. The terminator is normally four "0" bits, but is allowed to be shortened if the capacity (of 152 bits) is reached. So the optimal terminator is a single bit of "0", and then there must be no bit padding and no byte padding.
looking to understand meaning of two bytes in HTTP request made with curl --trace
tl;dr "What would the bytes 0x33 0x39 0x0d 0x0a between the end of HTTP headers and the start of HTTP response body refer to?"
I'm using the thoroughly excellent libcurl to make HTTP requests to various 3rd party endpoints. These endpoints are not under my control and are required to implement a specification. To help debug and develop these endpoints I have implemented the text output functionality you might see if you make a curl request from the command line with the -v flag using curl.setopt(pycurl.VERBOSE, 1) and curl.setopt(pycurl.DEBUGFUNCTION, debug_function)
This has been working great but recently I've come across a request which my debug function does not handle in the same way as curl's debug output. I'm sure is due to me not understanding the HTTP spec.
If making a curl request from the command line with --verbose I get the following returned.
# redacted headers
< Via: 1.1 vegur
<
{"code":"InvalidCredentials","message":"Bad credentials"}*
Connection #0 to host redacted left intact
If making the same request with --trace the following is returned
0000: 56 69 61 3a 20 31 2e 31 20 76 65 67 75 72 0d 0a Via: 1.1 vegur..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a ..
<= Recv data, 1 bytes (0x1)
0000: 33 3
<= Recv data, 62 bytes (0x3e)
0000: 39 0d 0a 7b 22 63 6f 64 65 22 3a 22 49 6e 76 61 9..{"code":"Inva
0010: 6c 69 64 43 72 65 64 65 6e 74 69 61 6c 73 22 2c lidCredentials",
0020: 22 6d 65 73 73 61 67 65 22 3a 22 42 61 64 20 63 "message":"Bad c
0030: 72 65 64 65 6e 74 69 61 6c 73 22 7d 0d 0a redentials"}..
<= Recv data, 1 bytes (0x1)
0000: 30 0
<= Recv data, 4 bytes (0x4)
0000: 0d 0a 0d 0a ....
== Info: Connection #0 to host redacted left intact
All HTTP client libs I've tested don't include these parts of the bytes in the response body so I'm guessing these are part of the HTTP spec I don't know about but I can't find a reference to them and I don't know how to handle them.
If it's helpful I think curl is using this https://github.com/curl/curl/blob/master/src/tool_cb_dbg.c for building the output in the first example bit I'm not really a c/c++ programmer and I haven't been able to reverse engineer the logic.
Does anyone know what these bytes are?
0d 0a are ASCII control characters representing carriage return and line feed, respectively. CRLF is used in HTTP to mark the end of a header field (there are some historic exceptions you should not worry about at this point). A double CRLF is supposed to mark the end of the fields section of a message. The 33 39 you observe there is "39" in ascii. This is the chunk size indicator - treated as a hexdecimal number. The presence of Transfer-Encoding: chunked in the response headers may support this.
What all encryption uses == in the last?
I am aware that most of the Base64 encoding has == at the end. Is there any other which uses does the same? For example, I found this: nijdRcCHIUnketWzFbcxmvqQKKDnFW05LSE3ttTjoqyBna7JT87AwxeKdoOszXYODMRm6UfA8jK97qgV8A== But it is not a Base64 kind. What else can it be?
The string you have posted is a valid Base64 string. A Base64 string will end with == if and only if the number of bytes it encodes, mod 3, equals 1. >>> for i in range(10): ... print(i, base64.b64encode(b"\x00"*i)) ... 0 b'' 1 b'AA==' 2 b'AAA=' 3 b'AAAA' 4 b'AAAAAA==' 5 b'AAAAAAA=' 6 b'AAAAAAAA' 7 b'AAAAAAAAAA==' 8 b'AAAAAAAAAAA=' 9 b'AAAAAAAAAAAA' Do you see the pattern? It happens that 16-byte (128-bit) encryption keys are very commonly encoded in Base64, and since 16 mod 3 = 1, their encoding will end with ==. But your string, decoded, is 61 bytes (488 bits) long. That is too big to be most sorts of encryption key, and too small to be an RSA key. This is your string, decoded, and then hexdumped: 00000000 9e 28 dd 45 c0 87 21 49 e4 7a d5 b3 15 b7 31 9a |.(.E..!I.z....1.| 00000010 fa 90 28 a0 e7 15 6d 39 2d 21 37 b6 d4 e3 a2 ac |..(...m9-!7.....| 00000020 81 9d ae c9 4f ce c0 c3 17 8a 76 83 ac cd 76 0e |....O.....v...v.| 00000030 0c c4 66 e9 47 c0 f2 32 bd ee a8 15 f0 |..f.G..2.....| 0000003d I don't see anything in there to tell me what it actually is, and file(1) is also stumped. It could be random enough to be encrypted, but I can't tell for sure by eye. (And if it is random, that doesn't mean it's encrypted! It could just be the output of a random number generator.) It is important to understand that Base64 is not encryption, because it has no key. I didn't need to know or guess any piece of secret information to reverse the Base64 encoding of your string. (The term 'encoding' can be confusing — it is more general. UTF-8, Base64, and DEFLATE are all encodings, and so is AES-CBC, but of all of them, only AES-CBC is encryption.)
Which standards are SS7 MAP Tags defined in?
Can anyone give me information on which standard contains MAP Tags - sm-RP-UI? 04 1a - sm-RP-UI 24 - TP-RP/UDHI/SRI/MMS/MTI 0b - length 91 26 18 18 55 32 f7 - TP-Originating-Address 00 - TP-PID 00 - TP-DCS 90 40 02 91 61 42 82 - TP-Service-Centre-Time-Stamp 07 - TP-User-Data-Length: (7) depends on Data-Coding-Scheme ca f0 3a 2c a7 87 01 - TP-User-Data The details are needed for coding and I'd like to know which standard they are in. I have been looking in GSM 29.002, GSM 23.040, and GSM 24.011 and I haven't found them. Any help would be greatly appreciated, Thank you.
The SMTL PDUs are defined in 3GPP TS 23.040 - Technical realization of the Short Message Service (SMS) More specifically: 04 1a This is ASN.1 tag a length (OCTET STRING). Since you say this is sm-RP-UI it would be the SignalInfo ASN.1 type defined in 3GPP TS 29.002 used with labels sm-RP-UI on different MAP operations. 24 First thing to look here are the last two bits (TP-Message-Type-Indicator: 9.2.3.1 of 23.040) Since you have H'24 -> B'00100100. This is an SMS-DELIVER (SC to MS) SMS-DELIVER (9.2.2.1) contains TP-Message-Type-Indicator (TP-MTI on 9.2.3.1) (bit 0-1 --> 00) TP-More-Messages-To-Send (TP-MMS on 9.2.3.2) (bit 2 --> 1: "No more messages are waiting for the MS in this SC TP-Status-Report-Indication (TP-SRI on 9.2.3.4) (bit 5 --> 1: "A status report shall be returned to the SME") TP-User-Data_Header-Indicator (TP-UDHI on 9.2.3.23) (bit 6 -> 0: "The TP-UD field contains only the short message") TP-Reply-Path (TP-RP on 9.2.3.17) (bit 7 -> 0: "Not set") 0b 91 26 18 18 55 32 f7 TP-Originating-Address (TP-OA 9.2.3.7 - Address fields in 9.1.2.5) that works: ** Address-Length: H'B = D'11 (not this is in semi-octets) ** Type-of-Address: H'91=B'10010001 with Type-of-Nuymber (B'001: International number) and Numbering-Plan (B'0001: ISDN/E.164) ** Address-Value: BCD: 62818155327 (F is filler) 00 TP-Protocol-Identifier (TP-PID 9.2.3.9) 00 TP-Data-Coding-Scheme (TP-DCS 9.2.3.10) 90 40 02 91 61 42 82 TP-Service-Centre-Time-Stamp (TP-SCTS 9.2.3.11) 2009/04/20 19:16:24 07 TP-User-Data-Length (TP-UDL 9.2.3.16) ca f0 3a 2c a7 87 01 * TP-User-Data (TP-UD 9.2.3.24)