I am trying to return hexadecimal string as response from my AWS Lambda function. When it reaches to the client the data seems to be modified.
Data :
47 49 46 38 39 61 01 00 01 00 80 00 00 00 00 00
ff ff ff 21 f9 04 01 00 00 01 00 2c 00 00 00 00
01 00 01 00 00 08 04 00 03 04 04 00 3b
Hexadecimal Excaped Data ( Sent Data ):
\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00"
"\xff\xff\xff\x21\xf9\x04\x01\x00\x00\x01\x00\x2c\x00\x00\x00\x00"
"\x01\x00\x01\x00\x00\x08\x04\x00\x03\x04\x04\x00\x3b
Received Data
47 49 46 38 39 61 01 00 01 00 c2 80 00 00 00 00
00 c3 bf c3 bf c3 bf 21 c3 b9 04 01 00 00 01 00
2c 00 00 00 00 01 00 01 00 00 08 04 00 03 04 04
00 3b
How to fix this?
Last time I checked it was not very explicit in the doc, but API Gateway is really made for json (or similar) and support for binary is 'on the roadmap' but clearly doesn't seem to be a priority. It converts everything it sends to utf-8.
Comparing precisely your original data with the received one you can see it :
47 49 46 38 39 61 01 00 01 00 80 00 00 00 00 00 ff ff ff 21 f9 04 01 00 00 01 00 2c 00 00 00 00 01 00 01 00 00 08 04 00 03 04 04 00 3b
47 49 46 38 39 61 01 00 01 00 c2 80 00 00 00 00 00 c3 bf c3 bf c3 bf 21 c3 b9 04 01 00 00 01 00 2c 00 00 00 00 01 00 01 00 00 08 04 00 03 04 04 00 3b
Everything under 0x7f is OK because the unicode code point is the same as the encoded byte (U+0047 -> 47), but for 0x80 or more the problem arises : U+0080 -> c2 80, U+00FF -> c3 bf and so on.
We had a similar problem recently : binary data was corrupted and bigger when sent through Gateway than with direct access to our backend. It was because a lot of bytes get replaced by Unicode special 'replacement character' aka 'U+FFFD' aka '0xEF 0xBF 0xBD'.
How to fix ? We just stopped using Gateway but if you can afford your data to be bigger, you can base64 encode it.
Related
I'm trying to decode a hex string to a datetime format from a BLE custom UUID. I don't have any information except the decoded value and datetime, that's why I put similar examples in milliseconds and minutes. It will be complicated to see the value of another year.
If you look closely, you will notice there are only 2 values repeated twice in each row (bold and italic). First starts with 0x01 (maybe a flag) and then 2 bytes and second 2 bytes too.
Similar minutes:
01 8E 6C 00 00 8E 6C 00 00 62 25 62 25 00 00 00 00 00 00 00 - 1669024172,5141 (Nov 21 2022 09:49:32,5387632)
01 9C 6C 00 00 9C 6C 00 00 62 49 62 49 00 00 00 00 00 00 00 - 1669024172,68598 (Nov 21 2022 09:49:32,7638451)
01 fd 6c 00 00 fd 6c 00 00 64 0f 64 0f 00 00 00 00 00 00 00 - 1669024174,23293 (Nov 21 2022 09:49:34,3170414)
01 0d 6d 00 00 0d 6d 00 00 6e 2f 6e 2f 00 00 00 00 00 00 00 - 1669024174,5767 (Nov 21 2022 09:49:34,5414078)
01 f0 6d 00 00 f0 6d 00 00 62 21 62 21 00 00 00 00 00 00 00 - 1669024178,01436 (Nov 21 2022 09:49:38,0746688)
01 00 6e 00 00 00 6e 00 00 67 81 67 81 00 00 00 00 00 00 00 - 1669024178,35812 (Nov 21 2022 09:49:38,2775026)
Similar milliseconds:
01 16 1d 00 00 16 1d 00 00 4d f4 4d f4 00 00 00 00 00 00 00 - 1669023854,70251 (Nov 21 2022 09:44:14,7728813)
01 1e 6e 00 00 1e 6e 00 00 67 c5 67 c5 00 00 00 00 00 00 00 - 1669024178,70189 (Nov 21 2022 09:49:38,7714679)
01 e2 6d 00 00 e2 6d 00 00 62 4d 62 4d 00 00 00 00 00 00 00 - 1669024177,84247 (2022-11-21 10:49:37,8263242)
01 5f dd 06 00 5f dd 06 00 cb 46 cb 46 00 00 00 00 00 00 00 - 1669110577,66758 (2022-11-22 10:49:37,8231004)
01 00 6e 00 00 00 6e 00 00 67 81 67 81 00 00 00 00 00 00 00 - 1669024178,35812 (2022-11-21 10:49:38,2775026)
01 7c dd 06 00 7c dd 06 00 c8 0c c8 0c 00 00 00 00 00 00 00 - 1669024178 (2022-11-22 10:49:38,2722736)
It's looks like:
It's not decoded in the unix epoch (JAN 1 1970)
The first value is increasing (0xXX6C, 0xXX6E, 0xXX6E), so maybe it's the int part.
I tried to decode like year, minutes, seconds but it seems it isn't.
Sometimes first value is bigger than 2 bytes (3 bytes):
01 DE 61 04 00 DE 61 04 00 4A C1 4A C1 00 00 00 00 00 00 00 - 1669104436 (Nov 22 2022 09:07:16,1902042)
I tried this solutions:
https://stackoverflow.com/questions/1389046/what-is-the-specification-of-hexadecimal-date-format-in-sql-server Error: System.ArgumentOutOfRangeException (trying one of my hex)
http://erbhavi.blogspot.com/search/label/Converting%20DateTime%20to%20Hex%20in%20C%23 DateTime yourDateTime = new DateTime( ticks1970 + gmt * 10000000L ); //Error CS0019: Operator '*' can't apply to types 'string' y 'long'
I'll update the question if I find more info.
I am struggling to get Windows to load the default WinUSB driver for my device. Please note that I am looking for a solution that is using BOS descriptor (and not the old 0xEE string index).
The device enumerates and Windows tells me that it is installing the device, but the WinUSB driver is not loaded. I have tried everything that I can think of, but still I can't get Windows to load the driver. I even uninstall the device and delete the USB flags in the registry whenever I re-try, but to no avail. Is there anyone who can help me to get this to work?
I don't want WebUSB capabilities or anything additional. This is a non-composite device.
This is my BOS descriptor (as sent over USB):
05 0F 21 00 01 1C 10 05 00 DF 60 DD D8 89
45 C7 4C 9C D2 65 9D 9E 64 8A 9F 00 00 03
06 B2 00 01 00
And this my BOS descriptor set:
0A 00 00 00 00 00 03 06 B2 00 08 00 01 00 ..............
00 00 A8 00 08 00 02 00 00 00 A0 00 14 00 ..............
03 00 57 49 4E 55 53 42 00 00 00 00 00 00 ..WINUSB......
00 00 00 00 84 00 04 00 07 00 2A 00 44 00 ..........*.D.
65 00 76 00 69 00 63 00 65 00 49 00 6E 00 e.v.i.c.e.I.n.
74 00 65 00 72 00 66 00 61 00 63 00 65 00 t.e.r.f.a.c.e.
47 00 55 00 49 00 44 00 73 00 00 00 50 00 G.U.I.D.s...P.
7B 00 46 00 37 00 32 00 46 00 45 00 30 00 {.F.7.2.F.E.0.
44 00 34 00 2D 00 43 00 42 00 43 00 42 00 D.4.-.C.B.C.B.
2D 00 34 00 30 00 37 00 44 00 2D 00 38 00 -.4.0.7.D.-.8.
38 00 31 00 34 00 2D 00 39 00 45 00 44 00 8.1.4.-.9.E.D.
36 00 37 00 33 00 44 00 30 00 44 00 44 00 6.7.3.D.0.D.D.
36 00 42 00 7D 00 00 00 00 00 6.B.}.....
The layout is:
typedef struct _SMSOS20DescriptorSet
{
SDeviceDescSetHeader sDescriptorSetHeader;
SConfigurationSubsetHeader sConfSubsetHeader;
SFunctionSubsetHeader sFuncSubsetHeader;
SDeviceCompatibleIdDescriptor sCompIdDescriptor;
SDeviceRegDescDeviceInterfaceGUID sRegistryDescDevInterfaceGuid;
} SMSOS20DescriptorSet;
I have follewed these guides and doc:
https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/automatic-installation-of-winusb#winusb-device-installation-by-using-the-in-box-winusbinf
MS_OS_2_0_desc.docx
https://thewindowsupdate.com/2018/10/12/how-to-install-winusb-sys-without-a-custom-inf/
https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/winusb-installation#automatic-installation-of--winusb-without-an-inf-file
UPDATE:
when you have a non-composite device that only has a single a configuration, then you are not use any subset headers (neither 'Configuration subset header' nor 'Function subset header'). So, the correct layout in this case is:
typedef struct _SMSOS20DescriptorSet
{
SDeviceDescSetHeader sDescriptorSetHeader;
SDeviceCompatibleIdDescriptor sCompIdDescriptor;
SDeviceRegDescDeviceInterfaceGUID sRegistryDescDevInterfaceGuid;
} SMSOS20DescriptorSet;
UPDATE: when you have a non-composite device that only has a single a configuration, then you are not use any subset headers (neither 'Configuration subset header' nor 'Function subset header'). So, the correct layout in this case is:
typedef struct _SMSOS20DescriptorSet
{
SDeviceDescSetHeader sDescriptorSetHeader;
SDeviceCompatibleIdDescriptor sCompIdDescriptor;
SDeviceRegDescDeviceInterfaceGUID sRegistryDescDevInterfaceGuid;
} SMSOS20DescriptorSet;
I want to parse a IPV6 packet that consists of extension headers. The "Hdr Ext Len" (second) field of extension headers such as Hop by Hop option, Routing, destination options and Authentication header, is defined as
8-bit unsigned integer. Length of the header in 8-octet units, not including the first 8 octets.
Consider the following IPV6 packet with authentication header (AH). The second byte of AH has a value 04 which signifies 24 bytes (got this information from hex packet decoder in https://hpd.gasmi.net/).
I would like to know how to decode the value of this field "Hdr Ext Len" to get the actual extension header size.
IPV6 Header
6E 00 00 00 00 34 33 01 FE 80 00 00 00 00 00 00 00 00 00 00 00 00 00 02 FE 80 00 00 00 00 00 00 00 00 00 00 00 00 00 01
Authentication extension header
59 04 00 00 00 00 01 00 00 00 00 15 8F 74 57 E3 C2 D4 13 EF 5E F1 FF 13
OSPF packet
03 02 00 1C 02 02 02 02 00 00 00 01 E9 DF 00 00 00 00 00 13 05 DC 00 01 00 00 0B 93
Here follows the hex dump of the MPEG2VideoDescriptor:
06 0e 2b 34 02 53 01 01 0d 01 01 01 01 01 51 00
83 00 00 f3 3c 0a 00 10 a3 be 51 b2 00 05 e7 11
bf 82 21 97 f7 a0 14 ed 30 06 00 04 00 00 00 02
30 01 00 08 00 00 ea 60 00 00 03 e9 80 00 00 04
01 c9 c3 80 30 04 00 10 06 0e 2b 34 04 01 01 02
0d 01 03 01 02 04 61 01 32 15 00 01 05 32 0e 00
08 00 00 00 10 00 00 00 09 32 0d 00 10 00 00 00
02 00 00 00 04 00 00 00 1a 00 00 00 00 32 0c 00
01 00 32 08 00 04 00 00 02 d0 32 09 00 04 00 00
05 00 32 02 00 04 00 00 02 d0 32 03 00 04 00 00
05 00 32 01 00 10 06 0e 2b 34 04 01 01 03 04 01
02 02 01 04 03 00 33 02 00 04 00 00 00 02 33 08
00 04 00 00 00 01 33 03 00 01 04 33 01 00 04 00
00 00 08 33 0b 00 01 00 33 07 00 02 00 00 33 04
The first 16 bytes:
06 0e 2b 34 02 53 01 01 0d 01 01 01 01 01 51 00 (UID)
Next 4 bytes is the BER size:
83 00 00 f3 (0xf3 bytes long)
Next 4 bytes:
3c 0a 00 10 (0x3c0a means Instance UUID and 0x0010 is the size)
Then follows the UUID:
a3 be 51 b2 00 05 e7 11 bf 82 21 97 f7 a0 14 ed
Next 4 bytes:
30 06 00 04 (0x3006 means Linked Track ID and 0x0004 is the size)
Next 4 bytes is the Linked Track ID: 00 00 00 02
Next 4 bytes: 30 01 00 08 (0x3001 means Sample Rate and 0x0008 is the size)
The following 8 bytes are actually frame rate numerator and denominator:
0000ea60 == 60000 and 000003e9 == 1001.
Now we have the bold part: 80 00 00 04
.
Can somebody please explain what does it mean?
The next four bytes are 01 c9 c3 80 and it is definitely the bitrate (30000000), but how can I know that for sure?
Edit:
Does 80 00 00 04 mean the following:
0x8000 is a dynamic tag. According to SMPTE 337, tags 0x8000-0xFFFF are dynamically allocated. The 0x0004 is the size (4 bytes). If that's true, how can I tell that the following 4 bytes 01 c9 c3 80 are actually the bitrate? It could be anything, or?
First you have to understand how local tags work.
Local tags 0x8000 and above are user defined.
You have to look at the primer pack of the header partition.
The primer pack translates the local tag to a global UL which may or may not be vendor specific.
Consider the primer pack being a translation table between the 2 byte local tag and the 16 byte UL.
There is one audio input device on my PC.
QAudioDeviceInfo::availableDevices returns empty list.
calling waveInGetDevCaps(0) returns next:
FF FF - Mid
FF FF - Pid
10 00 00 00 version
1C 04 38 04 3A 04 40 04 -name
3E 04 44 04 3E 04 3D 04
20 00 28 00 57 00 65 00
62 00 63 00 61 00 6D 00
20 00 43 00 32 00 31 00
30 00 29 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
FF FF 0F 00 - formats
02 00 - channels
00 00 - reserved
I propose the reason is Cyrillic device name.
Any ideas?
Thanks for all!
UPD:
QList<QAudioDeviceInfo> devices = QAudioDeviceInfo::availableDevices(QAudio::AudioInput);
foreach(QAudioDeviceInfo deviceInfo, devices)
{
printList(deviceInfo.supportedByteOrders(), "Orders: ");
printList(deviceInfo.supportedChannelCounts(),"Channs: ");
printList(deviceInfo.supportedCodecs(), "Codecs: ");
printList(deviceInfo.supportedSampleRates(), "Rates : ");
printList(deviceInfo.supportedSampleSizes(), "Sizes : ");
printList(deviceInfo.supportedSampleTypes(), "Types : ");
printFormat(deviceInfo.preferredFormat(), "preferred");
}