How can I convert a x12 EDI file from ANSI to UTF-8 in a receive pipeline? So Before it's been dissasembled.
Related
I have an issue using an R script as a data source in Microsoft PowerBi. I think this is fundermentally an issue with PowerBi, but in the short term I'll need to find a solution in R.
Essentially, PowerBi doesn't appear to be able to handle the messages that would be sent to the console if I was using R Studio.
Within the R script I'm using a REST API to request data from a URL. The JSON message that is received is converted into an R data frame. When using the script as a datasource in PowerBi, this only works if I set the verbose settings to FALSE i.e. if I was using R Studio no messages (in particular data in) are sent to the console.
response <- GET(<url>,
body = list(),
add_headers(.headers = c('<identity token>' = ID_to_use)),
verbose(data_out = FALSE,
data_in = FALSE,
info = FALSE,
ssl = FALSE),
encode = "json")
However, I do not have the option to switch off the incoming/outgoing JSON header messages (which is going to come back to bite!).
<< {"identity":" <token>"}
* Connection #54 to <host> left intact
No encoding supplied: defaulting to UTF-8.
-> GET <URL request> HTTP/1.1
-> Host: <host>
-> User-Agent: libcurl/7.64.1 r-curl/4.3 httr/1.4.1
-> Accept-Encoding: deflate, gzip
-> Accept: application/json, text/xml, application/xml, */*
-> <Identity>: <Identity>
->
<- HTTP/1.1 200 OK
<- X-Session-Expiry: 3599
<- Content-Type: application/json
<- Transfer-Encoding: chunked
<- Date: Thu, 06 Aug 2020 16:14:26 GMT
<- Server: <Server>
<-
No encoding supplied: defaulting to UTF-8.
No encoding supplied: defaulting to UTF-8.
No encoding supplied: defaulting to UTF-8.
From R help
.
.
verbose() uses the following prefixes to distinguish between different components of the http messages:
* informative curl messages
-> headers sent (out)
>> data sent (out)
*> ssl data sent (out)
<- headers received (in)
<< data received (in)
<* ssl data received (in)
.
.
Switching the verbose settings to FALSE works for a single request, however, I need to put the request into a loop and keep requesting more data until the API gateway indicates there is no more data to be received. PowerBi appears to fail when in the script five or more request/replies are sent/received.
Just from observation, I assume this is to do with the JSON Header messages piling up.
I've tried a number of approaches but nothing seems to work: sink('NUL'), invisible(), capture.output().
Any help would be appreciated.
I found a hacky solution, which at least solved the problem I had in R, but not in PowerBi.
By writing a "wrapper" R script (see below) which calls my main script THE_SCRIPT.R using a shell command. THE_SCRIPT dumps out a CSV file, which I then read in the wrapper script:
#Required by PowerBi
library(mice)
#set the directory, between R and the shell it's a pain to deal with spaces in the directories and quotes
setwd("C:/Program Files/R/R-3.6.2/bin/")
system("Rscript.exe C:\\Users\\<USER>\\Documents\\THE_SCRIPT.R > Nul 2>&1")
A_DATA_TABLE <- read.csv("C:\\Users\\<USER>\\Documents\\THE_FILE.csv")
However, this still didn't resolve the issue when running it in PowerBi.
Note, I tried sink('Nul 2>&1') in R, didn't work.
I am looking for an export function in Wireshark (or tshark, whatever) to save my decrypted ESP paquets (decrypted with SPI, AES128-CBC, HMAC-SHA1 keys).
Displayed paquets are decrypted but if I save them to a pcap file (With File > Save specific paquets), they are save as encrypted ...
Same with tshark -r my.pcap --w out.pcap ...
Any idea ?
I have no idea whether Wireshark supports such feature.
If the capture file was decrypted on PC1 and you want to see it on PC2, append your PC1's esp_sa file to PC2's esp_sa and add a newline to the end of PC2's esp_sa.
esp_sa is a text file and you can find it under
C:\Users\YourUserName\AppData\Roaming\Wireshark\
I'm trying to automate centos installs via PXE and kickstart with encrypted filesystems. In case we mislay the passphrase we want to use escrow files and encrypt them using the public key attached to an x509 certificate obtained from a web server. The relevant line in the kickstart file is
logvol /home --fstype ext4 --name=lv02 --vgname=vg01 --size=1 --grow --encrypted --escrowcert=http://10.0.2.2:8080/escrow.crt --passphrase=XXXX --backuppassphrase
Leaving the cert as PEM encoded on the web server rather than DER doesn't seem to matter, either work up to a point.
The filesystem is created and encrypted using the supplied passphrase and can be opened on reboot with no issues. Two escrow files are produced as expected and if by using the NSS database containing the private key and the first escrow file I obtain what I think is the passphrase but it doesn't unlock the disk. For example:
# volume_key --secrets -d /tmp/nss e04a93fc-555b-430b-a962-1cdf921e320f-escrow
Data encryption key:<span class="whitespace other" title="Tab">»</span>817E65AC37C1EC802E3663322BFE818D47BDD477678482E78986C25731B343C221CC1D2505EA8D76FBB50C5C5E98B28CAD440349DC0842407B46B8F116E50B34
I assume the string from 817 to B34 is the passphrase but using it in a cryptsetup command does not work.
[root#mypxetest ~]# cryptsetup -v status home
/dev/mapper/home is inactive.
Command failed with code 19.
[root#mypxetest ~]# cryptsetup luksOpen /dev/rootvg01/lv02 home
Enter passphrase for /dev/rootvg01/lv02:
No key available with this passphrase.
Enter passphrase for /dev/rootvg01/lv02:
When prompted I paste in the long numeric string but get the No key available message. However if I use the passphrase specified in the kickstart file or the backup escrow file the disk unlocks.
# volume_key --secrets -d /tmp/nss e04a93fc-555b-430b-a962-1cdf921e320f-escrow-backup-passphrase
Passphrase:<span class="whitespace other" title="Tab">»</span>QII.q-ImgpN-0oy0Y-RC5qa
Then using the string QII.q-ImgpN-0oy0Y-RC5qa in the crypsetup command works.
Has anyone any idea what I'm missing? Why don't both escrow files work?
I've done some more reading and the file ending in escrow is not an alternative passphrase for the luks volume but it contains the encryption key which is encrypted of course. When decrypted the long string is the encryption key and there's a clue in the rest of the text which I confess I didn't read very well.
how to deal with server name server\se
have message
Error: '\S' is an unrecognized escape in character string starting "'driver={SQL Server}; server=server\s"
what i need to do?
I use the following command to send SMS messages through SMS Adapters:
telnet localhost 0000 <<-EOF
helo x
MAIL FROM: Test
RCPT TO: 447999999999
DATA
Test £1234
.
logout
quit
EOF
However when ever I get the Message through it will be in the format:
Test ?£1234
Appending the ? to the front of the £ symbol, I have tried investigating a few methods including MIME however not quite sure how they can be implemented.
Any ideas on how I can stop and allow the successful passthroughs of £
Have you tried encoding the message first? You can do this using base64 in UTF-8 charset:-
e.g.
Convert:
msg="£1234"
To:
msg="wqMxMjM0"
NOTE:Try testing encoding/decoding using the online converter - https://www.base64encode.org/
Once you have encoded your text you can send the message via telnet by adding the MIME details after the DATA command in telnet by specifying MIME types, example script below:-
telnet localhost 0000 <<-EOF
helo x
MAIL FROM: Test
RCPT TO: 447999999999
DATA
Subject: Test
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
${msg}
.
logout
quit
EOF
Hope that helps.
In some installations, telnet is not 8-bit clean by default (It does not recognize 8-bit character encodings such as Unicode). In order to be able to send Unicode keystrokes to the remote host, you need to set telnet into "outbinary" mode. There are two ways to do this:
$ telnet -L <host>
and
$ telnet
telnet> set outbinary
telnet> open <host>
Source