Is there any way to conform that the file is encrypted - encryption

I have a file 'T019.T19975.ITEM.BARCD.DATA' and i'm trying to read this file but i can't see the data.
os -linux,
I tried some command on this like
file T019.T19975.ITEM.BARCD.DATA
T019.T19975.ITEM.BARCD.DATA: SysEx File -
file -bi T019.T19975.ITEM.BARCD.DATA
text/plain; charset=iso-8859-1
head T019.T19975.ITEM.BARCD.DATA
�������#####������������������#������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������######�������������������������������������������##

Related

Decrypt .m3u8 playlist and merge it into single .mp4 file with ffmpeg

I have a folder which contains .key file, .m3u8 file and a bunch of .ts files.
My .m3u8 looks like this:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:5
#EXT-X-KEY:METHOD=AES-128,URI="mykey.key"
#EXTINF:4.004000,
000000.ts
#EXTINF:4.004011,
000001.ts
#EXTINF:4.004000,
000002.ts
#EXTINF:4.004000,
000003.ts
...
#EXT-X-ENDLIST
What I want to do is to decrypt it and merge this playlist into a single .mp4 file
I tried this ffmpeg command on my Windows machine:
ffmpeg -i "myvid.m3u8" -codec copy output.mp4
But I get a following error:
[hls # 000002780f0a8dc0] Skip ('#EXT-X-VERSION:3')
[hls # 000002780f0a8dc0] Filename extension of 'mykey.key' is not a common multimedia extension, blocked for security reasons.
If you wish to override this adjust allowed_extensions, you can set it to 'ALL' to allow all
[hls # 000002780f0a8dc0] Unable to open key file mykey.key
[hls # 000002780f0a8dc0] Opening 'crypto:000000.ts' for reading
[hls # 000002780f0a8dc0] Opening 'crypto:000001.ts' for reading
[hls # 000002780f0a8dc0] Error when loading first segment '000000.ts'
myvid.m3u8: Invalid data found when processing input
I changed my command to following:
ffmpeg -allowed_extensions ALL -i "myvid.m3u8" -codec copy output.mp4
Then I got this error:
[hls # 000001a079cf8f80] Skip ('#EXT-X-VERSION:3')
[hls # 000001a079cf8f80] Opening 'mykey.key' for reading
[hls # 000001a079cf8f80] Opening 'crypto:000000.ts' for reading
[hls # 000001a079cf8f80] Opening 'crypto:000001.ts' for reading
[hls # 000001a079cf8f80] Error when loading first segment '000000.ts'
myvid.m3u8: Invalid data found when processing input
At this point I don't really understand what the problem is.
Any idea how can I fix this?
I'm open to use any other software if this can't be done with ffmpeg or smth
Thanks!
From your manifest,I could see that you have downloaded the necessary streams and key.
The issue here is that ffmpeg could not decrypt the stream files '.ts' using the key you mentioned in the manifest. This issue may be due to your key file. The aes key file is encrypted and should be in binary only. If your 'content-type' response header is 'application/octet-stream' then handle the response as a binary file.

Uploading release asset to Github is corrupting file

I am trying to automate our Github release process, and as part of it I am uploading release artifacts to Github release. Artifact I am uploading is a tar.gz file. I am using following python3.7 code to upload the artifact:
with open(filepath, 'rb') as file:
response = self._request_session.post(endpoint,
params={'name': local_artifact.filename},
files={local_artifact.filename: file})
Response status code I am getting is 201, which is expected as per Github upload release asset api documentation.
However, when I am downloading artifacts from Github, I am neither able to validate sha256 of the file, nor I am able open the tarball. On running tar -zxvf test.tar.gz I am getting following error:
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
I have already tested the file on local which I am uploading and that is working fine.
Can anyone please help me in identifying the problem here?
Here is response text I get after uploading file:
{
"url":"https://api.github.com/repos/c2tarun/upload_test_repo/releases/assets/17244134",
"id":17244134,
"node_id":"MDEyOlJlbGVhc2VBc3NldDE3MjQ0MTM0",
"name":"test.tar.gz",
"label":"",
"uploader":{
"login":"c2tarun",
"id":1129670,
...truncating for readability.
"type":"User",
"site_admin":false
},
"content_type":"multipart/form-data; boundary=1a13638ee5f5f57d303508eea4e64211",
"state":"uploaded",
"size":11969682,
"download_count":0,
"created_at":"2020-01-09T23:49:54Z",
"updated_at":"2020-01-09T23:50:18Z",
"browser_download_url":"https://github.com/c2tarun/upload_test_repo/releases/download/untagged-1baa5c7dd7f5a8d506cc/test.tar.gz"
}
Interesting update: Uploading same file multiple times, gives me different sha on downloading.
More Update: I thought that may be I am providing wrong Content-Type header for tar.gz file, so I created a zip file and used Content-Type as application/zip. I am still seeing same issue. Here are my request headers for zip file:
{
'User-Agent':'python-requests/2.22.0',
'Accept-Encoding':'gzip, deflate',
'Accept':'*/*',
'Connection':'keep-alive',
'Content-Type':'application/zip',
'Content-Length':'332',
'Authorization':'Basic xxxxxxxxx'
}
Thanks
You should set the Content-Type is 'application/tar+gzip' or debug with
file --mime-type -b test.tar.gz
command.

Why Do I Get an nginx Internal Server Error 500 on a PNG Image File Upload

For some reason I'm getting a Internal Server Error 500 on nginx when I upload a png file to my site. The site works with a jpg or jpeg file. I am getting nothing in the error log at all. My mime.types holds the following:
image/jpeg jpeg jpg;
image/png png;
Since I'm getting no feedback at all, I have nothing to place here as more information on the error. The only nginx.conf directives with regards to uploads is this:
client_max_body_size 12M;
I am guessing it is nginx that's having this issue because the application is handling the other file types (jpg, jpeg) and error reporting shows I'm not even getting to my route in Sinatra. I've looked for some kind of file type directives in the nginx directives list, and found nothing.
How can I get this file type to be received properly? Why is nginx not happy with this file submission?
Bad .png file for some reason. Fixed with a proper file.
In my case, I found out, after reading /var/log/nginx/error.log that a certain client_temp direvtory was being tried by nginx.
"http://somesite/"
2021/03/30 22:11:12 [crit] 7249#7249: *78 open() "/var/cache/nginx/client_temp/0000000007" failed (20: Not a directory),
There was a file client_temp but it should be a directory. After making it a directory and setting the owner to www-data, it worked.

UnicodeDecodeError: 'utf8' codec can't decode byte 0xa5 in position 14: invalid start byte

I am new to python. I am using Selenium library to automate one use case.I have to upload a file from my local. I want this path of the file to be absolute.
Below is the mentioned piece of code I have written:
${curdir}= OperatingSystem.Get File${CURDIR}\\..\\ImportFiles\\PUIMeasure_Report.xlsx
Choose File //*`[#id='importMeasureForm:measureUploadFileComponent:file'] ${curdir}.
The file is correctly located. When i run it, it is showing the following error in log file.
${curdir} = OperatingSystem . Get File
C:\Users\aha8kor\eclipse-workspace\ActionPlanSmokeTest\Tests\Resources\PO\..\ImportFiles\PUIMeasure_Report.xlsx
Start / End / Elapsed: 20180420 11:16:10.518 / 20180420 11:16:10.523 /
00:00:00.005
11:16:10.519 INFO Getting file
'C:\Users\aha8kor\eclipse-workspace\ActionPlanSmokeTest\Tests\Resources\ImportFiles\PUIMeasure_Report.xlsx'.
11:16:10.523 FAIL UnicodeDecodeError: 'utf8' codec can't decode byte
0xa5 in position 14: invalid start byte
enter image description here
Found the solution after too many attempts.
Instead of using "get file" which resulted in encoding and decoding the issue, I gave the path directly in "Choose File" keyword.
Choose File //*[#id='importMeasureForm:measureUploadFileComponent:file'] ${CURDIR}\ImportFiles\PUIMeasureReport.xlsx
I had messed up the path previously.This is just a workaround to avoid such kind of an issue.
I still want to know as how can i handle such encoding decoding issues with "Get File" method.
Thanks in advance.

How to extract data from file on an FTP server without downloading it all in R? - encoding error?

I am trying to get a large dataset (3+ GB) from the following server:
ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR
I know RCurl is a good package for getting data from FTP. The file is a compressed netcdf file. I need to uncompress it to read it into R using ncdf4. It's compressed as bz2.
Importantly, the file is larger than I want on my hard drive, so saving a copy locally is not an ideal option. How can I access data on the file without saving a copy to my disk first?
Here's my attempt so far:
library(RCurl); library(ncdf4)
d = getURL('ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR/2015/144/20150524-JPL-L4UHfnd-GLOB-v01-fv04-MUR.nc.bz2')
d = bzfile(d, open = 'r')
d = nc_open(d)
But I'm stuck at this cryptic error after the first line:
Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) :
embedded nul in string: 'BZh91AY&SY¦ÁÀÉ\0033[ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿáåÏ\035\017)³îÎ\u009dÍØcn]sw7½ÎkÜÞõï=uÎׯv]ìçn\u009dÎn½îê·±Þìê÷wS­M\u008có·+ÎçW¹Ý=Ù×¹\u009cγ­ÜëÞs½ÛN¹²w;\u009buÍÝ]{·k^çuªnìº-³6«[+Üå;\033m»Û½ow:w¹ïo{uyîî\u00937¬\\Ƶl¶½\u009dÖVìç¯{ÎõïoSm]Ý×\u009eî\u008dæî®î®î\vÛÕïgW\036î®wqîÝ\\ïw«6½Þï\036Ýrë§=¬Fg·\\íåÔÙº÷gu·3\u009bKmÛ\027­Þ»\u0092îî\016îêwwm»\u009b­·s;MÞÁ½½­ÎóÍso^»q¯o;k\033iµ\u009bÛuyÝÞní5w:ï]ÓuÎo[«\033:åÞvEÜíÎç½ÝË­\u009eìQNöÔ\u008e\u0094vmÝȯg»e lÍ^\u008a©'
It seems to be an encoding issue based on other similar problems but I tried both .encoding = 'UTF-8' and .encoding = 'ISO-8859-1' as shown in the getURL() documentation but neither work.
I've seen other answers to problems like this but they all seem to involve editing the source file. I don't have write access to this file, however. Any help?
I'd use httr for this
library("httr")
library("ncdf4")
url <- 'ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR/2015/144/20150524-JPL-L4UHfnd-GLOB-v01-fv04-MUR.nc.bz2'
res <- GET(url, write_disk(basename(url)))
# uncompress - I used OSX's default compression tool
nc_open(sub("\\.bz2", "", res$request$output$path))
the only step i didn't sort out programatically is un-compressing the bz2 file, just did that with OSX's default tool
I don't know much at all about R, but you should be able to do this with curl in FTP mode by changing the output to stdout rather than a local filename and then using bz2 to uncompress the file you want from its standard input.
So, for example, I can do this:
curl --output - --user user:password 'ftp://127.0.0.1/somefile.bz2' | bz2 ...
Maybe you can start that from within R? Or make a fifo with:
mkfifo fifo
curl ....
and then read from the file called fifo in R.
Or maybe R has a system() command, and you could do:
system('mkfifo fifo; curl ..... | bz2 .... > fifo &')
and then read from the file called fifo in R.

Resources