Mediainfo does not handle AWS S3 url with special characters in S3 keys - mediainfo

I am using latest mediainfo. mediainfo --version shows
MediaInfoLib - v17.12
When using special character in filename, it shows valid output.
>mediainfo 'mini & bar.mp4'
Complete name : mini & bar.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42 (mp41/mp42/isom)
File size : 380 KiB
Duration : 4 s 4 ms
Overall bit rate mode : Variable
Overall bit rate : 777 kb/s
Movie name : mini
Encoded date : UTC 2015-02-21 06:02:24
Tagged date : UTC 2015-02-21 06:02:26
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High#L3.1
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 4 s 4 ms
Bit rate mode : Variable
Bit rate : 768 kb/s
Width : 854 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Original frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.063
Stream size : 378 KiB (99%)
Title : Core Media Video
Encoded date : UTC 2015-02-21 06:02:24
Tagged date : UTC 2015-02-21 06:02:26
Color range : Limited
Color primaries : BT.601 NTSC
Transfer characteristics : BT.709
Matrix coefficients : BT.601
Although if I use filename(valid s3 key) with special characters, from AWS S3, it is not accepting it as valid input.
I used below url where username, secret and bucketname are specific to my aws account. and 'testing/mini & bar.mp4' is the key.
https://username:secret#s3-ap-southeast-2.amazonaws.com/bucketname/testing/mini & bar.mp4
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>testing/mini</Key><RequestId>157877D3B661D9A6</RequestId></Error>
If use it with single or double quotes, it shows signature mismatch error.
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
When looked into MediaArea/MediaInfoLib code, it seems only space is handled and other special characters are not handled for aws key.
From Reader_libcurl.cpp
// URL encoding
MediaInfo_Config::urlencode ShouldUrlEncode=Config.URLEncode_Get();
if (ShouldUrlEncode==MediaInfo_Config::URLEncode_Guess)
{
if (File_URL.Path.find(' '))
ShouldUrlEncode=MediaInfo_Config::URLEncode_Yes;
}
if (ShouldUrlEncode==MediaInfo_Config::URLEncode_Yes)
File_URL.Path=ZenLib::Format::Http::URL_Encoded_Encode(File_URL.Path);
When searched further for such issues, below issue talks about space in aws key, which seems to be resolved now.
https://sourceforge.net/p/mediainfo/bugs/960/
So my questions are :
why mediainfo is not handling special characters in AWS keys?
Based on code, --urlencode options sets url encoding. How to use this option, is this option force url encoding and aws keys with special characters will be accepted?
Is there any workaround to get result of mediainfo on s3 url having special characters?
how to fix this?

I have found workaround for my problem, which is to use pre-signed url.
Following are the commands I have used:
1. aws s3 presign 's3://bucketname/testing/mini & bar™©.mp4'
output is => "presignd url"
2. mediainfo 'presignd url'
Hope it helps!!

Related

TiFFReadDirectory: Warning, Unknown field with tag 293 (0x125) encountered

I have several tiff image files which I need to convert to pdf files.
Some of the files get converted successfully and I'm able to get the expected output while some of the files are having the above error while I try to convert them and I get the output file as a complete black image.
I'm trying to do the above task using Imagemagick.
I tried to find more about this problem on the internet. What I understood is that the problem is with the T6Options tag of the tiff file, which is (none) in my case. I'm not able to convert this file to any compression, I tried tiffcp, exiftool, tifftopdf but nothing works
Below is the information regarding the file, which I got using exiftool:
ExifTool Version Number : 11.01
File Name : 1000006_1_1007.tif
Directory : .
File Size : 105 kB
File Modification Date/Time : 2019:07:16 14:43:09+00:00
File Access Date/Time : 2019:07:16 14:43:20+00:00
File Inode Change Date/Time : 2019:07:16 14:43:09+00:00
File Permissions : rw-rw-r--
File Type : TIFF
File Type Extension : tif
MIME Type : image/tiff
Exif Byte Order : Little-endian (Intel, II)
Image Width : 1700
Image Height : 2200
Bits Per Sample : 1
Compression : T4/Group 3 Fax
Photometric Interpretation : WhiteIsZero
Orientation : Horizontal (normal)
Samples Per Pixel : 1
Rows Per Strip : 32
X Resolution : 200
Y Resolution : 200
Planar Configuration : Chunky
Resolution Unit : inches
Subfile Type : Single page of multi-page image
Fill Order : Normal
Strip Offsets : (Binary data 449 bytes, use -b option to
extract)
Strip Byte Counts : (Binary data 249 bytes, use -b option to extract)
T4 Options : Fill bits added
T6 Options : (none)
Page Number : 1 0
Image Size : 1700x2200
Megapixels : 3.7
I was able to convert these files using "tifffile" python library

Reading MarkLogic logs from Query Console using XQuery

I want to read MarkLogic logs (for eg : ErrorLog.txt) from query console using Xquery. I had the below code but the problem is output is not properly formatted. Result is like below
xquery version "1.0-ml";
for $hid in xdmp:hosts()
let $h := xdmp:host-name($hid)
return
xdmp:filesystem-file("file://" || $h || "/" ||xdmp:data-directory($hid) ||"/Logs/ErrorLog.txt")
Problem is result is coming as per host basis like first all log of one host is coming and then starting with time 00:00:01 of host 2 and then 00:00:01 of host 3 after running the Xquery.
2019-07-02 00:00:35.668 Info: Merging 2 MB from /cams/q06data02/testQA2/Forests/testQA2-2.2/0002b4cd to /cams/q06data02/testQA2/Forests/testQA2-2.2/0002b4ce, timestamp=15620394303480170
2019-07-02 00:00:36.007 Info: Merged 3 MB at 9 MB/sec to /cams/q06data02/testQA2/Forests/test2-2.2/0002b4ce
2019-07-02 00:00:38.161 Info: Deleted 3 MB at 399 MB/sec /cams/q06data02/test2/Forests/test2-2.2/0002b4cd
Is it possible to get the output with hostname included with log entries and also if we can sort the output by timelines something like
host 1 : 2019-07-02 00:00:01 : Info Merging ....
host 2 : 2019-07-02 00:00:02 : Info Deleted 3 MB at 399 MB/sec ...
Log files are text files. You can parse and sort them like any other text file.
Although they can get very large (GB+), so simple methods may not be performant.
Plus you need to be able to parse the text into fields in order to sort by a field.
Since the first 20 bytes of every line is the time stamp, and that timestamp is in ISO format which sorts lexically same as date, you can split the file by lines and sort using basic colation to get by time sorting of multiple files.
In V9 one can use the pair of xdmp:logfile-scan and xdmp:logmessage-parse to efficiently search over log files (remotely as well as local) and then transform the results into text, XML (attribute or element format) or JSON.
One can also use the REST API for the same.
see: https://docs.marklogic.com/REST/GET/manage/v2/logs
Once logfiles (ideally a selected subset of log messages that is small enough to manage) is converted to a structured format (xml , json or text lines) then sorting, searching, enriching etc is easily performed.
For something much better take a look at Ops Director https://docs.marklogic.com/guide/opsdir/intro

[DICOM][MergeCOM] MC3 E: Total attribute length (4) not a multiple of size

I have an older version of mergecom library( V4.4.0 ). And now I received the latest version(V5.4.0 ). When I tried to integrate the latest MergeCom library I am getting following error on C-ECHO( logged in merge.log ).
DICOM;(20936) 06-21 17:59:01.28 MC3 E: Total attribute length (4) not a multiple of size
DICOM;(20936) 06-21 17:59:01.28 MC3 E: for VR (UN): 8, tag '0x0'
DICOM;(20936) 06-21 17:59:01.28 MC3(ReadMessageToTag) E: Message received encoded improperly Invalid VR length in stream data .
Please find the attached wireshark logs snapshots
Wireshark
1. ASSOCIATION-RQ
2.ASSOCIATION-RSP
3.ECHO-RQ
4.ECHO-RSP
5.ABORT
The error log from MergeCOM-3 is implying a parsing error when reading the C-ECHO-RSP. The log message is implying MergeCOM-3 did not identify the group 0 element's value representation, and instead interpreted it as UN.
From the appearance of the C-ECHO-RSP in the WireShark capture, it appears to be encoded properly and WireShark was able to decode the C-ECHO-RSP.
Were there any other errors in the logs? Is your data dictionary being loaded properly, such that the library would know the VR of the group 0 length tag (0000,0000)?

VB6 Byte array in VB.NET

I have some image comparison DLL, and documentation for VB6, but I want to use it in ASP.NET/VB.NET
In documentation is stated following:
iDLLoadImage
Description : Load the captured image (in raw format), preparing for
face detection.
Declaration : iDLLoadImage (pByte As Byte, iWidth As Long, iHeight As
Long, iColor As Long)
Parameters : pByte : byte array which stored the (raw) captured image
(in) iWidth : for setting raw image width (in) iHeight : for setting
raw image height (in) iColor : 1 for Gray image, 3 for RGB image (in)
Return Values : 0 : Successful
-1 : This library is not enabled.
-2 : pByte is NULL.
-3 : Failed to initialize internal parameters
But, when I try to pass it byte array in VB.NET it says it expects byte.
Is there any solution for this?
(And here's screenshot) http://i.stack.imgur.com/giORJ.png
Thanks.
Try these things:
A Dabblernl said, try using image(0) as a parameter. Since it is byref, it may pass the memory location and from there process the entire array.
You may be able to declare the function as an alias, and change the first parameter to pByte As Byte().

Convert a DCR video file

I have a DCR (file has a .dcr extension) video file coming from a video surveillance device ( I don't know the make and model of the recorder )
I'm unable to read it with VLC, Media Player, and it won't open in Virtual Dub or can't be converted with the standard "ffmpeg.exe video.dcr output.avi" command line.
But I'm able to get a very basic** playback of the video stream with MPC-HC player of the Combined Community Codec Pack. Unfortunately, the audio stream (which I'm looking for) will not play.
According to the MPC-HC player file info, I'm dealing with this:
General
Format : MPEG-4 Visual
File size : 459 MiB
Video
Format : MPEG-4 Visual
Format profile : Advanced Simple#L5
Format settings, BVOP : Yes
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Muxing mode : Packed bitstream
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Writing library : XviD 64
** By very basic, I mean I can play the video, but not seek through it, and there is no keyframe at all in the video output.
Hopefully some of you guys will have dealt with DCR files from video surveillance equipment.
I would recommend downloading the latest version of the digital court player from http://www.bisdigital.com. This is the program that many courts use to view video from court hearings in .dcr format. You should be able to pause, rewind, fast forward, as well as control the speed of the video playback.

Resources