I have been trying to write a Powershell script based on some code online that will read the metadata info from picture, video and other files and then sort them based on one of the dates (date taken currently seems to be the best bet if it's available, but date modified works on files that have not yet been altered).
However, when I run the script and pull the info, I can't convert the string to a date. Here's roughly how I get the info through a COM object:
PS C:/> $objShell = New-Object -ComObject Shell.Application
PS C:/> $objFolder = $objShell.namespace("C:\MyFolder")
PS C:/> $date = $objFolder.GetDetailsOf($objFolder.Items().Item(0), 12)
PS C:/> $date
7/10/2014 7:09 PM
The problem is I should be able to convert this to a datetime object. For instance, if I manually write it in it works:
PS C:/> [datetime]::ParseExact("7/10/2014 7:09 PM","g",$null)
Thursday, July 10, 2014 7:09:00 PM
But if I substitute the variable it doesn't work:
PS C:/> [datetime]::ParseExact($date,"g",$null)
Exception calling "ParseExact" with "3" argument(s): "String was not recognized as a valid DateTime."
At line:1 char:1
+ [datetime]::ParseExact($date,"g",$null)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : FormatException
This is most likely due to the fact that the variable isn't actually what I'm seeing. It's in fact longer. Not to mention the fact that if you iterate through all the characters, you can see where the extra length is coming from:
PS C:\> $date.Length #should be about 16, you'd think
22
PS C:\> $datearray = #()
PS C:\> for ($i = 0; $i -lt $date.Length; $i++) {$datearray += $date[$i]}
PS C:\> $datearray #i'm printing on one line and in quotes for ease of viewing
" 7/ 10/ 2014 7:09 PM"
If you try printing the array with a join or something similar, the results are (to me, without knowing what's going on) unpredictable. It treats it like it has 22 characters, but prints ignoring the spaces.
I'm sure I could spend a bit of time and do some string formatting, but I'd rather just be able to parse the given date. What's going on?
Edit: I'm able to access the file info easily, though I prefer not to. I'm mainly focusing on why the results I'm seeing are inconstant and showing a length that doesn't match how it prints out, and how I can deal with them. If nothing else, I'm curious as to what is going on.
I wanted to know what the extra characters were ( you dont mention already looking at this. ). I Updated your array code $datearray += $date[$i] to $datearray += [int][char]$date[$i]. The truncated output showed two oddities 8207 and 8206 which translate to left-to-right mark and right-to-left mark. They are normally associated with html. Unfortunately i cannot provide insight to their presense. Good news is that they are easy to remove.
$date = ($date -replace [char]8206) -replace [char]8207
[datetime]::ParseExact($date,"g",$null)
Which nets the output
Thursday, March 26, 2009 1:43:00 PM
Hopefully this is a little bit closer of what you wanted. I tried searching for reasons for the presence of those ascii codes but i didn't find anything useful.
Extra Information for other readers
I did this since i didnt know what the index 12 was. So i made an array that contains the friendly names of all the file meta data possible (288 entries!). I have the here-string located here for brevity.
With this i was testing the following code against a picture of mine.
$objShell = New-Object -ComObject Shell.Application
$objFolder = $objShell.namespace("C:\Temp\")
0..$Meta.GetUpperBound(0)| %{
$metaValue = $objFolder.GetDetailsOf($objFolder.Items().Item(2), $_)
If ($metaValue) {Write-Host "$_ - $($meta[$_]) - $metaValue"}
}
$Meta is the array i spoke of earlier.
The code will cycle though all the details of my file, indicated by Item(2), writing to screen all file details that contain values. In the end there is a line converting the string to a date value. Script output below
0 - Name - IMG_0571.JPG
1 - Size - 3.12 MB
2 - Item type - JPEG image
3 - Date modified - 3/26/2009 3:34 PM
4 - Date created - 8/24/2014 5:19 PM
5 - Date accessed - 8/24/2014 5:19 PM
6 - Attributes - A
9 - Perceived type - Image
10 - Owner - TE_ST\Cameron
11 - Kind - Picture
12 - Date taken - 3/26/2009 1:43 PM
19 - Rating - Unrated
30 - Camera model - Canon EOS DIGITAL REBEL XS
31 - Dimensions - 3888 x 2592
32 - Camera maker - Canon
53 - Computer - TE_ST (this computer)
155 - Filename - IMG_0571.JPG
160 - Bit depth - 24
161 - Horizontal resolution - 72 dpi
162 - Width - 3888 pixels
....output truncated....
247 - Program mode - Normal program
250 - White balance - Auto
269 - Sharing status - Not shared
When I run your code, $objFolder.GetDetailsOf($objFolder.Items().Item(0), 12) I get an empty string. I even changed the item number and the folder I was looking in to make sure I was getting a file object.
However if I do this:
$objShell = New-Object -ComObject Shell.Application
$objFolder = $objShell.namespace("C:\Temp")
$date = $objFolder.Items().Item(3).ModifyDate
I get a value that is already a DateTime object.
(the code above uses my folder and item index)
Related
I recently installed GNAT Ada (2020) and GNAVI GWindows on a new PC.
On trying to build an old project developed under the 2015 version, I get this:
No candidate interpretations match the actuals:
Too many arguments in call to "Put"
expected private type "Printer_Canvas_Type" defined at gwindows-drawing.ads:603
found private type "Canvas_Type" defined at gwindows-drawing.ads:96
This is one of the several similar pieces of code that produce the same result (MapCanvas is declared elsewhere as Canvas_Type):
Put (MapCanvas, (DATUM_BASE_X +
(x * 10 * GRID_MONAD_SPACING)) +
(5 * GRID_MONAD_SPACING),
y - (GRID_MONAD_SPACING + 5),
Integer'Image (x));
Possibly related, I also get this in regard to the last argument in the call to Put - Integer'Image(x)
expected type "Standard.Wide_String"
found type"Standard.String"
As an experiment, I tried converting the last argument using To_Wide_String (Integer'Image (x)) but the result was the same.
Elsewhere, similar code with a literal compiles ok:
Put (MapCanvas, (DATUM_BASE_X - 1 +
(GRID_MONAD_SPACING / 2) +
(x * 10 * GRID_MONAD_SPACING)),
y + 20 + (60 * GRID_MONAD_SPACING),
"0 2 4 6 8 ");
On the previous PC with GNAT Ada 2015, everything compiled. I've compared the declarations
of Put in gwindows-drawing.ads in the old and new installations of GWindows and they are identical.
Unfortunately I can no longer build on the old PC without a lot of backtracking - the old project drive
is in use elsewhere, though I do have it all backed up.
I'd appreciate any ideas on where to look for the cause of the problem.
GWindows has two string modes matching Windows API: ANSI (8-bit character) and Unicode.
The type GString is either a String or a Wide_String.
You can switch the framework's mode with ansi.cmd and unicode.cmd .
Obviously your old project was in ANSI mode.
I want to read MarkLogic logs (for eg : ErrorLog.txt) from query console using Xquery. I had the below code but the problem is output is not properly formatted. Result is like below
xquery version "1.0-ml";
for $hid in xdmp:hosts()
let $h := xdmp:host-name($hid)
return
xdmp:filesystem-file("file://" || $h || "/" ||xdmp:data-directory($hid) ||"/Logs/ErrorLog.txt")
Problem is result is coming as per host basis like first all log of one host is coming and then starting with time 00:00:01 of host 2 and then 00:00:01 of host 3 after running the Xquery.
2019-07-02 00:00:35.668 Info: Merging 2 MB from /cams/q06data02/testQA2/Forests/testQA2-2.2/0002b4cd to /cams/q06data02/testQA2/Forests/testQA2-2.2/0002b4ce, timestamp=15620394303480170
2019-07-02 00:00:36.007 Info: Merged 3 MB at 9 MB/sec to /cams/q06data02/testQA2/Forests/test2-2.2/0002b4ce
2019-07-02 00:00:38.161 Info: Deleted 3 MB at 399 MB/sec /cams/q06data02/test2/Forests/test2-2.2/0002b4cd
Is it possible to get the output with hostname included with log entries and also if we can sort the output by timelines something like
host 1 : 2019-07-02 00:00:01 : Info Merging ....
host 2 : 2019-07-02 00:00:02 : Info Deleted 3 MB at 399 MB/sec ...
Log files are text files. You can parse and sort them like any other text file.
Although they can get very large (GB+), so simple methods may not be performant.
Plus you need to be able to parse the text into fields in order to sort by a field.
Since the first 20 bytes of every line is the time stamp, and that timestamp is in ISO format which sorts lexically same as date, you can split the file by lines and sort using basic colation to get by time sorting of multiple files.
In V9 one can use the pair of xdmp:logfile-scan and xdmp:logmessage-parse to efficiently search over log files (remotely as well as local) and then transform the results into text, XML (attribute or element format) or JSON.
One can also use the REST API for the same.
see: https://docs.marklogic.com/REST/GET/manage/v2/logs
Once logfiles (ideally a selected subset of log messages that is small enough to manage) is converted to a structured format (xml , json or text lines) then sorting, searching, enriching etc is easily performed.
For something much better take a look at Ops Director https://docs.marklogic.com/guide/opsdir/intro
Currently I'm executing:
frama-c -wp -wp-rte -report-rules test_rules.json -wp-split -wp-fct max -wp-status-maybe -wp-status-invalid -wp-timeout 10 -wp-prover alt-ergo -wp-par 12 -warn-signed-overflow -warn-unsigned-overflow -warn-special-float non-finite test.c -then -report-csv test.csv
I read documentation, but didn't find a good explanation how this JSON file works. I found some code at GitHub. But still it's not trivial for Frama-C novice.
I would like to have a CSV that has only rows with status different than Valid and only in test.c file (without dependencies). Is it possible this to be done from JSON config or I have to write custom parser?
I think there is some misunderstanding there: the -report-rules option is meant to be used in conjunction with -report-json. It has no effect on -report-csv, which will always output the whole list of properties. In fact, the very point of -report-csv is to import the resulting file into another tool in order to perform whatever operation you're interested in. For instance, you can simply open the file in your favorite spreadsheet editor and its built-in filters. But there are a lot of programming options as well. Building upon the script written here, here is an example using the python interpreter with the pandas library
>>> import pandas
>>> df = pandas.read_csv("test.csv",sep="\t")
>>> print('There are ' + str(len(df)) + ' properties.')
There are 77 properties.
>>> df = df[df['function']=='merge']
>>> print('There are ' + str(len(df)) + ' properties.')
There are 39 properties.
>>> df = df[df['status']=='Unknown']
>> print('There are ' + str(len(df)) + ' properties.')
There are 3 properties.
>>> print('There are ' + str(len(df)) + ' properties.')
>>> df.to_csv(path_or_buf='res.txt',sep='\t')
This gives me the 3 Unknown properties related to function merge in file res.csv (I hadn't a multi-file example available right away, but of course you would just have to use the file field in your first query). Just keep in mind that the "csv" file is in fact tabular-separated and not comma-separated (since commas tend to appear in ACSL formulas the latter would not be very practical).
I have been digging through the Apple Photos macOS app for a couple weekends now and I am stuck. I am hoping the smart people at StackOverflow can figure this out.
What I don't know:
How are new hex directories determined and how do they correspond to RK.modelId. Perhaps 16 mod of RKFace.ModelId, or mod 256 of RKFace.ModelId?
After a while, the facetile hex value no longer corresponds to the RKFace.ModelId. For example RKFace.modelId 61047 should be facetile_ee77.jpeg. The correct facetile, however, is face/20/01/facetile_1209b.jpeg. hex value 1209b is dec value 73883 for which I have no RKFace.ModelId.
Things I know:
Apple Photos leverages deep learning networks to detect and crop faces out of your imported photos. It saves a cropped jpeg these detected faces into your photo library in resources/media/face/00/00/facetile_1.jpeg.
A record corresponding to this facetile is inserted into RKFace where RKFace.modelId integer is the decimal number of tail hex version of the filename. You can use a standard dec to hex converter and derive the correct values. For example:
Each subdirectory, for example "/00/00" will only hold a maximum of 256 facetiles before it starts a new directory. The directory name is also in hex format with directories. For example 3e, 3f.
While trying to render photo mosaics, I stumbled upon that issue, too...
Then I was lucky to find both a master image and the corresponding facetile,
allowing me to grep around, searching for the decimal and hex equivalent of the numbers embedded in the filenames.
This is what I came up with (assuming you are searching for someone named NAME):
SELECT
printf('%04x', mr.modelId) AS tileId
FROM
RKModelResource mr, RKFace f, RKPerson p
WHERE
f.modelId = mr.attachedModelId
AND f.personId = p.modelId
AND p.displayName = NAME
This select prints out all RKModelResource.modelIds in hex, used to name the corresponding facetiles you were searching for. All that is needed now is the complete path to the facetile.
So, a complete bash script to copy all those facetiles of a person (to a local folder out in the current directory) could be:
#!/bin/bash
set -eEu
PHOTOS_PATH=$HOME/Pictures/Photos\ Library.photoslibrary
DB_PATH=$PHOTOS_PATH/database/photos.db
echo $NAME
mkdir -p out/$NAME
TILES=( $(sqlite3 "$DB_PATH" "SELECT printf('%04x', mr.modelId) AS tileId FROM RKModelResource mr, RKFace f, RKPerson p WHERE f.modelId = mr.attachedModelId AND f.personId = p.modelId AND p.displayName='"$NAME"'") )
for TILE in ${TILES[#]}; do
FOLDER=${TILE:0:2}
SOURCE="$PHOTOS_PATH/resources/media/face/$FOLDER/00/facetile_$TILE.jpeg"
[[ -e "$SOURCE" ]] || continue
TARGET=out/$NAME/$TILE.jpeg
[[ -e "$TARGET" ]] && continue
cp "$SOURCE" "$TARGET" || :
done
I'm getting an OutOfMemory exception from Pig when trying to execute a very simple GROUP BY on a tiny (3KB), randomly-generated, example data set.
The pig script:
$ cat example.pig
raw =
LOAD 'example-data'
USING PigStorage()
AS (thing1_id:int,
thing2_id:int,
name:chararray,
timestamp:long);
grouped =
GROUP raw BY thing1_id;
DUMP grouped;
The data:
$ cat example-data
281906 13636091 hide 1334350350
174952 20148444 save 1334427826
1082780 16033108 hide 1334500374
2932953 14682185 save 1334501648
1908385 28928536 hide 1334367665
[snip]
$ wc example-data
100 400 3239 example-data
Here we go:
$ pig -x local example.pig
[snip]
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:756)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
[snip]
And some extra info:
$ apt-cache show hadoop | grep Version
Version: 1.0.2
$ pig --version
Apache Pig version 0.9.2 (r1232772)
compiled Jan 17 2012, 23:49:20
$ echo $PIG_HEAPSIZE
4096
At this point, I feel like I must be doing something drastically wrong because I can't see any reason why 3 kB of text would ever cause the heap to fill up.
Check this: [link] http://sumedha.blogspot.in/2012/01/solving-apache-pig-javalangoutofmemorye.html
neil, you are right, let me explain the things like this: In the bin/pig script file, the source code is :
JAVA_HEAP_MAX=-Xmx1000m
# check envvars which might override default args
if [ "$PIG_HEAPSIZE" != "" ]; then
JAVA_HEAP_MAX="-Xmx""$PIG_HEAPSIZE""m"
fi
It is setting the Java_heap_size to maxium ("x") using the -Xmx switch only,but i didnot know why this script overriding is not working, that is the reason, i asked you to specify directly the java heap size using the paramters as specified in the link. I didnot got time to check why this problem is raising. If any one have idea please post it here.
You pig job is failing around the following code in MapTask.java:
931 final float recper = job.getFloat("io.sort.record.percent",(float)0.05);
932 final int sortmb = job.getInt("io.sort.mb", 100);
...
945 // buffers and accounting
946 int maxMemUsage = sortmb << 20;
947 int recordCapacity = (int)(maxMemUsage * recper);
948 recordCapacity -= recordCapacity % RECSIZE;
949 kvbuffer = new byte[maxMemUsage - recordCapacity];
So i suggest that you check what the configured value of io.sort.mb and io.sort.record.percent is, and whether following the above logic, maxMemUsage - recordCapacity this is close to, or bigger than your configured JVM heap size (4096 MB)
I toyed with it for a while and ended up switching from the debian packages for hadoop/pig to the raw tarballs, and the problem went away. Not sure what to make of that :)