exiftool—writing one tag by adding/subtracting from another - datetime

I have a directory containing many .jpg and .mov files.
Every .jpg has a DateTimeOriginal exactly three hours more than FileModifyDate and a GPSDateTime exactly eight hours more than FileModifyDate.
But every .mov has neither DateTimeOriginal nor GPSDateTime.
To support future work with these files, I'd like to make them all consistent.  After studying the man page for a while, I tried
exiftool '-DateTimeOriginal<FileModifyDate+03:00' \
'-GPSDateTime<FileModifyDate+08:00' *.MOV
but I got an error message saying I must use = instead of <  So I tried
exiftool '-DateTimeOriginal=FileModifyDate+03:00' \
'-GPSDateTime=FileModifyDate+08:00' *.MOV
and got another message:
Warning: Invalid date/time (use YYYY:mm:dd HH:MM:SS[.ss][+/-HH:MM|Z]) in ExifIFD:DateTimeOriginal (PrintConvInv)
which implies I have to do each tag with a literal time instead of computing from another.
I see that I can copy one tag to another and afterward use += or -= to change it in another step.  But is there a way to do it in one command?

First the answer, but see below for some possible complications.
If the time shift was the same for both values, then you would use the GlobalTimeShift option, but since that is not the case, you need to use the ShiftTime helper function.
Additionally, the GPSDateTime tag in a MOV file is going to be an XMP tag (specifically in the XMP-exif group) which allows for the inclusion of a time zone and does not need to be set to UTC. Since the FileModifyDate already includes a time zone and it looks like you want to shift it to UTC, you will have to strip away the time zone. You can do this by either using the -d (-dateFormat) option, which will set the date format globally, or the DateFmt helper function, which will work on individual tags. A Perl regex substitution would also be an option, but for this example, I'll use the DateFmt option.
Your command would be
exiftool '-DateTimeOriginal<${FileModifyDate;ShiftTime("3")}' '-GPSDateTime<${FileModifyDate;ShiftTime("8");DateFmt("%Y:%m:%d %H:%M:%S")}' *.MOV
For the ShiftTime function, you only need to list the hours to be shifted as this is the default in a tag that includes both date and time. See the ExifTool Date/Time Shift Module for details.
Now, there are some problems with the way you are writing this data. In a MOV file, exiftool will include a time zone in the DateTimeOriginal tag unless forced not to. If the time zone isn't included, then exiftool will default to the local time zone of the computer. So if the local time zone isn't the one that would apply to your shifted DateTimeOriginal time, then you will need to include it. You would now also have to strip away the time zone as above. The change in the above command would be
'-DateTimeOriginal<${FileModifyDate;ShiftTime("3");DateFmt("%Y:%m:%d %H:%M:%S")}}±##:00'
Also, if removing the time zone from both tag copy operations, you could switch to using the -d option.
exiftool -d '%Y:%m:%d %H:%M:%S' '-DateTimeOriginal<${FileModifyDate;ShiftTime("3")}' '-GPSDateTime<${FileModifyDate;ShiftTime("8")}' *.MOV
While the specs say that the DateTimeOriginal tag in a video file does not need to include a time zone, if the time zone isn't included then Apple programs will display wildly inaccurate date/times (see this Exiftool forum thread).
Exiftool will also write to the XMP-exif:DateTimeOriginal tag in addition to the Quicktime tag.
For the GPSDateTime, there might be a better way to write that, if the file's other time stamps are accurate. In a video file, the CreateDate is supposed to be set to UTC. If that tag is correctly set, it would be easier to set the GPSDateTime like this
'-GPSDateTime<CreateDate'
It should be noted that Quicktime:DateTimeOriginal is not commonly read by most programs, the Apple photo app being an exception. Also, XMP tags in video files, in this case XMP-exif:DateTimeOriginal and XMP-exif:GPSDateTime, are usually not used by most programs, with Adobe programs being the exception. The Quicktime:CreateDate is the most widely used tag and since it is supposed to be in UTC, it also works for GPS date/time.
To break down why your original command didn't work, in the first case, you added a static string to the name of the tag. This caused exiftool to treat the whole thing as a string, which is why it told you to use an equal sign = instead of the greater/less than signs </> and the greater/less than signs are only used when copying tags. When you changed it to the equal sign, you were now writing a static string and since the DateTimeOriginal tag requires a date/time format, you receive the Invalid date/time error.
When combining a tag name with a static string, you need to prefix the tag name with the dollar sign. See the paragraph starting "A powerful redirection feature" under the -TagsFromFile option for details.

Related

How to read DICOM private tags without reading/loading pixel data?

I would like to read DICOM private tags. These private tags are under hexadecimal tag x7fe11001.
I know one of the pydicom configurations that read till pixel data starts (so the memory is not loaded up).
pydicom.dcmread(raw, defer_size="2 MB", stop_before_pixels=True)
But the private tags I am trying to read are after the pixel data. So I am ending loading complete file in memory which is not optimal. What are the other ways to read it in an optimal way?
I know there is a config param for the above method, called specific_tags. But I could not find any examples of how to use it.
Any suggestions to read DICOM metadata without loading pixel data into memory would be awesome.
You are right, specific_tags is the correct way to do this:
ds = pydicom.dcmread(raw, specific_tags=[Tag(0x7fe1, 0x1001)]
In this case, ds shall contain only your private tag and the Specific Character Set tag (which is always read).
As DICOM is a sequential format, the other tags still have to be skipped over one by one, but their value is not read.
Note that you can put any number of tags into the specific_tags argument.

TinyButStrong magnet tag shows in output docx

Hi my colleague and I has been trying to get the TinyButStrong plugin openTBS to create some docx files.
We have a live system which creates some RTF files, with data from MySQL. We want to change this to docx, use openTBS. A couple of super users then in Word manage the templates.
We have a problem with creating the files, as we need to remove a line, if data isn't present.
If we in the Word template do
<w:p>[*fieldname*;magnet=w:p]*some kind of text*</w:p>
it hiddes the line if fieldname contains no data, and if if contains data, it will show the line. GREAT :-)
The problem is, that it also shows <w:p> and </w:p> when it contains data, and we don't like that.
How do we get it to stop showing these tags?
The TBS parameter ope=minv is done for thus purpose: it performs the magnet behavior but keep the field invisible (minv stands for magnet invisible).
So the solution is:
<w:p>[*fieldname*;magnet=tbs:p;ope=minv]*some kind of text*</w:p>
By the way, magnet=tbs:p is better than magnet=w:p because your template stays compatible when converted to another other format (LibreOffice).

Why does Win32_Product.InstallDate2 always return null when using WMI?

UPDATE: If you want to help test this quickly in your locale. Launch PowerShell (hold down the Windows key, tap R, release the Windows key, type in "powershell" and press OK) and run this
command - (is the InstallDate2 column empty? Please let us know what you see.):
Get-WmiObject Win32_Product | Format-Table -Property InstallDate, InstallDate2, Name
I should start by saying that this issue is not critical for me (yet), but I am baffled. I am accessing the WMI object Win32_Product, and there are two properties related to dates: InstallDate and InstallDate2 respectively.
InstallDate seems to be a string representation of a DateTime value without the time part. Sort of a cut-off UTC format. Example: 20170819. The field is a string value, and not a DateTime format.
InstallDate2 is supposed to be in real DateTime format, but it is consistently null for all Win32_Product items on all my systems (Windows 7, Windows 10, various editions).
Using WMIExplorer.exe I see this description of InstallDate: "...the InstallDate property has been deprecated in favor of the InstallDate2 property which is of type DateTime rather than String. New implementations should use the InstallDate2 property."
As stated I get null back on all my machines, so I was assuming that InstallDate2 was simply not implemented properly, but then I found this article: Working with Software Installations. This article clearly shows InstallDate2 returning a WMI DateTime (with missing time part?): InstallDate2 = 20060506000000.000000-000.
I am simply wondering what can be causing this InstallDate2 null value on all my systems when I see it working on other systems? Some possibilities that has crossed my mind (one more far fetched than the next perhaps):
OS issue? I don't have access to older OSs, but I see null on several different editions (home, ulitmate, etc...) of Windows 7 and Windows 10 systems. Perhaps it worked on XP and older? I find this unlikely.
Locale issue? I am in a Norwegian locale. Could this affect things? I tried briefly to change the locale on a Windows 10 system, but it had some language pack issues and the result was the same.
Domain (network) issue? Is it conceivable that machines in workgroups work differently than machines that are part of a domain?
Patch level? Could a recent Windows Update have changed something in WMI?
What else could conceivably affect this?
And a second part of the question (in addition to what the cause of the null value for InstallDate2 is). The string I get back for InstallDate (the string format date): 20170819. I am assuming this is just a chopped of version of Universal Time Coordinate (UTC) format. And that it is a locale-independent format? It seems like it when I read about CIM_DATETIME.
I tried filling the rest of the chopped off date string with zeros and passing it to a SWbemDateTime scripting object, and it seemed to work, but that really smells as a "solution". Doesn't seem reliable at all:
Set dateTime = CreateObject("WbemScripting.SWbemDateTime")
dateTime.Value = "20170601000000.000000+000"
MsgBox dateTime.Year
MsgBox dateTime.UTC
It shows 2017 for year, and 0 for UTC. With such a "solution" I obviously really want to get the InstallDate2 DateTime field working correctly so I get a proper Datetime back.
So in all its verbosity, basically a two-part question:
What could be causing Win32_Product.InstallDate2 to consistently report null seemingly only on my systems?
The short string returned by InstallDate - is this in a locale-independent format?
Since I am in a Norwegian locale and see what looks like an English style format, it would seem that it is? Since the field is just a string, the content could basically be "anything" though.
It would be great if someone in another locale could verify that the format returns as: Win32_Product.InstallDate = yyyymmdd.
Reading about CIM_DATETIME seems to indicate the string is locale-independent (even if chopped off).
If you want to test, the quickest way is probably using wbemtest.exe (or better yet PowerShell if you have it available on your box - as mentioned by JosefZ in the comment below):
Launch wbemtest.exe (Hold down the Windows key, tap R, release the Windows key, type in "wbemtest.exe" and press OK).
Click "connect" and then OK (namespace defaults to root\cimv2), and click "connect" again.
Click "Query" and type in this WQL command: SELECT Name,InstallDate2 FROM Win32_Product and click "Use" (or equivalent).
Double click any entry returned, and check InstallDate2 in the middle list box (scroll down).
In my opinion you won't get a full date-time back. The only information available is effectively from MsiGetProductInfo (...INSTALLPROPERTY_INSTALLDATE....). This format is YYYYMMDD. You're only ever going to get the data because that seems to all there is. Note that the documentation for INSTALLPROPERTY_INSTALLDATE says that it is the installation date only if there has been no other servicing since the product was installed, so it appears that you can't even rely on it being the date the product was installed.
See Detecting the time when the program was installed
I suspect the best way to get the time the product was installed is to get the local package path with MsiGetProductInfo (Ex) ...INSTALLPROPERTY_LOCALPACKAGE... and get the cached MSI's CreationDate.

extract elasticsearch date from a <start-date>/<duration> XBRL-JSON format

I am storing XBRL JSON using elasticsearch.
This xBRL-JSON OIM spec describes the oim:period property:
Otherwise, an ISO 8601 time interval representing the {interval}
property, expressed in one of the following forms:
<start>/<end>
<start>/<duration>
<duration>/<end>
Where <start> and <end> are valid according to the xsd:dateTime datatype, and <duration> is valid according to xsd:duration.
Examples from arelle's plugin look like this:
2016-01-01T00:00:00/PT0S
2015-01-01T00:00:00/P1Y
I notice that arelle's plugin exclusively produces this format:
<start>/<duration>
My question
Is there a way to save at least the <start> part as a date type in elasticsearch?
Ideas I had:
elastichsearch only (my preference)
Use a custom date format which anticipates the /<duration> part, but ignores it
I haven't checked Joda yet; will it ignore characters in the date format if they aren't part of the special character? Like the "/" delimiter or the "P" which precedes any duration value (like PT0S and P1Y above)?
EDIT So the single-quote character escapes literals; this works yyyy'/P' will accept a value '2015/P'. However, the rest of the duration could be more dynamic
Re: dynamic; will Joda accept regex or wildcard character like "\d" or "+" qualifier so I can ignore all the possible variations following the P?
Use a character filter to strip out the /<duration> part before saving only <start>as datetime. But I don't know if character filters happen before saving as type: date. If they don't, the '/`part isn't stripped, and I wouldn't be passing valid date strings.
Don't use date type: Use a pattern tokenizer to split on /, and at least the two parts will be saved as separate tokens. Can't use date math, though.
Use a transformation; although it seems like this is deprecated. I read about using copy_to instead, but that seems to combine terms, and I want to break this term apart
Some sort of plugin? Maybe a plugin which will fully support this "interval" datatype described by the OIM spec... maybe a plugin which will store its separate parts...?
change my application (I prefer to use elasticsearch-only techniques if possible)
I could edit this plugin or produce my own plugin which uses exclusively <start> and <end> parts, and saves both into separate fields;
But this breaks the OIM spec, which says they should be combined in a single field
Moreover it can be awkward to express an "instant" fact (with no duration; the PT0S examples above); I guess I just use the same value for end property as start property... Not more awkward than a 0-length duration (PT0S) I guess.
Not a direct answer, but it's worth noting that the latest internal drafts of the xBRL-JSON specification have moved away from the the single-field representation. Although the "/" separated notation is an ISO standard, tool support for it appears to be extremely poor, and so the working group has chosen to switch to separate fields for start and end dates. I would expect Arelle support to follow suit in due course.

What's the correct format for TCDL linkAttributes?

I can see the technology-independent Tridion Content Delivery Language (TCDL) link has the following parameters, which are pretty well described on SDL Live Content.
type
origin
destination
templateURI
linkAttributes
textOnFail
addAnchor
VariantId
How do we add multiple attribute-value pairs for the linkAttributes? Specifically, what do we use to escape the double quotes as well as separate pairs (e.g. if we need class="someclass" and onclick="someevent").
The separate pairs are just space delimited, like a normal series of attributes. Try XML encoding the value of linkAttributes however. So, " become &quote;, etc...
If you are using some Javascript, you might take care of the Javascript quotes too, as in \".
Edit: after I figured out your real question, the answer is a lot simpler:
You should wrap the values inside your linkAttributes in single quotes. Spaces inside linkAttributes are typically handled fine; but if not, escape then with %20.
If you need something more or want something that isn't handled by the standard tcdl:ComponentLink, remember that you can always create your own TCDL tag and and use a TagHandler or TagRenderer (look them up in the docs for examples or search for Jaime's article on TagRenderer) to do precisely what you want.
My original answer was to a question you didn't ask: what is the format for TCDL tags (in general). But the explanation might still be useful to some, so remains below.
I'd suggest having a look at what format the default building blocks (e.g. the Link Resolver TBB in the Default Finish Actions) output and use that as a guide line.
This is what I could quickly get from the transport package of a published page:
<tcdl:Link type="Page" origin="tcm:5-199-64" destination="tcm:5-206-64"
templateURI="tcm:0-0-0" linkAttributes="" textOnFail="true"
addAnchor="" variantId="">Home</tcdl:Link>
<tcdl:ComponentPresentation type="Embedded" componentURI="tcm:5-69"
templateURI="tcm:5-133-32">
<span>
...
One of the things that I know from experience: your entire TCDL tag will have to be on a single line (I wrapped the lines above for readability only). Or at least that is the case if it is used to invoke a REL TagRenderer. Clearly the tcdl:ComponentPresentation tag above will span multiple lines, so that "single line rule" doesn't apply everywhere.
And that is probably the best advice: given the fact that TCDL tags are processed at multiple points in Tridion Publishing, Deployment and Delivery pipeline, I'd stick to the format that the default TBBs output. And from my sample that seems to be: put everything on a single line and wrap the values in (double) quotes.

Resources