So, I have created a variable "batch" with datatype datetime. Now my OLEBD source has a column "addDate" eg 2012-05-18 11:11:17.470 so does empty destination which is to be populated.
now this column addDate has many dates and I want to copy all dates which are "2012-05-18 11:11:17.470"
When I put value of the variable as this date, it automatically changes to mm/dd/yyyy hh;mm AM format and hence in my conditional split transformation, it couldn't match the date with the variable and hence no records are getting copied to the destination !!
Where exactly is the problem?
Thanks!
I had this issue and the best solution I found is not “pretty”.
Basically you need to change the “expression” of the variable and the “evaluate as expression” to true (otherwise it will ignore the value on expression).
The secret is (and kind of the reason I said it is not a pretty solution) to create a second variable to evaluate the expression of the first variable because you can’t change the value of a variable based on a expression.
So let’s say your variable is called “DateVariable” and you have 23/05/2012, create a variable called “DateVar2” for example and set its expression to
(DT_WSTR,4)YEAR(#[User::DateVariable]) + "/"+RIGHT("0" +
(DT_WSTR,2)MONTH(#[User::DateVariable]),2) + "/" + RIGHT("0" +
(DT_WSTR,2)DAY(#[User::DateVariable]),2)
That will give you 2012/05/23
Just keep going to get the date on the format you want
I found the easier solution. Select datatype as string. put any desired value.
Before conditional split, you need data conversion transformation.
convert it into DT_DBTIMESTAMP then run the package.
It works!
Related
I have data frame which a column name starts with number, I want to delete the column with the following code, but there is error:
delete!(features, [:3SsnPorchH])
UndefVarError: SsnPorchH not defined
Your problem is that :3SsnPorchH is not correctly parsed as a symbol, but as follows:
julia> :(:3SsnPorchH)
:($(QuoteNode(3)) * SsnPorchH)
When a symbol cannot be correctly parsed, it most often works to put the "name" into parentheses:
julia> :(3SsnPorchH)
:(3SsnPorchH)
Another thing you could do is using the Symbol constructor directly:
julia> Symbol("3SsnPorchH")
Symbol("3SsnPorchH")
(But I'm not sure if that's a good idea -- maybe you lose interning then.)
That being said, it's probably a good idea to give columns a name which is a valid Julia identifier. This gives you construction using DataFrame with keyword arguments, and allows for certain macros to identify variables with columns. You'll just have an easier time.
In data lake I have file names with pattern yyyyMM_data.csv. Now I want to read previous 3 days data. I am using below code -
DECLARE #ReportDate DateTime= DateTime.Parse("05/08/2017");
DECLARE #FeatureSummaryInput string=#"/FolderPath/{InputFileDate:yyyy}{InputFileDate:MM}_data.csv";
#FeaturedUsed =
EXTRACT Id string,InputFileDate DateTime
FROM #FeatureSummaryInput
USING Extractors.Csv(silent : true, skipFirstNRows : 1);
#FeaturedUsed=
SELECT *
FROM #FeaturedUsed
WHERE InputFileDate BETWEEN #ReportDate.AddDays(-3) AND #ReportDate;
If I run above code it runs with empty input. Please let me know if I am missing something. Why it is not reading correct file?
It seems like we need to must have "day" in file name pattern to work this.
Possibly I am missing something but, as you cast InputFileDate to DateTime it defaults to the first of the month, as no day is specified. For your test ReportDate set to 05/08/2017, your WHERE clause basically evaluates to Between 2017-08-02 And 2017-08-05, which will never be true.
Where do you expect the day element to come in with your files structured as yyyyMM?
I'm using a SQLite3 database, and I have a table that looks like this:
The database is quite big and running queries is very slow. I'm trying to speed up the process by indexing some of the columns. One of the columns that I want to index is the QUOTE_DATETIME column.
Problem: I want to index by date (YYYY-MM-DD) only, not by date and time (YYYY-MM-DD HH:MM:SS), which is the data I currently have in QUOTE_DATETIME.
Question: How can I use CREATE INDEX to create an index that uses only dates in the format YYYY-MM-DD? Should I split QUOTE_DATETIME into 2 columns: QUOTE_DATE and QUOTE_TIME? If so, how can I do that? Is there an easier solution?
Thanks for helping! :D
Attempt 1: I tried running CREATE INDEX id ON DATA (date(QUOTE_DATETIME)) but I got the error Error: non-deterministic functions prohibited in index expressions.
Attempt 2: I ran ALTER TABLE data ADD COLUMN QUOTE_DATE TEXT to create a new column to hold the date only. And then INSERT INTO data(QUOTE_DATE) SELECT date(QUOTE_DATETIME) FROM data. The date(QUOTE_DATETIME) should convert the date + time to only date, and the INSERT INTO should add the new values to QUOTE_DATE. However, it doesn't work and I don't know why. The new column ends up not having anything added to it.
Expression indexes must not use functions that might change their return value based on data not mentioned in the function call itself. The date() function is such a function because it might use the current time zone setting.
However, in SQLite 3.20 or later, you can use date() in indexes as long as you are not using any time zone modifiers.
INSERT adds new rows. To modify existing rows, use UPDATE:
UPDATE Data SET Quote_Date = date(Quote_DateTime);
I´m new to MDX and I have a simple question. I work with the TFS Cube it is named as Team System. My problem:
I have an IIF expression where I want to check additional my expression with an AND operator. There I want to compare two DateTime objects. The report should only show me the data from the actual date. Here my code:
IIF(ISEMPTY(SUM(YTD(
[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember),
[Measures].[EffectivelyValue]))
AND[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember < Now()
, [Measures].[EffectivelyValue]
, SUM(YTD(
[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember),
[Measures].[EffectivelyValue]) )
Planned Week is a self created field which has the DateTime datatype. The Now() function has also a DateTime datatype so the comparision should be right but it happens nothing.
Thanking you in anticipation
Eugen
Hierarchy members in MDX have a data type of 'member', and do not have a 'primitive' data type like datetime, string, or integer. Only member properties have 'primitive' data types. You could either define a property like datetime of your week attribute. Assuming you are SQL Server Analysis Services, this would be done via relationships.
Or you could use string operations to extract the date information from the UniqueName property which avoids having to change the cube. The UniqueName contains the data that you defined as the key in your cube design. Assuming your week hierarchy members have a key from which you can extract something like 20130820 for August 20, 3013 via string functions (I just will use Mid(, 30, 8) as an example below), you could do something like
CLng(Mid([Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName, 30, 8))
<
CLng(Format(Now(), "yyyymmdd"))
You will have to check what exactly the CurrentMember.UniqueName shows in your cube to adapt the above code.
And finally, you could of course also use string methods to extract the relevant parts from the UniqueName and then the CDate function on that to compare to an unchanged Now(), i. e. do all operations on the left side of the <.
Hello thank you very much for you answer. I tried:
Mid([Work Item].[xxxx_PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName,58,10)
shows e.g. 2013-07-21, 2013-07-28 (it shows the week endings)
so I tried this:
AND CLng(Mid([Work Item].[xxxx_PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName,58,10))
<CLng(Format(Now(), "yyyy-mm-dd"))
But it happens nothing. If I execute it in a single way it shows everywhere "true". But I have datasets with dates which are e.g. > 2013-08-23. So there should be false values too.
EDIT: OK I solved the problem. The
Format(Now(), "yyyy-mm-dd")
must be
Format(Now(), "yyyy-MM-dd")
I'm trying to do a query like this on a table with a DATETIME column.
SELECT * FROM table WHERE the_date =
2011-03-06T15:53:34.890-05:00
I have the following as an string input from an external source:
2011-03-06T15:53:34.890-05:00
I need to perform a query on my database table and extract the row which contains this same date. In my database it gets stored as a DATETIME and looks like the following:
2011-03-06 15:53:34.89
I can probably manipulate the outside input slightly ( like strip off the -5:00 ). But I can't figure out how to do a simple select with the datetime column.
I found the convert function, and style 123 seems to match my needs but I can't get it to work. Here is the link to reference about style 123
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.ase_15.0.blocks/html/blocks/blocks125.htm
I think that convert's slightly wrongly documented in that version of the docs.
Because this format always has century I think you only need use 23. Normally the 100 range for convert adds the century to the year format.
That format only goes down to seconds what's more.
If you want more you'll need to past together 2 x converts. That is, past a ymd part onto a convert(varchar, datetime-column, 14) and compare with your trimmed string. milliseconds comparison is likely to be a problem depending on where you got your big time string though because the Sybase binary stored form has a granularity of 300ms I think, so if your source string is from somewhere else it's not likely to compare. In other words - strip the milliseconds and compare as strings.
So maybe:
SELECT * FROM table WHERE convert(varchar,the_date,23) =
'2011-03-06T15:53:34'
But the convert on the column would prevent the use of an index, if that's a problem.
If you compare as datetimes then the convert is on the rhs - but you have to know what your milliseconds are in the_date. Then an index can be used.