In data lake I have file names with pattern yyyyMM_data.csv. Now I want to read previous 3 days data. I am using below code -
DECLARE #ReportDate DateTime= DateTime.Parse("05/08/2017");
DECLARE #FeatureSummaryInput string=#"/FolderPath/{InputFileDate:yyyy}{InputFileDate:MM}_data.csv";
#FeaturedUsed =
EXTRACT Id string,InputFileDate DateTime
FROM #FeatureSummaryInput
USING Extractors.Csv(silent : true, skipFirstNRows : 1);
#FeaturedUsed=
SELECT *
FROM #FeaturedUsed
WHERE InputFileDate BETWEEN #ReportDate.AddDays(-3) AND #ReportDate;
If I run above code it runs with empty input. Please let me know if I am missing something. Why it is not reading correct file?
It seems like we need to must have "day" in file name pattern to work this.
Possibly I am missing something but, as you cast InputFileDate to DateTime it defaults to the first of the month, as no day is specified. For your test ReportDate set to 05/08/2017, your WHERE clause basically evaluates to Between 2017-08-02 And 2017-08-05, which will never be true.
Where do you expect the day element to come in with your files structured as yyyyMM?
Related
All,
I am trying to nest a macro within a macro but am unsuccessful. The Start_Cycle variable is set every few months and updated manually. I want to create a start_point variable that goes back 6 months and I successfully created it, however, the output includes a space after the %STR as seen below
%let start_cycle = '01JUL2022:00:00:00'dt; /set to beginning month of this cycle/
%let start_point = %STR(%')%sysfunc(intnx(DTMONTH,&start_cycle.,-6,b),datetime19.)%STR(%')dt;
%put &start_point;
Output below
%let start_cycle = '01JUL2022:00:00:00'dt; /set to beginning month of this cycle/
%let start_point =
%STR(%')%sysfunc(intnx(DTMONTH,&start_cycle.,-6,b),datetime19.)%STR(%')dt;
%put &start_point; ' 01JAN2022:00:00:00'dt
^^Does anyone know why there is a space after the single quote? ' 01JAN2022:00:00:00'dt
Since it runs without issues, I decided to create another macro variable that does the same thing, but instead, the output needs to be converted to a character string in this format below (current Macro)
%let start_pointSales = '2022/01';
I tried multiple times using different ways of going about this, spent many hours looking through forum from SAS Communities to StackOverflow and even SAS youtube videos to no luck. Anyone have any luck in combating this?
To-Be Macro:
%let NEW_start_pointSales = %sysfunc(intnx(month,&start_cycle.,-6,b),yymms.);
%put &NEW_start_pointSales;
The NEW_start_pointSales will be used in the WHERE clause with Data type Varchar (21) using PROC SQL.
left join EDWSALE.DSCOE_LLD (where=( &NEW_start_pointSales. <= SALES_MONTH < &end_pointSales.
Output Error below:
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
24
25 GOPTIONS ACCESSIBLE;
WARNING: An argument to the function INTNX referenced by the %SYSFUNC or %QSYSFUNC macro function is out of range.
NOTE: Mathematical operations could not be performed during %SYSFUNC function execution. The result of the operations have been set
to a missing value.
Any help is appreciated!
You cannot apply a format, YYMMS, designed to work on DAYS to a value that is in SECONDS. If you want to use a date format on a datetime value you need to convert the value from seconds to days. You could use the DATEPART() function or just divide by the number of seconds in a day.
Why are you trying to compare a variable that is CHARACTER to numbers? If you generate a macro variable with a string like 2022/01 and then use it to generate code like:
&NEW_start_pointSales. <= SALES_MONTH
that will result in code like:
2022/01 <= SALES_MONTH
which is comparing the number 2022 (any number divided by 1 is itself) to the value SALES_MONTH, which you just said was a character string.
What types of strings does SALES_MONTH contain? That will determine how (or whether) you can make inequality tests against it.
PS There is space in the output of DATETIME19 because that is how that particular format works. Note that there is a bug in the DATETIME format and you cannot use DATETIME18. to produce a string with four digits for the year even though only 18 characters would be used. The extra space does not matter to using the string in a datetime literal as the DATETIME informat will recognize the string even with the extra space.
What is wrong with my code:
ExecSql('DELETE FROM STLac WHERE RegN=99 AND BegDate>= 2016-12-14');
This runs, but deletes ALL the rows in STLac for RegN, not just the rows with BegDate on or after 2016-12-14.
Originally I had:
ExecSql('DELETE FROM STLac WHERE RegN=99 AND BegDate>= :myDdate,[myDate]);
which has the advantage I hoped of not being particular to the date format. So I tried the literal date should in the format SQLite likes. Either way I get all rows deleted, not just those on or after the specified date.
Scott S.
Try double quote while putting date. As any value must be provided in between quotes until and unless that column is not int
ExecSql('DELETE FROM STLac WHERE RegN=99 AND BegDate>= "2016-12-14"');
SQLite does not have datetime format as such, so you have to figure out how date is actually represented in the table and change your query to provide the same format. First execute the "select" statement in some kind of management tool,
select * from STLac where RegN = 99 and BegDate >= '2016-12-14' --(or '2016.12.04' or something else)
which displays the result in the grid; when you see the expected rows, change it to "delete" query and copy into your Delphi program.
I´m new to MDX and I have a simple question. I work with the TFS Cube it is named as Team System. My problem:
I have an IIF expression where I want to check additional my expression with an AND operator. There I want to compare two DateTime objects. The report should only show me the data from the actual date. Here my code:
IIF(ISEMPTY(SUM(YTD(
[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember),
[Measures].[EffectivelyValue]))
AND[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember < Now()
, [Measures].[EffectivelyValue]
, SUM(YTD(
[Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember),
[Measures].[EffectivelyValue]) )
Planned Week is a self created field which has the DateTime datatype. The Now() function has also a DateTime datatype so the comparision should be right but it happens nothing.
Thanking you in anticipation
Eugen
Hierarchy members in MDX have a data type of 'member', and do not have a 'primitive' data type like datetime, string, or integer. Only member properties have 'primitive' data types. You could either define a property like datetime of your week attribute. Assuming you are SQL Server Analysis Services, this would be done via relationships.
Or you could use string operations to extract the date information from the UniqueName property which avoids having to change the cube. The UniqueName contains the data that you defined as the key in your cube design. Assuming your week hierarchy members have a key from which you can extract something like 20130820 for August 20, 3013 via string functions (I just will use Mid(, 30, 8) as an example below), you could do something like
CLng(Mid([Work Item].[PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName, 30, 8))
<
CLng(Format(Now(), "yyyymmdd"))
You will have to check what exactly the CurrentMember.UniqueName shows in your cube to adapt the above code.
And finally, you could of course also use string methods to extract the relevant parts from the UniqueName and then the CDate function on that to compare to an unchanged Now(), i. e. do all operations on the left side of the <.
Hello thank you very much for you answer. I tried:
Mid([Work Item].[xxxx_PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName,58,10)
shows e.g. 2013-07-21, 2013-07-28 (it shows the week endings)
so I tried this:
AND CLng(Mid([Work Item].[xxxx_PlannedWeek__HierarchyByWeek].CurrentMember.UniqueName,58,10))
<CLng(Format(Now(), "yyyy-mm-dd"))
But it happens nothing. If I execute it in a single way it shows everywhere "true". But I have datasets with dates which are e.g. > 2013-08-23. So there should be false values too.
EDIT: OK I solved the problem. The
Format(Now(), "yyyy-mm-dd")
must be
Format(Now(), "yyyy-MM-dd")
So, I have created a variable "batch" with datatype datetime. Now my OLEBD source has a column "addDate" eg 2012-05-18 11:11:17.470 so does empty destination which is to be populated.
now this column addDate has many dates and I want to copy all dates which are "2012-05-18 11:11:17.470"
When I put value of the variable as this date, it automatically changes to mm/dd/yyyy hh;mm AM format and hence in my conditional split transformation, it couldn't match the date with the variable and hence no records are getting copied to the destination !!
Where exactly is the problem?
Thanks!
I had this issue and the best solution I found is not “pretty”.
Basically you need to change the “expression” of the variable and the “evaluate as expression” to true (otherwise it will ignore the value on expression).
The secret is (and kind of the reason I said it is not a pretty solution) to create a second variable to evaluate the expression of the first variable because you can’t change the value of a variable based on a expression.
So let’s say your variable is called “DateVariable” and you have 23/05/2012, create a variable called “DateVar2” for example and set its expression to
(DT_WSTR,4)YEAR(#[User::DateVariable]) + "/"+RIGHT("0" +
(DT_WSTR,2)MONTH(#[User::DateVariable]),2) + "/" + RIGHT("0" +
(DT_WSTR,2)DAY(#[User::DateVariable]),2)
That will give you 2012/05/23
Just keep going to get the date on the format you want
I found the easier solution. Select datatype as string. put any desired value.
Before conditional split, you need data conversion transformation.
convert it into DT_DBTIMESTAMP then run the package.
It works!
I'm trying to do a query like this on a table with a DATETIME column.
SELECT * FROM table WHERE the_date =
2011-03-06T15:53:34.890-05:00
I have the following as an string input from an external source:
2011-03-06T15:53:34.890-05:00
I need to perform a query on my database table and extract the row which contains this same date. In my database it gets stored as a DATETIME and looks like the following:
2011-03-06 15:53:34.89
I can probably manipulate the outside input slightly ( like strip off the -5:00 ). But I can't figure out how to do a simple select with the datetime column.
I found the convert function, and style 123 seems to match my needs but I can't get it to work. Here is the link to reference about style 123
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.ase_15.0.blocks/html/blocks/blocks125.htm
I think that convert's slightly wrongly documented in that version of the docs.
Because this format always has century I think you only need use 23. Normally the 100 range for convert adds the century to the year format.
That format only goes down to seconds what's more.
If you want more you'll need to past together 2 x converts. That is, past a ymd part onto a convert(varchar, datetime-column, 14) and compare with your trimmed string. milliseconds comparison is likely to be a problem depending on where you got your big time string though because the Sybase binary stored form has a granularity of 300ms I think, so if your source string is from somewhere else it's not likely to compare. In other words - strip the milliseconds and compare as strings.
So maybe:
SELECT * FROM table WHERE convert(varchar,the_date,23) =
'2011-03-06T15:53:34'
But the convert on the column would prevent the use of an index, if that's a problem.
If you compare as datetimes then the convert is on the rhs - but you have to know what your milliseconds are in the_date. Then an index can be used.