I am trying to split a flowfile into multiple flow files on the basis of adding a month to a date which i am getting in the coming flowfile.
eg.
{"to":"2019-12-31T00:00:00Z","from":"2019-03-19T15:36:48Z"}
be the dates i am getting in a flowfile . so i have to split this single flow file into 11 flowfiles with date ranges like
{"to":"2019-04-19","from":"2019-03-19"}
{"to":"2019-05-19","from":"2019-04-19"}
{"to":"2019-06-19","from":"2019-05-19"}
....... and so till
{"to":"2019-12-31","from":"2019-12-19"} .
i have been trying with example inputs to split files with this into day wise flowfiles:
`
begin = '2018-02-15'
end = '2018-04-23'
dt_start = datetime.strptime(begin, '%Y-%m-%d')
dt_end = datetime.strptime(end, '%Y-%m-%d')
one_day = timedelta(days = 1)
start_dates = [dt_start]
end_dates = []
today = dt_start
while today <= dt_end:
tomorrow = today + one_day
print(tomorrow)
`
but i get a error in my Execute script processor. nifi flow screenshot
Since you're using Jython, you may have to cast today to some Jython/Python time variable or call today.getTime() in order to do arithmetic operations on it.
Related
I have an issue trying to load bigquery data in R and using a date filter (the filter date is variable).
projectid = "projectid"
sql <- sprintf("SELECT *
from [table]
where num_start <21 and t < %s ", paste("'", Sys.Date() -12, "'", sep=""))
I have the following error:
Error: 4.34 - 4.45: Could not parse '2019-01-04' as a timestamp. Required format is YYYY-MM-DD HH:MM[:SS[.SSSSSS]] [invalidQuery]
Then I changed Sys.Date() to Sys.time() the query works but I would like to fetch the data n days prior to a the current day.
You must be using legacy-SQL. Instead, change your query to use standard-SQL and it will for fine then. For example:
WITH
input AS (
SELECT
TIMESTAMP_MILLIS(1547636619070) AS ts)
SELECT
*
FROM
input
WHERE
ts > '2019-01-04'
I am working on statistics page of my app and trying to query data by date.
To get the date range, I use Calendar.Date
date_range = Date.days_after_until(start_date, end_date, true)
|> Enum.to_list
And it returns date list of dates and each date looks like "2017-04-07". So with the date I got from date_range, I tried to query but it triggers an error like below.
where cannot be cast to type Ecto.DateTime in query: from o in Myapp.Order,
where: o.created_date >= ^~D[2017-04-07]
For created_date field of Order, I made field like this,
field :created_date, Ecto.DateTime.
If I want to query by date, how can I query it?
Thank in advance.
It looks like you're trying to compare a date and datetime. You need to cast one of them to the other type so the comparison works, e.g. convert the datetime in the database to a date:
date = ~D[2017-01-01]
from p in Post, where: fragment("?::date", p.inserted_at) >= ^date
or convert the Elixir Date to NaiveDateTime:
{:ok, datetime} = NaiveDateTime.new(date, ~T[00:00:00])
from p in Post, where: p.inserted_at >= ^datetime
If you have a start and end date, you just need to add an and to either. You don't need to generate the whole list of dates using any library.
from p in Post,
where: fragment("?::date", p.inserted_at) >= ^start_date and
fragment("?::date", p.inserted_at) <= ^end_date
or
from p in Post,
where: p.inserted_at >= ^start_datetime and
p.inserted_at <= ^end_datetime
For my AgingCalendar field, I have 3 conditions using CASE WHEN:
CASE WHEN A.[END_DTTM] > A.[STRT_DTTM] THEN C2.[DY_OF_CAL_NUM] - C1.[DY_OF_CAL_NUM]
WHEN A.[END_DTTM] IS NULL and A.[STRT_DTTM] IS NOT NULL THEN C3.[DY_OF_CAL_NUM] - C1.[DY_OF_CAL_NUM]
WHEN A.[END_DTTM] = A.[STRT_DTTM] THEN 1
END AS AgeCalendar
For my third condition, I'm trying to basically say when the End Datetime = Start Datetime, the age in Calendar days should be set to 1 calendar day.
However, in some of the records I'm bringing in, the start date equals the end date, but the times associated with each datetime are different. When this happens, those records are receiving a NULL in the AgeCalendar field.(For example I could have 6/6/2014 0:00:00 = 6/6/2014 0:00:00, and that will give me 1...but if I had 6/6/2014 0:00:00 = 6/6/2014 0:03:59 (or something like that)...it'll give me a NULL value because it's not matching.
How can I update the code above so that I'm basically saying when End Date = Start Date, then 1...regardless of not having matching times?
CASTor CONVERT them as dates to ignore the time.
WHEN CONVERT(DATE, A.[END_DTTM]) = CONVERT(DATE, A.[STRT_DTTM]) THEN 1
OR
WHEN CAST(A.[END_DTTM] AS DATE) = CAST(A.[STRT_DTTM] AS DATE) THEN 1
I have one variable
Dim tt="2008-10-20 10:00:00.0000000"
I want to change it into date,
Try CDATE(tt) see http://www.w3schools.com/vbscript/func_cdate.asp. I used
vbscript cdate
as keywords at Google. There were more results.
Edit: Based on the comment below (I'm sorry for mixing up), using
FormatDateTime(date,format)
Format contains following constants:
0 = vbGeneralDate - Default. Returns date: mm/dd/yy and time if
specified: hh:mm:ss PM/AM.
1 = vbLongDate - Returns date: weekday, monthname, year
2 = vbShortDate - Returns date: mm/dd/yy
3 = vbLongTime - Returns time: hh:mm:ss PM/AM
4 = vbShortTime - Return time: hh:mm
(copied from http://www.w3schools.com/vbscript/func_formatdatetime.asp)
This link, (MS CDate page), explains that:
adate = CDate(astring)
converts a string into a date object. For there, you can format it with the FormatDateTime function
str = FormatDateTime(Date)
the FormatDateTime function is "smart" -- it will format as date and time if both are present, otherwise it will format with whichever of date or time is present.
I propose a safe solution which returns the result only if the conversion is successful:
s="2008-10-20 10:00:00.0000000"
On Error Resume Next
d=CDate(Left(s,19))
On Error Goto 0
if not IsEmpty(d) then MsgBox d
Try it for a non-valid date or non-valid format. The result will be empty.
s="2008-02-31 10:00:00"
In same contexts, it is necessary to initialize the variable collecting result of CData. I recommend to initialize it as Empty. Example below shows such case - counting valid dates in a string array:
Lines = array("2008-10-20 10:00:00.0000000", "2008-10-20 10:00:00", "", "2008-02-31", "Today", "2017-02-7")
On Error Resume Next
Count=0
for each Line in Lines
d=Empty
d=CDate(Line)
if not IsEmpty(d) then Count=Count+1
next
On Error Goto 0
MsgBox "Number of valid dates is "&Count
The correct answer is 2. Without initialization we get 5 as the CDate does not do anything on error so variable keeps the value from a recent iteration in the loop.
If do not need your milliseconds, your could use the following:
<script type="text/vbscript">
s="2008-10-20 10:00:00.0000000"
arr= Split(s, ".")
d=CDate(arr(0))
document.write(d)
</script>
I believe cdate is dependent on local settings to parse the string. This is no good in many situations.
To avoid this you need to use
DateSerial()
and if needed add any time components to the result separately.
The date literal in classic asp is unreliable. If the first or second part is greater than 12, it takes that value for day, the other as month. If both parts are less than 12, the interpretation is unpredictable: sometimes american and sometimes british.
A work-around is to force the entry of dates into separate fields or use a date entry module which can set the date into british or american style.
A date literal should be treated as american and use a function to convert that into a date variable using Dateserial().
function amerdate(str)
'str 3/5/2023 form: in american format: use for calculations including date
Dim d
d = Split(str, "/")
amerdate = Dateserial(d(2), d(0), d(1))
end function
Use only american date in calculations like Dateadd() etc.
someday = "3/5/2023" '5th march, american date literal
nextday = Dateadd("d", 1, amerdate(someday))
Whenever a date display is required, convert to british date and show it.
function britdate(str)
'str: 3/5/2023 form: display in british form. not for calculations
Dim d
d= Split(str, "/")
britdate = d(1) & "/" & d(0) & "/" & d(2)
end function
Foo.group(:start_at).count(:id)
How I can group this by date ? the "start_at" column is an datetime column
This should work (rails 3):
Foo.order(:start_at).group("DATE(start_at)").count
edit: if you're using PostgreSQL, the query should be
Foo.order("DATE(start_at)").group("DATE(start_at)").count
or you'll get an error
("PGError: ERROR: column "foos.start_at" must appear in the GROUP BY clause or be used in an aggregate function")
Based on
Graphing new users by date in a Rails app using Seer
and
http://www.pastbedti.me/2009/11/grouping-a-timestamp-field-by-date-in-ruby-on-rails-postgresql/
I created a gem for this. https://github.com/ankane/groupdate
Foo.group_by_day(:created_at).count
You can even specify a time zone.
Foo.group_by_day(:created_at, time_zone: "Pacific Time (US & Canada)").count
This is another way to solve this if you manage different time zones. The idea is to add or substract the hours of your timezone:
hours_diff = Time.zone.now.time_zone.utc_offset/(60*60) rescue 0
hours_diff = hours_diff.to_i
date_group = "DATE(form_answers.created_at)"
if hours_diff > 0
date_group = "DATE(DATE_ADD(form_answers.created_at, INTERVAL #{hours_diff} HOUR ))"
elsif hours_diff < 0
date_group = "DATE(DATE_SUB(form_answers.created_at, INTERVAL #{hours_diff.abs} HOUR ))"
end
Foo.group(date_group).count