I want to use pandas.date_range function as follows:
import pandas as pd
start_date = '2013-01-01'
end_date = '2014-03-01'
dates = pd.date_range(start_date, end_date, freq='M')
When I print dates, the first value in the range is '2013-01-31' instead of my defined start_date, while the last value jumps to the end of the month of the value defined by end_date. This happens with every kind of date I define.
print dates
# output:
#<class 'pandas.tseries.index.DatetimeIndex'>
#[2013-01-31 00:00:00, ..., 2014-03-31 00:00:00]
#Length: 15, Freq: M, Timezone: None
Am I doing something wrong?
The freq='M' indicates that the date range will use dates which are the ends of months. To use dates which are the start of months, use freq='MS'. There is a list of the available aliases and their meanings, here.
>>> pd.date_range(start_date, end_date, freq='MS')
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-01-01, ..., 2014-03-01]
Length: 15, Freq: MS, Timezone: None
Related
I am trying to split a flowfile into multiple flow files on the basis of adding a month to a date which i am getting in the coming flowfile.
eg.
{"to":"2019-12-31T00:00:00Z","from":"2019-03-19T15:36:48Z"}
be the dates i am getting in a flowfile . so i have to split this single flow file into 11 flowfiles with date ranges like
{"to":"2019-04-19","from":"2019-03-19"}
{"to":"2019-05-19","from":"2019-04-19"}
{"to":"2019-06-19","from":"2019-05-19"}
....... and so till
{"to":"2019-12-31","from":"2019-12-19"} .
i have been trying with example inputs to split files with this into day wise flowfiles:
`
begin = '2018-02-15'
end = '2018-04-23'
dt_start = datetime.strptime(begin, '%Y-%m-%d')
dt_end = datetime.strptime(end, '%Y-%m-%d')
one_day = timedelta(days = 1)
start_dates = [dt_start]
end_dates = []
today = dt_start
while today <= dt_end:
tomorrow = today + one_day
print(tomorrow)
`
but i get a error in my Execute script processor. nifi flow screenshot
Since you're using Jython, you may have to cast today to some Jython/Python time variable or call today.getTime() in order to do arithmetic operations on it.
I have some data in an SQLite DB of the form:
id column1 date
111 280 1/1/2014
114 275 1/2/2014
The date field is of type TEXT. I've been made aware (https://www.sqlite.org/lang_datefunc.html) that I should have the dates formatted like YYYY-MM-DD to take advantage of SQLite's datetime functionality. Is there a query I could run to change the format from
mm/dd/yyyy
to
YYYY-MM-DD
in place?
Your current date format has four possible forms:
m/d/yyyy
m/dd/yyyy
mm/d/yyyy
mm/dd/yyyy
To rearrange the fields, extract them with substr() and then combine them again.
It might be possible to determine the positions of the slashes with instr(), but for a one-off conversion, just using four queries is simpler:
UPDATE MyTable
SET date = substr(date, 6, 4) || '-' ||
substr(date, 1, 2) || '-' || '0' ||
substr(date, 4, 1)
WHERE date LIKE '__/_/____';
-- this is mm/d/yyyy; similarly for the other forms, modify positions and zeros
Without any frills such as exception handling!
This approach is slightly simpler because strptime doesn't mind about presence or absence of leading zeroes in days and months.
>>> from datetime import datetime
>>> import sqlite3
>>> con = sqlite3.connect(':memory:')
>>> cur = con.cursor()
>>> cur.execute('CREATE TABLE EXAMPLE (date_column text)')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> cur.execute('INSERT INTO EXAMPLE VALUES ("1/1/2014")')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> def transformDate(aDate):
... tempDate = datetime.strptime(aDate, '%d/%m/%Y')
... return tempDate.strftime('%Y-%m-%d')
...
>>> transformDate('1/1/2014')
'2014-01-01'
>>> con.create_function('transformDate', 1, transformDate)
>>> cur.execute('UPDATE EXAMPLE SET date_column = transformDate(date_column)')
<sqlite3.Cursor object at 0x00000000038D07A0>
Field of Model :
time = models.DateTimeField()
How to get objects which were created (only) today (from 00:00:00 to 23:59:59)
like:
objects = Model.objects.filter(time__gt=?????????)
or ?
Thanks
You can use datetime.date.today() to get the current date and then filter objects based on today's date.
You can do something like:
import datetime
today = datetime.date.today() # date representing today's date
qs = MyModel.objects.filter(time__gt=today) # filter objects created today
Here, qs represents the objects which were created today.
Another solution is to use range which is used to perform lookup between two dates.
Here, start_date represents 00:00:00 and end_date represents 23:59:59.
import datetime
today = datetime.datetime.today()
start_date = datetime.datetime(year=today.year, month=today.month, day=today.day, hour=0, minute=0, second=0) # represents 00:00:00
end_date = datetime.datetime(year=today.year, month=today.month, day=today.day, hour=23, minute=59, second=59) # represents 23:59:59
qs = MyModel.objects.filter(time__range=(start_date, end_date)) # today's objects
Since you are using Django Rest Framework, you might need to override the get_queryset() method in your view and return the queryset containing the objects that were created today.
class MyView(..):
def get_queryset(self):
..
return qs # return the queryset created using the above logic
import datetime
today = datetime.date.today()
qs = MyModel.objects.filter(time__date=today)
I am using JodaTime2 library to create a date object with a given timezone as follow:
import org.joda.time.DateTime;
import org.joda.time.DateTimeZone;
DateTimeZone tz = DateTimeZone.forID("America/New_York");
System.out.println("timezone=" + tz);
Date d = new DateTime(2013, 1, 1, 0, 0, tz).toDate();
System.out.println("Cur Date = " + d);
However when I print this date, the timezone reported is CST. What am I missing ?
timezone=America/New_York
Cur Date = Tue Jan 01 13:00:00 CST 2013
You're printing out the value of a Date object. Date doesn't have a time zone - Date.toString() always just uses the "default" time zone. A Date is just a number of milliseconds since the Unix epoch; it doesn't know about calendars or time zones.
You should either just stick within the Joda Time world, or (if you must) use a SimpleDateFormatter to convert a Date to a String - you can set the time zone on the formatter.
Not sure how to approach this one.
User supplies an argument, ie, program.exe '2001-08-12'
I need to add a single day to that argument - this will represent a date range for another part of the program. I am aware that you can add or subtract from the current day but how does one add or subtract from a user supplied date?
import datetime
...
date=time.strptime(argv[1], "%y-%m-%d");
newdate=date + datetime.timedelta(days=1)
Arnauds Code is valid,Just see how to use it :) :-
>>> import datetime
>>> x=datetime.datetime.strptime('2001-08-12','%Y-%m-%d')
>>> newdate=x + datetime.timedelta(days=1)
>>> newdate
datetime.datetime(2001, 8, 13, 0, 0)
>>>
Okay, here's what I've got:
import sys
from datetime import datetime
user_input = sys.argv[1] # Get their date string
year_month_day = user_input.split('-') # Split it into [year, month, day]
year = int(year_month_day[0])
month = int(year_month_day[1])
day = int(year_month_day[2])
date_plus_a_day = datetime(year, month, day+1)
I understand this is a little long, but I wanted to make sure each step was clear. I'll leave shortening it up to you if you want it shorter.