Not sure how to approach this one.
User supplies an argument, ie, program.exe '2001-08-12'
I need to add a single day to that argument - this will represent a date range for another part of the program. I am aware that you can add or subtract from the current day but how does one add or subtract from a user supplied date?
import datetime
...
date=time.strptime(argv[1], "%y-%m-%d");
newdate=date + datetime.timedelta(days=1)
Arnauds Code is valid,Just see how to use it :) :-
>>> import datetime
>>> x=datetime.datetime.strptime('2001-08-12','%Y-%m-%d')
>>> newdate=x + datetime.timedelta(days=1)
>>> newdate
datetime.datetime(2001, 8, 13, 0, 0)
>>>
Okay, here's what I've got:
import sys
from datetime import datetime
user_input = sys.argv[1] # Get their date string
year_month_day = user_input.split('-') # Split it into [year, month, day]
year = int(year_month_day[0])
month = int(year_month_day[1])
day = int(year_month_day[2])
date_plus_a_day = datetime(year, month, day+1)
I understand this is a little long, but I wanted to make sure each step was clear. I'll leave shortening it up to you if you want it shorter.
Related
I have the TIMESTAMP data like:
[29:23:59:45]
This stands for whatever month 29, 23:59:45
How can I convert in PySpark to like DAY 29, TIME:23:59:45?
Possibly using something like
from datetime import datetime
dVal = datetime.strptime('[29:23:59:45]', '%d/%h/%m/%s')
This is a classic example for which is needed to use a User Defined Function (UDF).
from datetime import datetime
from spark.sql import functions as F
def toDate(x):
return datetime.strptime(x, '%m %H:%M:%S')
toDate = F.udf(toDate)
new_df = df.withColumn('date', toDate(F.col('timestamp'))
where, df is supposed to be the old dataframe containing a column named 'timestamp' as you reported.
I am trying to split a flowfile into multiple flow files on the basis of adding a month to a date which i am getting in the coming flowfile.
eg.
{"to":"2019-12-31T00:00:00Z","from":"2019-03-19T15:36:48Z"}
be the dates i am getting in a flowfile . so i have to split this single flow file into 11 flowfiles with date ranges like
{"to":"2019-04-19","from":"2019-03-19"}
{"to":"2019-05-19","from":"2019-04-19"}
{"to":"2019-06-19","from":"2019-05-19"}
....... and so till
{"to":"2019-12-31","from":"2019-12-19"} .
i have been trying with example inputs to split files with this into day wise flowfiles:
`
begin = '2018-02-15'
end = '2018-04-23'
dt_start = datetime.strptime(begin, '%Y-%m-%d')
dt_end = datetime.strptime(end, '%Y-%m-%d')
one_day = timedelta(days = 1)
start_dates = [dt_start]
end_dates = []
today = dt_start
while today <= dt_end:
tomorrow = today + one_day
print(tomorrow)
`
but i get a error in my Execute script processor. nifi flow screenshot
Since you're using Jython, you may have to cast today to some Jython/Python time variable or call today.getTime() in order to do arithmetic operations on it.
Learning Kusto query and looking for a way to get beginning of current month datetime.
As of time I post this it is 2/25/2020 so output should looks like below represents Feb 1, 2020
This is what I have so far and works, but there should be better way of doing this.
Can anyone please let me know if this query can be improved?
What's the common practice of getting beginning of current month?
Below, get year and month, add leading 0 if needed for month then concatenate the string and assign to variable "d" which then look like "2020-02-01" and pass that string to todatetime()
let year = datetime_part("Year",now());
let month = datetime_part("Month",now());
let m = case(month < 10, strcat("0", month), tostring(month));
let d = strcat(year, "-", m, "-01" );
print todatetime(d);
Try the startofmonth() function.
Example:
MyKustoTable
| project MonthStart = startofmonth(datetime('2020-2-5'))
Reference: https://learn.microsoft.com/en-us/azure/kusto/query/startofmonthfunction
there's a startofmonth() function: https://learn.microsoft.com/en-us/azure/kusto/query/startofmonthfunction
I have some data in an SQLite DB of the form:
id column1 date
111 280 1/1/2014
114 275 1/2/2014
The date field is of type TEXT. I've been made aware (https://www.sqlite.org/lang_datefunc.html) that I should have the dates formatted like YYYY-MM-DD to take advantage of SQLite's datetime functionality. Is there a query I could run to change the format from
mm/dd/yyyy
to
YYYY-MM-DD
in place?
Your current date format has four possible forms:
m/d/yyyy
m/dd/yyyy
mm/d/yyyy
mm/dd/yyyy
To rearrange the fields, extract them with substr() and then combine them again.
It might be possible to determine the positions of the slashes with instr(), but for a one-off conversion, just using four queries is simpler:
UPDATE MyTable
SET date = substr(date, 6, 4) || '-' ||
substr(date, 1, 2) || '-' || '0' ||
substr(date, 4, 1)
WHERE date LIKE '__/_/____';
-- this is mm/d/yyyy; similarly for the other forms, modify positions and zeros
Without any frills such as exception handling!
This approach is slightly simpler because strptime doesn't mind about presence or absence of leading zeroes in days and months.
>>> from datetime import datetime
>>> import sqlite3
>>> con = sqlite3.connect(':memory:')
>>> cur = con.cursor()
>>> cur.execute('CREATE TABLE EXAMPLE (date_column text)')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> cur.execute('INSERT INTO EXAMPLE VALUES ("1/1/2014")')
<sqlite3.Cursor object at 0x00000000038D07A0>
>>> def transformDate(aDate):
... tempDate = datetime.strptime(aDate, '%d/%m/%Y')
... return tempDate.strftime('%Y-%m-%d')
...
>>> transformDate('1/1/2014')
'2014-01-01'
>>> con.create_function('transformDate', 1, transformDate)
>>> cur.execute('UPDATE EXAMPLE SET date_column = transformDate(date_column)')
<sqlite3.Cursor object at 0x00000000038D07A0>
I want to use pandas.date_range function as follows:
import pandas as pd
start_date = '2013-01-01'
end_date = '2014-03-01'
dates = pd.date_range(start_date, end_date, freq='M')
When I print dates, the first value in the range is '2013-01-31' instead of my defined start_date, while the last value jumps to the end of the month of the value defined by end_date. This happens with every kind of date I define.
print dates
# output:
#<class 'pandas.tseries.index.DatetimeIndex'>
#[2013-01-31 00:00:00, ..., 2014-03-31 00:00:00]
#Length: 15, Freq: M, Timezone: None
Am I doing something wrong?
The freq='M' indicates that the date range will use dates which are the ends of months. To use dates which are the start of months, use freq='MS'. There is a list of the available aliases and their meanings, here.
>>> pd.date_range(start_date, end_date, freq='MS')
<class 'pandas.tseries.index.DatetimeIndex'>
[2013-01-01, ..., 2014-03-01]
Length: 15, Freq: MS, Timezone: None