I have a label in my asp.net web site that will shows the time. I want the output like here. in the morning like this: 08:26 and after 12 am,it shows 15:28
My code does not work. It only supports the first part.
DateTime tim = DateTime.Now;
int hh = p.GetHour(tim);
int mm = p.GetMinute(tim);
Label7.Text = DateTime.Now.ToString("hh:mm");
According to the Custom date and time format strings docs page - you can see:
"hh" The hour, using a 12-hour clock from 01 to 12.
"HH" The hour, using a 24-hour clock from 00 to 23.
So in your case - just use the capitalized HH for your formatting:
Label7.Text = DateTime.Now.ToString("HH:mm");
and you should get what you're looking for.
In the azure data explorer documentation, there is a lot of supported formats but not the one that i am looking for.
What I need is to format the datetime like "yyyy-MM-dd HH" to set the minutes and seconds as 0
Input datetime
2020-04-21T17:44:27.6825985Z
expected results
2020-04-21 17:00:00
you can use bin() to round down to the hour, and if you still need to remove the datetime parts lower than seconds, you can use substring() (or format_datetime()). e.g.:
print d = datetime(2020-04-21T17:44:27.6825985Z)
| extend h = bin(d, 1h)
| extend h2 = substring(h, 0, 19)
hmm, if you always just want the rest to be 0, can you just use string concatenation?
let d = datetime(2020-04-21T17:44:27.6825985Z);
print strcat(format_datetime(d, "yyyy-MM-dd HH"), ":00:00")
the above code will give you the result of
2020-04-21 17:00:00
I am trying to process website login session data by each user. I am reading an S3 session log file into an RDD. The data looks something like this.
----------------------------------------
User | Site | Session start | Session end
---------------------------------------
Joe |Waterloo| 9/21/19 3:04 AM |9/21/19 3:18 AM
Stacy|Kirkwood| 8/4/19 3:06 PM |8/4/19 3:54 PM
John |Waterloo| 9/21/19 8:48 AM |9/21/19 9:05 AM
Stacy|Kirkwood| 8/4/19 4:16 PM |8/4/19 5:41 PM
...
...
I want to find out how many users were logged in each second of the hour on a given day.
Example: I might be processing this data for 9/21/19 only. So, I would need to remove all other records and then SUM user sessions for each second of the hour for all 24 hours of 9/21/19. The output should be possibly 24 rows for all the hours of 9/21/19 and then counts for each second of the day(yikes, second by second data!).
Is this something possible to do in pyspark using either rdds or DF?
(Apologize for the tardiness in building the grid).
Thanks
my dataset
data=[['Joe','Waterloo','9/21/19 3:04 AM','9/21/19 3:18 AM'],['Stacy','Kirkwood','8/4/19 3:06 PM','8/4/19 3:54 PM'],['John','Waterloo','9/21/19 8:48 AM','9/21/19 9:05 AM'],
['Stacy','Kirkwood','9/21/19 4:06 PM', '9/21/19 4:54 PM'],
['Mo','Hashmi','9/21/19 1:06 PM', '9/21/19 5:54 PM'],
['Murti','Hash','9/21/19 1:00 PM', '9/21/19 3:00 PM'],
['Floo','Shmi','9/21/19 9:10 PM', '9/21/19 11:54 PM']]
cSchema = StructType([StructField("User", StringType())\
,StructField("Site", StringType())
, StructField("Sesh-Start", StringType())
, StructField("Sesh-End", StringType())])
df= spark.createDataFrame(data,schema=cSchema)
display(df)
parse timestamp
df1=df.withColumn("Start", F.from_unixtime(F.unix_timestamp("Sesh-Start",'MM/dd/yyyy hh:mm aa'),'20yy-MM-dd HH:mm:ss').cast("timestamp")).withColumn("End", F.from_unixtime(F.unix_timestamp("Sesh-End",'MM/dd/yyyy hh:mm aa'),'20yy-MM-dd HH:mm:ss').cast("timestamp")).drop("Sesh-Start","Sesh-End")
build and register udf, for multiple hours per person
def yo(a,b):
from datetime import datetime
d1 = datetime.strptime(str(a), '%Y-%m-%d %H:%M:%S')
d2 = datetime.strptime(str(b), '%Y-%m-%d %H:%M:%S')
y=[]
if d1.hour == d2.hour:
y.append(d1.hour)
else:
for i in range(d1.hour,d2.hour+1):
y.append(i)
return y
rng= udf(yo, ArrayType(IntegerType()))
explode list of hours into column
df2=df1.withColumn("new", rng(F.col("Start"),F.col("End"))).withColumn("new1",F.explode("new")).drop("new")
get seconds for each hour
df3=df2.withColumn("Seconds", when(F.hour("Start")==F.hour("End"), F.col("End").cast('long') - F.col("Start").cast('long'))
.when(F.hour("Start")==F.col("new1"), 3600-F.minute("Start")*60)
.when(F.hour("End")==F.col("new1"), F.minute("End")*60)
.otherwise(3600))
create temp view and query it
df3.createOrReplaceTempView("final")
display(spark.sql("Select new1, sum(Seconds) from final group by new1 order by new1"))
The above answer by Lennart could be more perfomant because he uses a join to get all the different hours, instead I use a UDF which could be slower. My code will work for any user who can be online for any amount of hours. My data used only the day required, so you could use day filter given above to limit your query to the day in question.. Final output
Try to check this:
Initiaize filter.
val filter = to_date("2019-09-21")
val startFilter = to_timestamp("2019-09-21 00:00:00.000")
val endFilter = to_timestamp("2019-09-21 23:59:59.999")
Generate range (0 .. 23).
hours = spark.range(24).collect()
Get actual user sessions that match the filter.
df = sessions.alias("s") \
.where(filter >= to_date(s.start) & filter <= to_date(s.end)) \
.select(s.user, \
when(s.start < startFilter, startFilter).otherwise(s.start).alias("start"), \
when(s.end > endFilter, endFilter).otherwise(s.end).alias("end"))
Combine match user sessions with range of hours.
df2 = df.join(hours, hours.id.between(hour(df.start), hour(df.end)), 'inner') \
.select(df.user, hours.id.alias("hour"), \
(when(hour(df.end) > hours.id, 360).otherwise(minute(df.end) * 60 + second(df.end)) - \
when(hour(df.start) < hours.id, 0).otherwise(minute(df.start) * 60 + second(df.start))).alias("seconds"))
Generate summary: calculate users count and sum of seconds for each hour of sessions.
df2.groupBy(df2.hour)\
.agg(count(df2.user).alias("user counts"), \
sum(dg2.seconds).alias("seconds")) \
.show()
Hope this helps.
I have users entering a date and a time zone (e.g. "America/Los Angeles") for that date and I'd like to convert that to UTC but to do that I need the utc offset for the time on that date.
I can easily convert a date to the offset for the time zone if I already know the UTC date but I need the other way around...
The utc offset can change depending on the date due to daylight saving so I need a way to enter a date and a timezone and get back the offset from UTC using that.
Knowing the most recent switch from PST to PDT On march 11 at 2AM I tried using
var tzOffset = moment.tz("3/11/2018 3:00 AM", "America/Los_Angeles").utcOffset();
document.write('utc offset is : ' + tzOffset + '<br/>') ;
but that gives 480 when the correct answer is 420
I can get the correct answer 420 if I use parseZone like so:
var tzOffset2 = moment.parseZone("3/11/2018 3:00 AM -07:00").utcOffset();
document.write('utc offset2 is : ' + tzOffset2 + '<br/>') ;
but that means I need to already know the -7 offset that I'm trying to find...
So how do I find the utcOffset for a specific date/time like "3/11/2018 3:00 AM" and timezone like "America/Los_Angeles"? Thanks
Your input is not in a ISO 8601 or RFC 2822 format recognized by moment(String), so you have to specify the format as second parameter using moment(String, String) (please note that, as docs states: The moment.tz constructor takes all the same arguments as the moment constructor, but uses the last argument as a time zone identifier.)
Your code could be like the following:
var tzOffset = moment.tz("3/11/2018 3:00 AM", "D/M/YYYY h:mm A", "America/Los_Angeles").utcOffset();
document.write('utc offset is : ' + tzOffset + '<br/>') ;
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.22.1/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment-timezone/0.5.14/moment-timezone-with-data-2012-2022.min.js"></script>
I have these lines
SetLocale(3081)
Response.Write "<p>TEST: " & Date() & " | " & isDate("3/22/2014") & " --> " & GetLocale() & "</p>"
which outputs
TEST: 3/07/2014 | True --> 3081
now correct me if i'm wrong but isn't there only 12 months in a year?, according to IsDate the date i've passed, which should be wrong because i have put in 22 as the month, is valid despite the local settings saying otherwise.
i want to validate the date to be the correct format to insert into the database and if it isn't give a more friendly error, "3/22/2014" will output "Error converting data type varchar to date." when i try to inster it into the database because it's getting by the IsDate check
What have i done wrong here?
Yes, this is indeed a valid date.
Why? Because VBScript is smart/generous/stupid (choose your favorite) enough to treat numbers as dates. And "3/22/2014" can be parsed as formula: 3 / 22 / 2014 = 6.770786313983931e-5
Now take this number and convert to date:
Dim myNumber, myDate
myNumber = 6.770786313983931e-5
myDate = CDate(myNumber)
The variable myDate will be perfectly valid date, which is December 30th 1899, 00:00:06
So bottom line: the value is a date, just not what you expect. You've done nothing wrong, but to really check if a string is valid date you will have to check it yourself, no out of the box methods.