I'm trying to get acquainted with weatherData in R.
Having downloaded a set of temperature data I've then exported it to CSV.
Opening the CSV in Libre Calc shows the date and time for each temperature reading as a string of ten digits. In spite of some Googling I have not found a way of successfully converting the string into the format in which it appears in R.
For example: 1357084200 I believe should translate to 2013-01-01 23:50:00
Any help in getting the correct date in the same date format to appear in Calc via the CSV greatly appreciated.
Here is the direct way:
as.POSIXct(1357084200, origin="1970-01-01", tz="GMT")
#[1] "2013-01-01 23:50:00 GMT"
If it's really a character:
as.POSIXct(as.numeric("1357084200"), origin="1970-01-01", tz="GMT")
I'm not aware of a direct way of doing this, but I believe I've figured out a workaround.
For starters your example is correct. The long number (timestamp) is the number of seconds passed since 1970-01-01 00:00:00. Knowing this you can actually calculate the exact date and time from the timestamp. It's a bit complicated due to needing to take into account the leap years.
What comes in handy is the ability to supply an arbitrary number of days/months/years to LibreOffice function DATE. So in essence you can find out the number of days represented in timestamp by dividing it by 60*60*24 (number of seconds in a minute, number of minutes in an hour, number of hours in a day). And then supply that number to the date function.
timestamp = 1357084200
days = timestamp / FLOOR(timestamp / (60*60*24); 1) // comes out at 15706
actualdate = DATE(1970; 1; 1 + days) // comes out at 2013-01-01
seconds = timestamp - days * 60 * 60 * 24 // comes out at 85800
actualtime = TIME(0; 0; seconds) // comes out at 23:50:00
Then you can concatenate these or whatever else you want to do.
Related
I have decimal hours in format 245.85 equalling to 245:51:00 in [hh]:mm:ss format.
I want to transform the decimal hours to hh:mm format, but how do I do it?
the original calculation that renders 245.85 is:
library(lubridate)
time_length(hm("7 27")*33,unit = "hours")
what I want is 245:51 or 245:51:00
If I use as.period I get days too - like in:
as.period(dhours(time_length(hm("7 27")*33,"hours")))
[1] "10d 5H 51M 0S"
and for background - my aim is to multiply hours and minutes (e.g. 7:27) by an arbitrary integer (e.g. 33) and get result back in hh:mm format - avoiding days (as in as.period example above). Say if a piece of work takes 7 hours and 27 minutes and we give me 33 pieces of such work to do per year, it should take me about this many work hours (and minutes) to do.
If it's really only the H:M:S format that gives you trouble, try
library(hms)
hms(hours=245.85)
which yields 245:51:00
SAS documentation states the following for data and datetime values:
SAS time value: is a value representing the number of seconds since midnight of the current day. SAS time values are between 0 and 86400.
SAS datetime value: is a value representing the number of seconds between January 1, 1960 and an hour/minute/second within a specified date.
I'm willing to convert the following date and hour values with R, I have a big doubt for the hour (datetime) conversion, which one of the "HH:MM:SS" values within R_hour1 and R_hour2 is correct ?
I have to separate columns, SAS date = 20562 and SAS hour = 143659, in my table
R: R_date <- as.Date(as.integer(20562), origin="1960-01-01"); R_date
[1] "2016-04-18"
R: R_hour1 <- as.POSIXct(143659, origin = R_date); R_hour1
[1] "2016-04-19 17:54:19 CEST"
R: R_hour2 <- as.POSIXct(143659, origin = "1960-01-01"); R_hour2
[1] "1960-01-02 16:54:19 CET"
Similar to R, SAS Date and DateTime values can have whatever origin you wish them to. The default formats have a default (1/1/1960 for both), but you can use the datetime field to mean any origin you wish, and it will generally still work perfectly well with any of the datetime functions (though it will not display properly unless you write a custom format). It is very possible to have a different origin, as you show above with R_hour1.
As such, you would have to ask the person who generated the data what the meaning of the field is and what its origin should be.
I imported date variables as strings from SQL (date1) into Stata and then created a new date variable (date2) like this:
gen double date2 = clock(date1, "YMDhms")
format date2 %tc
However, now I want to calculate the number of days between two dates (date3-date2), formatted as above, but I can't seem to do it.
I don't care about the hms, so perhaps I should strip that out first? And then deconstruct the date into YYYY MM DD as separate variables? Nothing I seems to do is working right now.
It sounds like by dates you actually mean timestamp (aka datetime) variables. In my experience, there's usually no need to cast dates/timestamps as strings since ODBC and Stata will handle the conversion to SIF td/tc formats nicely.
But perhaps you exported to a text file and then read in the data instead. Here are a couple solutions.
tc timestamps are in milliseconds since 01jan1960 00:00:00.000, assuming 1000*60*60*24=86,400 seconds/day (that is, ignoring leap seconds). This means that you need to divide your difference by that number to get elapsed days.
For example, 2016 was a leap year:
. display (tc(01jan2017 00:00:00) - tc(01jan2016 00:00:00))/(1000*60*60*24)
366
You can also use the dofc() function to make dates out of timestamps and omit the division:
. display (dofc(tc(01jan2018 00:00:00)) - dofc(tc(01jan2016 00:00:00)))
731
2017 is not a leap year, so 366 + 365 = 731 days.
You can use generate with all these functions, though display is often easier for debugging initial attempts.
I have some numbers that represent dates in milliseconds since epoch, 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970
1365368400000,
1365973200000,
1366578000000
I'm converting them to date format:
as.Date(as.POSIXct(my_dates/1000, origin="1970-01-01", tz="GMT"))
answer:
[1] "2013-04-07" "2013-04-14" "2013-04-21"
How to convert these strings back to milliseconds since epoch?
Here are your javascript dates
x <- c(1365368400000, 1365973200000, 1366578000000)
You can convert them to R dates more easily by dividing by the number of milliseconds in one day.
y <- as.Date(x / 86400000, origin = "1970-01-01")
To convert back, just convert to numeric and multiply by this number.
z <- as.numeric(y) * 86400000
Finally, check that the answer is what you started with.
stopifnot(identical(x, z))
As per the comment, you may sometimes get numerical rounding errors leading to x and z not being identical. For numerical comparisons like this, use:
library(testthat)
expect_equal(x, z)
I will provide a simple framework to handle various kinds of dates encoding and how to go back an forth. Using the R package ‘lubridate’ this is made very easy using the period and interval classes.
When dealing with days, it can be easy as one can use the as.numeric(Date) to get the number of dates since the epoch. To get any unit of time smaller than a day one can convert using the various factors (24 for hours, 24 * 60 for minutes, etc.) However, for months, the math can get a bit more tricky and thus I prefer in many instances to use this method.
library(lubridate)
as.period(interval(start = epoch, end = Date), unit = 'month')#month
This can be used for year, month, day, hour, minute, and smaller units through apply the factors.
Going the other way such as being given months since epoch:
library(lubridate)
epoch %m+% as.period(Date, unit = 'months')
I presented this approach with months as it might be the more complicated one. An advantage to using period and intervals is that it can be adjusted to any epoch and unit very easily.
I am trying to extract data from my workbook because I made the workbook wrong but I want to keep the data. So I have to go through each worksheet and extract the specific bits of data I need. (Repetitive task = vba code to the rescue)
I'm done, but I have one nasty little problem.
I am using ws.Range("C13") to extract each cell that I need. It's a cell that has time values like 10:00 or 22:13. When I debug.print that value however I get a decimal. Why? How do I prevent it from converting my "10:30" to "0.4375"
From what I can tell, the decimal isn't even related to the time value.
WRONG it is simply the time value as a representation of a 24 hour period. So it is related. But still.
ws.Range("C13") is the equivalent of ws.Range("C13").Value which returns the underlying data from that cell.
If instead you use ws.Range("C13").Text, you will get the displayed text from the cell. If you have set a specific date/time format for that cell then calling ws.Range("C13").Text should return text in that specific format
The time values are correct; it's the way Excel stores date and time values internally.
Dates are stored as floating point values, where the whole portion represents the number of days since 12/30/1899 00:00.00 (date 0.0) and the decimal part represents the fractional part of a single day (eg., 408875.5 represents 11/28/2011 12:00:00 PM).
Here are some conversion factors you might find useful:
Decimal value Time Value Calculation
============= ========== ===========
0.00094444444 12:01:00 AM 1.0 / 24 / 60 (1 day/24 hours/60 minutes per hour)
0.01041666666 12:15:00 AM 1.0 / 24 / 60 * 15
0.02083333333 12:30:00 AM 1.0 / 24 / 60 * 30
0.04166666666 01:00:00 AM 1.0 / 24
hours = MOD(INT(0.tttttttt*24), 24)
minutes = MOD(INT(0.tttttttt*24*60), 60)
seconds = MOD(INT(0.tttttttt*24*60*60), 60)
milliseconds = MOD(INT(0.tttttttt*24*60*60*1000), 1000)