Related
I am attempting to do something seemingly trivial, but running into all sorts of problems.
I would like to plot certain basic information on a label every time there is an update to the current price--regardless of the timeframe of my chart.
I am able to accurately display volume and price information, however displaying the time has been a challenge.
My first attempt was to use the following code:
if (barstate.islast)
label.set_text(
id=myLabel,
text="\nTime: " + tostring(hour) + ":" + tostring(minute) + ":" + tostring(minute)
)
I quickly learned that, even though my chart is set to the timezone for New York (i.e., UTC-4), calling tostring(hour) displays the hour of UTC.
Figuring out how to specify that I want it displayed time to correspond to my chart's timezone has been the first major hurdle, and I have tangled endlessly with timestamp() and syminfo.timezone to no avail.
My second major problem is that tostring(second) does not properly display the seconds, even for UTC time.
While working on a 1m chart, I thought I managed to solve this by implementing
tostring((timenow-time)/1000)
However, the seconds do not display properly on different time frames.
This is all in addition to the fact that charts from different exchanges in different time zones will all display time "incorrectly" with respect to UTC time.
It must to be the case that I am missing something fairly basic, since time is such crucial data, but I just can't determine the proper syntax.
Thanks in advance for any assistance.
A few different issues are at play here:
Pine scripts have no visibility on the chart's timezone you may have selected manually. That only affects the display of the chart.
The minute variable returns the minute at the beginning of the bar, so will not change on script iterations in the realtime bar, until a new bar begins. To get the current minute you will need to use an overloaded version of minute where you can specify a timestamp in ms. The timenow built-in variable returns the timestamp for the time of a particular script iteration (that is true in the realtime bar; when the script is running on historical bars, timenow is only updated every second during the script's execution). So you need to use minute(timenow).
If you want minute() to return a time in another timezone than the exchange's, you can use a second parameter to specify a timezone, which is what we do in the second example here. In our example you can change the timezone through the script's "Settings/Inputs". Used with the timezone, minute() will look something like:
minute(timenow, "GMT-4").
//#version=4
study("", "Time", true)
i_timeZone = input("GMT-4")
f_print(_txt) => var _lbl = label.new(bar_index, highest(10)[1], _txt, xloc.bar_index, yloc.price, #00000000, label.style_none, color.gray, size.large, text.align_left), label.set_xy(_lbl, bar_index, highest(10)[1]), label.set_text(_lbl, _txt)
f_print(tostring(hour(timenow), "00:") + tostring(minute(timenow), "00:") + tostring(second(timenow), "00") + " (Exchange)\n")
f_print(tostring(hour(timenow, i_timeZone), "00:") + tostring(minute(timenow, i_timeZone), "00:") + tostring(second(timenow, i_timeZone), "00") + " (Input: " + i_timeZone + ")")
Trying to convert time as a string to a time variable.
Use Date/Dates/Convert String to Date...... for format I use %H:%M:%S....
Here is the syntax from the GUI
[Convert String Variables to Date]
BSkystrptime (varNames = c('Time'),dateFormat = "%H:%M:%S",prefixOrSuffix = "prefix",prefixOrSuffixValue = "Con_",data = "Dataset2")
BSkyLoadRefreshDataframe(dframe=Dataset2,load.dataframe=TRUE)
A screen shot of result is attached....
Compare variables Time [string] to Con_Time [date/time]
The hours are 2 hours out [wrong!] - the Minutes and Seconds are correct.
What am I doing wrong here?
Screen Shot
I believe you are running into a known issue with a prior release of BlueSky Statistics. This issue is fixed with the current stable release available on the download page.
The reason for this was although the time is converted correctly into the local time zone, BlueSky Statistics was reading the time zone in the local time zone and converting it to UTC.
You are probably +2 hours ahead of UTC, so you are seeing the time move 2 hrs back. Give us a couple of days to post a patch.
You can also confirm this by writing and executing the following syntax in the syntax window
Dataset2$Con_Time
Short question: I need momentjs to humanize 60 minutes into 1 hour instead of an hour. Can't figure out how to get it to work.
Long question:
Just started using momentjs, works great. We are using it to display how often a dashboard is updated.
The timers are set as a integer in minutes. We are using the humanize moment option to display 30 as 30 minutes and 360 as 6 hours etc.
This works great but not in 2 cases. 60 gets humanized to an hour. We need it to be 1 hour. And 1440 is displayed as a day, instead of 1 day.
We need this change because the column is answering the question "How often does your metric update?"
The answer is "every 1 hour". "every an hour" doesn't quite work.
I read through the docs and googled, but couldn't find a way to customize just a few humanized display formats.
We are already setting true as the second parameter to get just the value without the suffix from this question and answer - How to Customize Humanized Moment js Date Result
But the value comes back as 'an hour' instead of '1 hour'.
You can use updateLocale method documented in the Customize -> Relative time section of the docs. This will affect output of from, fromNow, to, toNow and humanize.
In your case you can simply update h and d keys of the relativeTime object.
Here a working sample:
moment.updateLocale('en', {
relativeTime : {
h: "1 hour",
d: "1 day",
}
});
var d1 = moment.duration(60, 'm');
var d2 = moment.duration(1440, 'm');
console.log(d1.humanize());
console.log(d2.humanize());
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.14.1/moment.min.js"></script>
You can find further examples of relative time customization here and here.
you can use the methods like minutes(), months(), years() available on the duration object and then you can create your own string. the humanize() method gives an approximate value against the duration and not the exact duration.
I brought in a time series into R using the parse_date_time function in the library(lubridate) and I brought it in as EST.
streamflowDateTime<-parse_date_time(streamflowDateTime,"%m%d%Y %H%M",tz="EST")
However, the data experiences DST on 04-03-2005 01:45 and the next time step is 03:00. I want to convert this occurrence and all the time stamps that follow to EST by subtracting an hour so that it is continuous. It would be preferred if there was an automated way to do it where the program figures out where DST starts taking place and moves back an hour itself, since DST does not take effect every year on the same day at the same time.
Here's a sample of the data
structure(c(1112475600, 1112476500, 1112477400, 1112478300, 1112479200,
1112480100, 1112481000, 1112481900, 1112482800, 1112483700, 1112484600,
1112485500, 1112486400, 1112487300, 1112488200, 1112489100, 1112490000,
1112490900, 1112491800, 1112492700, 1112493600, 1112494500, 1112495400,
1112496300, 1112497200, 1112498100, 1112499000, 1112499900, 1112500800,
1112501700, 1112502600, 1112503500, 1112504400, 1112505300, 1112506200,
1112507100, 1112508000, 1112508900, 1112509800, 1112510700, 1112515200,
1112516100, 1112517000, 1112517900, 1112518800, 1112519700, 1112520600,
1112521500, 1112522400, 1112523300, 1112524200, 1112525100, 1112526000,
1112526900, 1112527800, 1112528700, 1112529600, 1112530500, 1112531400,
1112532300, 1112533200, 1112534100, 1112535000, 1112535900, 1112536800,
1112537700, 1112538600, 1112539500, 1112540400, 1112541300, 1112542200,
1112543100, 1112544000, 1112544900, 1112545800, 1112546700, 1112547600,
1112548500, 1112549400, 1112550300, 1112551200, 1112552100, 1112553000,
1112553900, 1112554800, 1112555700, 1112556600, 1112557500, 1112558400,
1112559300, 1112560200, 1112561100, 1112562000, 1112562900, 1112563800,
1112564700, 1112565600, 1112566500, 1112567400, 1112568300, 1112569200
), class = c("POSIXct", "POSIXt"), tzone = "EST")
Edits:
streamflowDateTime[8840:length(streamflowDateTime)] <- streamflowDateTime[8840:length(streamflowDateTime)]-hours(1)
In the full entire data set, the occurence happens at location 8840, which I know manually, I want the code to automatically find the position where the time difference between two consecutive time stamps is not 15 minutes and replace the '8840' in code with that automated value. for loops are too slow
You can probably just supply the full IANA time zone ID America/New_York instead of the time zone abbreviation.
parse_date_time(streamflowDateTime,"%m%d%Y %H%M",tz="America/New_York")
Using America/New_York will properly account for both EST and EDT, including the correct transitions between them.
This seems to be supported, as seen in this blog post - at least on systems that provide IANA/Olson time zones, such as Linux or Mac.
According to the docs:
... R does not come with a predefined list zone names, but relies on the user's OS to interpret time zone names. As a result, some names will be recognized on some computers but not others. Most computers, however, will recognize names in the timezone data base originally compiled by Arthur Olson. These names normally take the form "Country/City." ...
Since Windows uses its own set of time zones, you will probably not be able to use IANA/Olson identifiers. However:
The equivalent Windows time zone id would be "Eastern Standard Time". (Despite the name, this covers both EST and EDT). I am uncertain if R supports these or not.
The fully qualified POSIX time zone for the current rule would be "EST5EDT,M3.2.0,M11.1.0". This should work on all OS's - however it only represents the US Eastern Time Zone since the 2007 change.
From 1987-2006 the rule would have been "EST5EDT,M4.1.0,M10.5.0". Use the appropriate rule for the values you're working in. If you have dates that span these periods, you'll need to split them up and process them separately, or if possible, write a function to use the correct rule for the data.
See also, the timezone tag wiki.
Unless I use the .toLocaleDateString() method, the function below returns a date that is one day less than it should be. Am I doing something wrong or is this a defect?
function myDate() {
var sDate = '05/10/2012';
var parts = sDate.split('/');
var d = new Date( parts[2], parts[1]-1, parts[0]);
Logger.log(d);
Logger.log(d.toLocaleDateString());
};
The Logger returns:
Thu Oct 04 16:00:00 PDT 2012
05 October 2012 00:00:00 BST
I'm in the UK, hence the UK date format. I've checked that my project properties have the time zone set to "(GMT+00:00) London" so why does the Logger show the first date in PDT? Is that the reason for the wrong date? I've reproduced the problem in a stand-alone project. Here's a link to it.
I wanted to convert the string variable into a date object in order to do some date math so having to convert back to a string with .toLocaleDateString() method isn't helpful.
I've checked for consistence, thinking perhaps I could work around it, by testing with other dates. Bizarrely, if I change the value of sDate to anything between 01/01/2012 & 04/03/2012 it returns accurately. From 05/03/2012 onwards it drops a day. With 2013 dates, it returns correctly until 29/04/2013 when it starts dropping a day again. So it's not consistent. It seems similar to the problem reported here.
I found that dates read by the getValues() function on a range that were between the old DST rules and the new DST rules were decremented by 1 hour. So, the implicit time for the date "10/30/2012" is 00:00:00. But, when read in, it was decrementing by an hour, landing it in the previous day, i.e. 10/29/2012 23:00:00.
Any date between these two timeframes, not inclusive of the start day but inclusive of the end day, will exhibit this behavior currently, at least as read by the getValues() function:
Between the last Sunday in October and the first Sunday in November
Between the second Sunday in March and first Sunday in April.
I ended up writing code that would dynamically calculate these dates for the current year and if the date landed in the target range, I would simply increment it by an hour. This may be the the long way to fix it, but it works.
Here is my code:
/*
The old rules for DST stated that the time change occurred on the last Sunday in October,
which would be 10/28/2012. So, when you type in 10/29/2012, the timestamp associated with the date (00:00:00) is being decremented by an hour (23:00:00),
which drops it into the previous day. The new rules for DST states that DST ends on the 1st Sunday in November, which is 11/04/2012. Also, the DST rules
for springtime are also an impacted range of dates that exhibit this behavior, between the second sunday in March and the first sunday in April.
Note: Running this function from the script editor does not produce the strange DST behavior. It seems to be an issue with the getValues() function on a Range object.
*/
function fixDateDSTProblem(lstrDate){
var d = new Date(lstrDate);
//Example ranges affected
//10/29/2012 - 11/04/2012
//03/11/2013 - 04/07/2013
if(isNaN(d.getMonth())){
//return what was passed in if it's not a date or null.
return lstrDate;
}
else if(isAffectedDate(d)){
//increment by one hour
return new Date(d.getTime() + (60 * 60 * 1000));
}
else{
//date is not affected, simply return it as it was passed.
return lstrDate;
}
}
//Check to see if a date is within a beginning and end date
function dateWithin(beginDate,endDate,checkDate) {
var b,e,c;
b = Date.parse(beginDate);
e = Date.parse(endDate);
c = Date.parse(checkDate);
//slightly modified this for the date checking since beginning date is not inclusive, so needs to be > instead of >=
if((c <= e && c > b)) {
return true;
}
return false;
}
function isAffectedDate(targetDate){
var yearNum = targetDate.getFullYear();
//Get the last Sunday in October
var lastSunOctDateStr = getLastOccurrenceDate(0, 10, yearNum);
//Get the first Sunday in November
var firstSunNovDateStr = getOccurrenceDate(1, 0, 11, yearNum);
//Get the second Sunday in March
var secondSunMarDateStr = getOccurrenceDate(2, 0, 3, yearNum);
//Get the first Sunday in April
var firstSunAprDateStr = getOccurrenceDate(1, 0, 4, yearNum);
//if the date is between the last sunday in october and the first sunday in november
// or if the date is between the second sunday in march and the first sunday and april, fix it up!
if(dateWithin(lastSunOctDateStr, firstSunNovDateStr, targetDate) ||
dateWithin(secondSunMarDateStr, firstSunAprDateStr, targetDate)){
return true;
}
return false;
}
function getOccurrenceDate(numOccurrence, dayIndex, monthCalendar, yearNum){
//"Get date of first occurrence of Monday in June 2013"
//"Get date of the second occurrence of Sunday in April 2013"
//dayIndex: Sunday = 0, Saturday = 6
var monthIndex = monthCalendar - 1;
var numFirstXDay = null;
var firstDay = new Date(monthCalendar+"/01/"+yearNum);
var numDayOfWeek = firstDay.getDay();
if(numDayOfWeek == dayIndex){
numFirstXDay = 1;
}
else if(numDayOfWeek > dayIndex){
numFirstXDay = 1+(6-numDayOfWeek)+1+dayIndex+(7*(numOccurrence-1));
}
else if(numDayOfWeek < dayIndex){
numFirstXDay = 1+(dayIndex - numDayOfWeek)+(7*(numOccurrence-1));
}
return monthCalendar+"/"+numFirstXDay+"/"+yearNum;
}
function getLastOccurrenceDate(dayIndex, monthCalendar, yearNum){
//Example: "Get date of last occurrence of Monday in June 2013"
var monthIndex = monthCalendar - 1;
var numLastXDay = null;
//TODO: Handle Leap Year!
var monthMaxDaysArray = {
'1':'31',
'2':'28',
'3':'31',
'4':'30',
'5':'31',
'6':'30',
'7':'31',
'8':'31',
'9':'30',
'10':'31',
'11':'30',
'12':'31'}
var lastDay = new Date(monthCalendar + "/"+monthMaxDaysArray[monthCalendar]+"/" + yearNum);
var numDayOfWeek = lastDay.getDay();
if(numDayOfWeek == dayIndex){
numLastXDay = 31;
}
else if(numDayOfWeek > dayIndex){
numLastXDay = monthMaxDaysArray[monthCalendar] - (numDayOfWeek - dayIndex);
}
else if(numDayOfWeek < dayIndex){
numLastXDay = (monthMaxDaysArray[monthCalendar] - numDayOfWeek) - (6 - (dayIndex-1));
}
return monthCalendar + "/" + numLastXDay + "/" + yearNum;
}
The logger always logs dates in PDT. It's the same instance of time, just represented a different way. You are seeing that + differences in daylight savings time.
Even if Corey's answer doesn't need any confirmation (he knows what he is talking about ;) let me add some practical details ...
every javascript manipulation you do on date object (getDay, getTime, getFulYear...) will return correct values even if the logger value you see seems wrong so you don't need to convert to string and back to numbers if you need to manipulate dates
If these dates are shown in a spreadsheet then the spreadsheet locale settings will be used to show the desired values.
Sometimes you will see the values in other format in the logger... don't ask when or why but it happens to me sometimes (I'm in GMT+1 Belgium)
The date shift you notice in october and april are indeed happening when daylight savings starts and stops and this can become tricky when you need to define a date in a calendar event in one period from another (for example today I create an event in december there will be 1 hour difference) so you need to use the right GMT+0X00 value when you read a date object in a user interface using utilities.formatDate
If for some reason you show a list of dates starting before daylight savings switch day and ending after, and you use utilities.formatDate, it is better to user a variable to play with 'GMT+0X00' dynamically : I get it simply using something like this :
* EDIT* since a few weeks the syntax has changed and this string has to be formated differently, so I updated this code accordingly. (see issue 2204)
`var FUS1=new Date().toString().substr(25,6)+":00";// FUS1 gets the GMT+02:00 or GMT+01:00 string`
Hoping it's getting clear enough
Sorry to join the party a few months late...
The problem with these answers is that none of them actually make it clear that this is a BUG. Or, to be more precise, this is something that Google have decided is "correct" behaviour (see Issue 1271, for example, closed with no action), when it quite clearly is completely the wrong thing to do.
Consider this. I want to put someone's birthdate in a spreadsheet, and I have a large list of them. I then want to read out the value and put it in a form:
function getAge(ss, index) {
var ages = ss.getRangeByName("birthDates").getValues();
return ages[index];
}
...
app.createLabel(getAge(ss, index));
The form now contains the wrong birthdate for everyone born between April and October. Google has decided that they were actually born one day earlier. Or try to read out the ages, and put them in another spreadsheet, with no processing, where both spreadsheets have the same time zone setting. Same story, and it leaves me looking really stupid when people mail to tell me that I've got their birthdates wrong.
This is how it should be done: if someone manually specifies a date, for example "06/06/1997", then reads that date with getValue, they expect to get "06/06/1997" back, not the day before. They really don't expect you to try to second guess them: "Ok, this date happens to be during DST, so what they really expect to read back from getValue is something corrected to non-DST, so why don't we subtract an hour and give them the day before?"
It's crazy. If Excel did that, it would be a laughing stock. Fix it, please.
EDIT
Serge, that's very interesting, but your spreadsheet shows a work-around which extracts a TZ, and then displays according to that TZ. But that just avoids the issue that this is wrong to start with. Consider this. Spreadsheets almost invariably contain anniversaries of fixed dates - someone's birthday, the date of a battle, the time/date of an appointment, whatever. These are independent of TZ. If your birthday is 6th July 1960 in Belgium, then it's also 6th July if you happen to live in California. If you were born at 00:30AM, your passport still shows 6th July, even though that time was actually 5th July in CET and PDT. For an anniversary, or an appointment, no-one gives a damn about timezones. In a spreadsheet, only a tiny proportion of users care about TZ's. I can't immediately think of a single use-case where it actually matters.
Lotus 1-2-3 got this right. Excel got this right. A date is an integer serial number, with no time zone information. Copy a date from one spreadsheet to another, it's always right. Get a date from a spreadsheet using VB/whatever, put it in another spreadsheet, it's always right (I think). Much as I hate to say it, Excel defines spreadsheets, and they do it Good. Apart from VB.
In GAS, if I copy a date using getValues, it's wrong. Google have over-smarted this. Ok, there is a complex work-around that involves finding what TZ your spreadsheet was created in, assuming that the date you entered was actually an epoch date relative to UTC, and displaying it according to the spreadsheet's current TZ setting (at least, I think that's what you're doing). But so what? And, does it always work? What if you mail the spreadsheet to someone who lives in PDT? I'm sure you can get it to work, but that misses the point - this is just a work-around, which I think might be fragile (note that there are 3 TZ settings: the Google account setting, the sheet setting, and the script setting, and you have to take account of all of them - see ahab's post here). The fundamental issue is that Google spreadsheets store dates relative to a specific TZ (is the underlying format ms relative to Unix epoch??). This was just wrong, and is now unfixable, and is only one part of a large can of worms to do with date handling.
As a short-term fix, I can probably add 12 hours to all dates in a spreadsheet. As a longer-term fix, I should probably handle all dates as text strings and do it myself in JavaScript. I've already had to replace all elapsed times with "integral" number of milliseconds to handle the issues to do with lost milliseconds in GAS dates. Go on, convince me I'm wrong.
EDIT by Serge :
Hi, I took the liberty to answer in your edit for ease of reading (sorry about that).
I do understand your point of view but I don't share it.
The UI example you refer to needs indeed a workaround because this is pure javascript, not spreadsheet date ... That said, Excel spreadsheets and Google spreadsheets work the same way (next day is day+1, time = decimals and ref date 0 = 12/31/1899)) as long as you stay in the spreadsheet universe, the problem comes when you quit the spreadsheet and go to Javascript (ref date = 01/01/1970, count in milliSecs)...
If they had chosen not to care about TZ and keep dates 'static' across any time zone it would have had a lot of other issues : for example what if you want to setup an appointment on skype between me in Belgium and President Obama in Washington ? If we decide to have that conversation on monday at 8 AM will it be 8 AM in Belgium or in Washington ?
With the Google solution he will just have to wake up a bit early but we will both be there since the date and time would show the same 'moment', with the way you suggest there won't be any chance we talk to each other (how sad that would be^^) .
There are a lot more situation where it is obviously a better choice : communication with Calendar service is also a good example.
The static excel solution works nicely because such a document isn't shared the same way and doesn't convert dates to Javascript date Objects. An Excel SS is related to the computer's user, a Google SS keeps this relation to the user's computer using the Time Zone setting because it is (by nature) not related to user's computer.
I have little hope to convince you (;-) but I hope this will at least make things a bit clearer... (and anyway I'm not a Google engineer and have no way to influence them in any way ^^).
one more EDIT...
Here is a small test sheet with TZ settings set to GMT-8 that gets the value of the other test sheet (set to GMT+2 Belgian winter time) and shows it the right way by using this simple script :
function timeCorrection() {
var thisSheet = SpreadsheetApp.getActiveSheet();
var otherSS = SpreadsheetApp.openById('0AnqSFd3iikE3dENrc2h1M0ktQTlSNWpNUi1TS0lrNlE');
var otherSheet = otherSS.getSheetByName('Sheet1');
var otherValue = otherSheet.getRange(1,1).getValue();
var tz = otherSS.getSpreadsheetTimeZone();
var thisSheetValue = new Date(Utilities.formatDate(otherValue,tz,'MM/dd/yyyy HH:mm:ss'));
thisSheet.getRange(1,1).setValue(thisSheetValue);
}
As simple as that ;-) , settings of the 'foreign sheet' are set to the same value as its script settings which is default for a newly created sheet created in Vancouver.
This is not a bug...
May I offer another solution to the problem that is less programmatic? I isolated this issue and spend a day trying to figure out why my script was pulling dates with a previous day and 23:00PM.
Then, I realized that Spreadsheet is not a comma separated database but has its auto-formatting. So, the problem is solved when in Spreadsheet I selected my column containing dates and from menu chose Format>Numbers>Plain Text. Now my script runs perfect without tons of way around. Applying this to an entire column affects even a newly added cells. So its a sweet solution.
Since this is stackoverflow, here is the java solution (I check it works):
sheet.getRange(1,n).setNumberFormat('#STRING#');
I took it from christian.simms, from here:
Format a Google Sheets cell in plaintext via Apps Script
The main problem is incorrect time zone for a date object.
So I not found way to get a string from a cell if it has date type. The cell will return representation based on the time zone but it is not the same as on local computer, so you can get date ± one day.
You can get the time zone for the current account:
var userTimeZone = CalendarApp.getDefaultCalendar().getTimeZone();
Warning! This is not time zone of local computer, so this is another problem.
So you can get correct string of the date:
var date1 = SpreadsheetApp.getActiveSheet().getRange(1, 1).getValue();
var str = date1.toLocaleString("en-US", {timeZone: userTimeZone });