SAS web scraping - web-scraping

I would like to get latest HIBOR rate from the website of [HKAB][1].
Here is the following code i tried: While it give error in filename statement
data parm;
format yyyymmdd $8.;
yyyymmdd = put(date(), yymmddn8.);
yyyy = put(substr(yyyymmdd, 1, 4), $4.);
mm = put(substr(yyyymmdd, 5, 2), $2.);
dd = put(substr(yyyymmdd, 7, 2), $2.);
*url = 'http://www.hkab.org.hk/hibor/listRates.do?lang=en&Submit=Search&year='||yyyy||'&month='||mm||'&day='||dd;;
format url $1024.;
url = 'http://www.hkab.org.hk/hibor/listRates.do?lang=en&Submit=Search&year=' || yyyy || '&month=' || mm || '&day=' || dd;
url2 = 'curl ' || url;
call symput('url',url);
call symput('url',url);
call symput('url',url);
call symput('url',url);
call symput('url2',url2);
put _all_;
run;
%put &url2.;
filename hiborpage url &url.;
[1]: http://www.hkab.org.hk/hibor/listRates.do?lang=en&Submit=Search&year=2015&month=1&day=26

Your fileref exceeds 8 characters. Change hiborpage to something shorter, like hibor. Also, as Reeza mentioned, put &url in double quotes. Macro variables won't resolve in single quotes. That should fix the errors.
There are also ampersands in the url. Ordinarily, you'd just put a string like this in single quotes, but because it's in a macro variable, you need the double quotes. The problem is that SAS tries to resolve what it thinks are macro variables in the url and gives you warnings when it fails. To avoid these warnings, use the macro function %nrstr() to mask the ampersands in your macro variable, e.g.:
%let url3 = %nrstr(&url);
filename hibor url "&url3.";

Related

Workaround for case-sensitive input to dir

I am using Octave 5.1.0 on Windows 10 (x64). I am parsing a series of directories looking for an Excel spreadsheet in each directory with "logbook" in its filename. The problem is these files are created by hand and the filenaming isn't consistent: sometimes it's "LogBook", other times it's "logbook", etc...
It looks like the string passed as input to the dir function is case-sensitive so if I don't have the correct case, dir returns an empty struct. Currently, I am using the following workaround, but I wondered if there was a better way of doing this (for a start I haven't captured all possible upper/lower case combinations):
logbook = dir('*LogBook.xls*');
if isempty(logbook)
logbook = dir('*logbook.xls*');
if isempty(logbook)
logbook = dir('*Logbook.xls*');
if isempty(logbook)
logbook = dir('*logBook.xls*');
if isempty(logbook)
error(['Could not find logbook spreadsheet in ' dir_name '.'])
end
end
end
end
You need to get the list of filenames (either via readdir, dir, ls), and then search for the string in that list. If you use readdir, it can be done like this:
[files, err, msg] = readdir ('.'); # read current directory
if (err != 0)
error ("failed to readdir (error code %d): %s", msg);
endif
logbook_indices = find (cellfun (#any, regexpi (files, 'logbook'));
logbook_filenames = files(logbook_indices);
A much less standard approach could be:
glob ('*[lL][oO][gG][bB][oO][kK]*')

moment.js will not parse UK format date even when setting the locale

Quite simply, this is my code:
http://jsfiddle.net/NibblyPig/k9zb4ysp/
moment.locale('en-GB');
var d = moment('22/12/2019');
alert(d);
I would expect this to parse, however it says invalid date.
I have referenced moment.js and the locale/en-gb.js
I'm writing a global control so the date may come in in a variety of formats.
If I put in a variety of American dates they all work, for example 12/12/2019, 12/12/2019 23:04 etc.
However the locale command does not appear to do anything and I cannot get a single date to parse. What am I doing wrong?
You need to pass the format as the second argument for moment(), as discussed here:
moment.locale('en-GB');
var d = moment('22/12/2019', 'DD/MM/YYYY');
alert(d);
https://jsfiddle.net/a4gu6kfz/
From the docs:
If you know the format of an input string, you can use that to parse a
moment.
moment("12-25-1995", "MM-DD-YYYY");
I think that there is no need to write your own complex logic to parse your input, you can use moment(String, String) (or moment(String, String[], String, Boolean)), as suggested by Thales Minussi's answer.
moment(String) is the good choice only if your input is in ISO 8601 or RFC 2822 compliant form.
In your case, you can probably use Localized formats listed in the format section of the docs. If you have a list of possible formats, I think that the best choice is tho use moment(String, String[]).
Please note that, by default: Moment's parser is very forgiving, so using default Forgiving Mode will handle "any" character as separator.
Here a live sample:
moment.locale('en-GB');
['22/12/2019', '22/12/2019 15:00',
'22-12-2019', '22-12-2019 15:00',
'1-3-2019', '1-12-2019', '22-1-2019'
].forEach((elem) => {
var d = moment(elem, 'L LT');
console.log(d.format());
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/locale/en-gb.js"></script>
Still hoping there's a nice moment js way to do this but in the meantime I just bashed this together. Pretty nasty and it will probably go wrong in 80 years or so.
http://jsfiddle.net/NibblyPig/k9zb4ysp/22/
var a = "23/03/19 12:42:21.123";
var datePart = a.substring(0, a.indexOf(" "));
var timePart = a.substring(a.indexOf(" ") + 1);
var dateParts = datePart.split("/");
if (dateParts[0].length == 1) dateParts[0] = "0" + dateParts[0];
if (dateParts[1].length == 1) dateParts[1] = "0" + dateParts[1];
if (dateParts[2].length == 2) {
var threshold = parseInt(new Date().getFullYear().toString().substring(2)) + 10;
if (parseFloat(dateParts[2]) > threshold ) {
dateParts[2] = "19" + dateParts[2];
}
else
{
dateParts[2] = "20" + dateParts[2];
}
}
alert (parseFloat(dateParts[2] + dateParts[1] + dateParts[0] + timePart.replace(/:/g, "").replace(/\./g, "")));
This won't solve every usecase, but in your specific example if you want just a simple date (with no time component) auto-parsed in UK format you can just use the 'L' format string having set the locale to 'en-GB'
Your example with this change (your jsfiddle also)
moment.locale('en-GB');
// just pass 'L' i.e. local date format as a parsing format here
var d = moment('22/12/2019', 'L');
alert(d);
It's quite nice because you get the auto parsing of various formats you wanted for free. For instance this works just the same:
var d = moment('22-12-2019', 'L');
You can return a date using moment.js in a desired format -
return moment(aDateVar).format('DD/MM/YYYY');

Format zero currency value with {0:C} in VB.Net

I am trying to format a zero currency value as an empty string, so that when the currency value is 0.00 then an empty string gets displayed rather than $0.00.
This code is part of an ASP.Net app that will display currency value to end user.
I have used following code to achieve this goal.
Question : Is it possible to achieve this by just using {0:C} format string or another version of this format string instead of using if then else coding for this? If I use ###,###,###.## as the data format string then an empty string shows for zero currency value and also I get rid of the if then else coding but for non-zero values no currency symbol shows.
If Double.Parse(Decimal.Parse(CDec(currencyValue))) = 0 Then
charValue = Nothing
Else
charValue = String.Format("{0:C}", CDec(currencyValue))
End If
UPDATE
I ended up using the following code, which is working fine. If is better than IIf because it does short-circuiting, which means that IIf will evaluate all expressions whether the condition is true or false but If will evaluate the first expression only if condition is true and evaluate the second expression only if condition is false.
Dim d As Decimal
Decimal.TryParse(currencyValue, d)
charValue = If(d = 0D, Nothing, String.Format("{0:C}", d))
I don't think there is a way using formatting to display an empty string.
But you can write it like:
charValue = If( currencyValue = 0D, "", currencyValue.ToString("C") )
using the If Operator (Visual Basic).
Also this is something I would not do:
If Double.Parse(Decimal.Parse(CDec(currencyValue))) = 0 Then
If currencyValue is Decimal:
If (currencyValue = 0D) Then
If currencyValue is Double:
If (currencyValue = 0R) Then
Also, if you are using a database and this is a Sql Server mind SQL Server Data Type Mappings
I don't think you can when using C or the other similar standard formats, since they are already defining a culture-specific format that will include a format for zero.
But if you specify your own custom format, you can specify three different formats separated by ;s, one each for positive numbers, negative numbers, and zero, respectively.
For example (giving an empty string for the zero format, resulting in blank zeroes):
charValue = String.Format("{0:#,##0.00;-#,##0.00;""""}", CDec(currencyValue))
And from what I can see, omitting the format for negative gives a default that matches the positive, whereas omitting the format for zero gives blank, which is what you're looking for, so this should be sufficient as well:
charValue = String.Format("{0:#,##0.00;;}", CDec(currencyValue))
(Using whichever custom format you wish.)
UPDATE: You can get the current currency symbol and manually put it into your custom format. IE:
Dim symbol = CultureInfo.CurrentCulture.NumberFormat.CurrencySymbol
charValue = String.Format("{0}{1:#,##0.00;;}", symbol, CDec(currencyValue))
From the sound of it, though, I think I would actually recommend doing basically what you started with, maybe with an extension method.
<Extension>
Public Function ToCurrencyString(pValue As Decimal) As String
Return IIf(pValue = 0, "", pValue.ToString("C"))
End Function
Dim someValue As Decimal = 1.23
Console.WriteLine(someValue.ToCurrencyString())
This gives you exactly what you're looking for. The exact same format as C gives, but with blank zeroes.

Reformatting date in google spreadsheet

I'm setting up a spreadsheet for someone else with a form to enter data.
One of the columns is supposed to hold a date. The input date format is like this example: "Jan 26, 2013" (there will be a lot of copy & paste involved to collect data, so changing the format at input step is not a real option).
I need this date column to be sortable, but the spreadsheet doesn't recognize this as a date but simply as a string. (It would recognize "Jan-26-2013", I've tried.)
So I need to reformat the input date.
My question is: how can I do this? I have looked around and google apps script looks like the way to go (though I haven't found a good example of reformatting yet).
Unfortunately my only programming experience is in Python, and of intermediate level. I could do this in Python without a problem, but I don't know any JavaScript.
(My Python approach would be:
splitted = date.split()
newdate = "-".join([splitted[0], splitted[1][:-1], splitted[2]])
return newdate
)
I also don't know how I'd go about linking the script to the spreadsheet - would I attach it to the cell, or the form, or where? And how? Any link to a helpful, understandable tutorial etc. on this point would help greatly.
Any help greatly appreciated!
Edit: Here's the code I ended up with:
//Function to filter unwanted " chars from date entries
function reformatDate() {
var sheet = SpreadsheetApp.getActiveSheet();
var startrow = 2;
var firstcolumn = 6;
var columnspan = 1;
var lastrow = sheet.getLastRow();
var dates = sheet.getRange(startrow, firstcolumn, lastrow, columnspan).getValues();
newdates = []
for(var i in dates){
var mydate = dates[i][0];
try
{
var newdate = mydate.replace(/"/g,'');
}
catch(err)
{
var newdate = mydate
}
newdates.push([newdate]);
}
sheet.getRange(startrow, firstcolumn, lastrow, columnspan).setValues(newdates)
}
For other confused google-script Newbies like me:
attaching the script to the spreadsheet works by creating the script from within the spreadsheet (Tools => Script Editor). Just putting the function in there is enough, you don't seem to need a function call etc.
you select the trigger of the script from the Script Editor (Resources => This Project's Triggers).
Important: the script will only work if there's an empty row at the bottom of the sheet in question!
Just an idea :
If you double click on your date string in the spreadsheet you will see that its real value that makes it a string instead of a date object is this 'Jan 26, 2013 with the ' in front of the string that I didn't add here...(The form does that to allow you to type what you want in the text area, including +322475... for example if it is a phone number, that's a known trick in spreadsheets cells) You could simply make a script that runs on form submit and that removes the ' in the cells, I guess the spreadsheet would do the rest... (I didn't test that so give it a try and consider this as a suggestion).
To remove the ' you can simply use the .replace() method **
var newValue = value.replace(/'/g,'');
here are some links to the relevant documentation : link1 link2
EDIT following your comment :
It could be simpler since the replace doesn't generate an error if no match is found. So you could make it like this :
function reformatDate() {
var sheet = SpreadsheetApp.getActiveSheet();
var dates = sheet.getRange(2, 6, sheet.getLastRow(), 1).getValues();
newdates = []
for(var i in dates){
var mydate = dates[i][0];
var newdate = mydate.replace(/"/g,'');
newdates.push([newdate]);
}
sheet.getRange(2, 6, sheet.getLastRow(), 1).setValues(newdates)
}
Also, you used the " in your code, presumably on purpose... my test showed ' instead. What made you make this choice ?
Solved it, I just had to change the comma to dot and it worked

How to insert complex strings into Actionscript?

How to insert complex strings into Actionscript?
So I have a string
-vvv -I rc w:// v dv s="60x40" --ut="#scode{vcode=FV1,acode=p3,ab=128,ch=2,rate=4400}:dup{dt=st{ac=http{mime=v/x-flv},mux=mpeg{v},dt=:80/sm.fv}}"
How to insert it into code like
public var SuperPuperComplexString:String = new String();
SuperPuperComplexString = TO_THAT_COMPLEX_STRING;
That string has so many problems like some cart of it can happen to be like regexp BUTI DO NOT want it to be parsed as any kind of reg exp - I need It AS IT IS!)
How to put that strange string into variable (put it not inputing it thru UI - hardcode it into AS code)?
SInce the only problem characters in your string are the double quotes ( " " ), just enclose your String in single quotes ( ' ' ). That will solve any problems.
It also depends on how you are loading that string into your code, as well.
You could even go so far as to encase that String in XML CDATA to ensure it's all delimited for when you want to use it.
var myString:XML = new XML();
myString = "<string><![CDATA[-vvv -I rc w:// v dv s="60x40" --ut="#scode{vcode=FV1,acode=p3,ab=128,ch=2,rate=4400}:dup{dt=st{ac=http{mime=v/x-flv},mux=mpeg{v},dt=:80/sm.fv}}"]]></string>"
Then, you can access it as a string from anywhere by just referencing myString.
var myString:String = '-vvv -I rc w:// v dv s="60x40" --ut="#scode{vcode=FV1,acode=p3,ab=128,ch=2,rate=4400}:dup{dt=st{ac=http{mime=v/x-flv},mux=mpeg{v},dt=:80/sm.fv}}"'
Not sure where your problem is..

Resources