Unable to resolve an Argument is of length zero error error - r

I get an Argument is of length zero error when I run the below code
The code is from this blog -http://giventhedata.blogspot.in/2012/08/r-and-web-for-beginners-part-iii.html.
library(XML)
url<- "http://news.bbc.co.uk/2/hi/uk_politics/8044207.stm"
first<-"Abbott, Ms Diane"
url.tab <- readHTMLTable(url)
for (i in 1:length(url.tab)){
if (as.character(url.tab[[i]][1,1]) == first ) {print(first)}
}
I know that the url.tab[[5]][1,1]) does contain the string "Abbott, Ms Diane", and when I run IF statement in isolation replacing the i with 5, it runs fine. Any help would be appreciated. I also tried declaring i<-1 upfront. DInt change anything.

Some of your tables are in fact NULL.
So you have to test for is.null before trying to subset the table:
for (i in 1:length(url.tab)){
this.tab <- url.tab[[i]]
if(!is.null(this.tab)) if(as.character(this.tab[1,1]) == first ) {print(first)}
}
[1] "Abbott, Ms Diane"

Related

How to Speed up read_html runtime in r?

I have a character string of 400 URLs called URLs.
I have a loop that has been working for a while but now it takes way too long. It used to just report the url as an error and then I would omit but its is getting hung up.
dput(URLs)
c("http://www.chinadaily.com.cn/a/202102/04/WS601b5bd7a31024ad0baa736d.html",
"http://www.xinhuanet.com/english/2021-02/02/c_139716479.htm",
"http://www.china.org.cn/world/Off_the_Wire/2021-02/02/content_77181645.htm",
"http://english.sina.com/world/af/2021-02-02/detail-ikftssap2511288.shtml",
"https://www.beijingnews.net/news/267750643/fox-takes-clubhouse-lead-as-johnson-makes-move-in-saudi-arabia",
"https://www.beijingnews.net/news/267768819/johnson-excited-for-season-after-second-saudi-title",
"https://en.wtcf.org.cn/GlobalNews/2021020320227.html", "https://www.ladepeche.fr/2021/02/08/golf-un-top-4-royal-pour-victor-perez-9360378.php",
"https://sport24.lefigaro.fr/golf/tour-europeen/actualites/victor-perez-dans-les-pas-de-dustin-johnson-en-arabie-saoudite-1032163",
"https://sport24.lefigaro.fr/golf/tour-europeen/actualites/european-tour-victor-perez-a-longtemps-tenu-tete-a-dustin-johnson-en-arabie-saoudite-1032273",
"https://www.france24.com/en/live-news/20210206-johnson-seizes-two-shot-lead-in-saudi-international",
"https://www.france24.com/en/live-news/20210205-fox-takes-clubhouse-lead-as-johnson-makes-move-in-saudi-arabia",
"https://www.france24.com/en/live-news/20210203-big-hitting-dechambeau-happy-to-take-longer-clubs-out-of-rivals-hands",
"https://www.france24.com/en/live-news/20210203-as-bubble-life-drags-on-psychologists-say-cricketers-need-more-support",
"https://www.sports.fr/golf/circuit-europeen/golf-perez-gratin-arabie-saoudite-426859.html",
"https://www.sport.fr/golf/lopen-de-france-est-sauve-758291.shtm",
"https://www.ffgolf.org/Actus/Pro/European-Tour/Saudi-International-ET-Perez-n-est-pas-passe-loin",
"https://www.ffgolf.org/Actus/Pro/European-Tour/Saudi-International-ET-Perez-a-rendez-vous-avec-DJ-dimanche",
"https://www.ffgolf.org/Actus/Pro/European-Tour/Saudi-International-ET-Rozner-au-sec-a-6-Perez-a-7",
"https://www.ffgolf.org/Actus/Pro/European-Tour/Saudi-International-ET-Rozner-et-Perez-demarrent-bien",
"https://www.ffgolf.org/Actus/Pro/LPGA-Tour/Franck-Riboud-On-va-pouvoir-continuer-a-travailler-sereinement",
"https://www.ffgolf.org/Actus/Pro/Feuilletons/Paroles-de-coach/Paroles-de-coach-6-Gwladys-Nocera",
"https://franceracing.fr/other/porsche-et-tag-heuer-scellent-un-partenariat-strategique/",
"https://www.rfi.fr/en/sports/20210206-johnson-seizes-two-shot-lead-in-saudi-international",
"https://www.rfi.fr/en/sports/20210205-fox-takes-clubhouse-lead-as-johnson-makes-move-in-saudi-arabia",
"https://www.rfi.fr/en/sports/20210203-big-hitting-dechambeau-happy-to-take-longer-clubs-out-of-rivals-hands",
"https://www.rfi.fr/en/sports/20210203-as-bubble-life-drags-on-psychologists-say-cricketers-need-more-support",
"https://www.jeudegolf.org/EasyBlog/Agathe-sauzon.html", "http://topactu.net/2021/02/viktor-hovland-vaults-into-farmers-lead-at-wet-torrey-pines/",
"https://www.sueddeutsche.de/sport/golf-kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-dpa.urn-newsml-dpa-com-20090101-210207-99-337940",
"https://www1.wdr.de/sport/golf-martin-kaymer-saudi-arabien-100.html",
"https://www.augsburger-allgemeine.de/sport/sonstige-sportarten/Kaymer-18-bei-Golf-Turnier-in-Saudi-Arabien-Johnson-siegt-id59059886.html",
"https://www.schwaebische.de/sport/ueberregionaler-sport_artikel,-kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-_arid,11325827.html",
"https://www.sport.de/news/ne4341625/golf--kaymer-beendet-turnier-in-saudi-arabien-als-18/",
"https://www.mz-web.de/sport/golf/kaymer-18--bei-golf-turnier-in-saudi-arabien---johnson-siegt-38027428",
"https://www.nwzonline.de/sport-meldungen/european-tour-kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_a_50,12,475833623.html",
"https://www.volksstimme.de/golf/news/kaymer-18.-bei-golf-turnier-in-saudi-arabien---johnson-siegt/1612702615000",
"https://www.wn.de/Sport/Weltsport/Golf/4360897-European-Tour-Kaymer-18.-bei-Golf-Turnier-in-Saudi-Arabien-Johnson-siegt",
"https://www.mainpost.de/sport/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-art-10562664",
"https://www.moz.de/nachrichten/sport/news/european-tour-kaymer-18.-bei-golf-turnier-in-saudi-arabien-johnson-siegt-54931493.html",
"https://www.svz.de/sport/weitere-sportarten/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-id31187247.html?nojs=true",
"https://www.rhein-zeitung.de/sport/aus-aller-welt/aus-aller-welt-golf_artikel,-kaymer-18-bei-golfturnier-in-saudiarabien-johnson-siegt-_arid,2220135.html",
"https://www.rhein-zeitung.de/sport/aus-aller-welt/aus-aller-welt-golf_artikel,-martin-kaymer-sagt-olympiastart-in-tokio-ab-_arid,2274019.html",
"https://www.allgemeine-zeitung.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.echo-online.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.mittelhessen.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.muensterschezeitung.de/Sport/Sportarten/Golf/4360897-European-Tour-Kaymer-18.-bei-Golf-Turnier-in-Saudi-Arabien-Johnson-siegt",
"https://www.wiesbadener-kurier.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.giessener-anzeiger.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://newsroom.porsche.com/de/2021/unternehmen/porsche-sportwagenhersteller-tag-heuer-luxusuhren-schmiede-zusammenarbeit-videostream-23558.html",
"https://www.azonline.de/Sport/Weitere-Sportarten/Golf/4360897-European-Tour-Kaymer-18.-bei-Golf-Turnier-in-Saudi-Arabien-Johnson-siegt",
"https://www.borkenerzeitung.de/welt/sport/Kaymer-18-bei-Golf-Turnier-in-Saudi-Arabien-Johnson-siegt-327224.html",
"https://www.golfpost.de/european-tour-saudi-international-2021-ergebnisse-runde-2-7777396527/",
"https://www.golfpost.de/396354-7777396354/", "https://www.golfpost.de/german-challenge-powerd-by-vcg-golf-challenge-tour-kehrt-nach-deutschland-zurueck-7777396396/",
"https://www.golfpost.de/die-macht-der-moneten-saudi-arabien-auf-dem-weg-zum-big-player-im-golf-7777396387/",
"https://www.kreis-anzeiger.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.wormser-zeitung.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://m.azonline.de/Sport/Weitere-Sportarten/Golf/4361712-PGA-Turnier-US-Golfstar-Koepka-triumphiert-bei-Phoenix-Open",
"https://www.mv-online.de/sport/sportmix/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-409658.html",
"https://www.golf.de/publish/dgv-sport/golf-team-germany/news/60228375/sophia-popov-nach-major-sieg-in-elite-team-germany",
"https://www.golf.de/publish/tournews/nachrichten-tour/60228372/einmal-saudi-einmal-etwas-gaudi",
"https://www.golf.de/publish/tournews/nachrichten-tour/60228387/koepka-comeback-und-eine-wuestenbilanz",
"https://www.ev-online.de/sport/sportmix/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt-409655.html",
"https://www.nach-welt.com/dustin-johnson-setzt-masstabe-aber-jordan-spieth-justin-rose-und-brooks-koepka-kehren-zur-form-zuruck/",
"https://www.nach-welt.com/ryan-fox-wird-sechster-wahrend-dustin-johnson-saudi-international-gewinnt/",
"https://www.usinger-anzeiger.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.gaeubote.de/Nachrichten/Golf-Turnier-in-Muenchen-Kaymer-faellt-zurueck-86604.html",
"https://www.gaeubote.de/Nachrichten/Kaymer-nach-Traumrunde-Zweiter-bei-Golf-Turnier-in-Muenchen-86664.html",
"https://www.main-spitze.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.lauterbacher-anzeiger.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.oberhessische-zeitung.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://de.advfn.com/p.php?pid=nmona&article=84265497", "https://www.buerstaedter-zeitung.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.golftime.de/golf-nachrichten/challenge-tour-in-deutschland-neues-profi-turnier/",
"https://www.golftime.de/golf-nachrichten/martin-kaymer-saudi-international-tour-news/",
"https://www.golftime.de/magazin/distanz-usga-ra-elite-spieler-regel-anpassung/",
"https://www.dmm.travel/nc/news/porsche-und-tag-heuer-arbeiten-zusammen/",
"https://www.lampertheimer-zeitung.de/sport/golf/kaymer-18-bei-golf-turnier-in-saudi-arabien-johnson-siegt_23109750",
"https://www.hongkongherald.com/news/267768819/johnson-excited-for-season-after-second-saudi-title",
"https://www.hongkongherald.com/news/267750643/fox-takes-clubhouse-lead-as-johnson-makes-move-in-saudi-arabia",
"http://hongkongcityportal.com/saudi-international-englands-david-horsey-leads-from-scotlands-stephen-gallacher/",
"http://hongkongcityportal.com/bryson-dechambeau-flattered-and-welcomes-proposed-rule-changes/",
"http://hongkongcityportal.com/paul-casey-englishman-defends-saudi-international-u-turn/",
"https://as.com/masdeporte/2021/02/03/golf/1612378989_020231.html",
"https://www.marca.com/golf/2021/02/07/601fd7c122601d860c8b45dc.html",
"https://www.marca.com/golf/2021/05/02/608ece1b22601d9d5d8b45f0.html",
"https://www.marca.com/golf/2021/02/03/601ad5d7268e3ef01e8b4670.html",
"https://www.republicworld.com/sports-news/other-sports/johnson-eases-to-another-victory-at-saudi-international.html",
"https://www.republicworld.com/sports-news/other-sports/dustin-johnson-within-1-shot-of-lead-at-saudi-international.html",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/dustin-johnson-excited-for-season-after-second-saudi-title/articleshow/80737390.cms",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/johnson-eases-to-another-victory-at-saudi-international/articleshow/80736264.cms",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/ryan-fox-takes-surprise-lead-at-saudi-international/articleshow/80711869.cms",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/horsey-goes-on-birdie-blitz-for-saudi-international-lead/articleshow/80691513.cms",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/shubhankar-shoots-69-in-opening-round-at-saudi-international/articleshow/80691501.cms",
"https://timesofindia.indiatimes.com/sports/golf/top-stories/big-hitting-dechambeau-happy-to-take-longer-clubs-out-of-rivals-hands/articleshow/80672723.cms",
"https://timesofindia.indiatimes.com/sports/cricket/news/as-bubble-life-drags-on-psychologists-say-cricketers-need-more-support/articleshow/80662353.cms",
"https://www.abc.es/deportes/abci-sergio-garcia-apunta-ryder-202102070038_noticia.html",
"https://www.abc.es/deportes/abci-golfistas-golpe-gimnasio-202102050031_noticia.html",
"https://www.investing.com/news/general/golf-johnson-holds-on-to-clinch-second-saudi-international-title-2411514"
)
####I have tried this:
html_reader<- function(x){return( tryCatch(xml2::read_html(URLs[k]), error = function(e) NULL))}
for (k in seq_along(URLs)) parsed_pages[k] <-lapply(as.list(URLs), html_reader)
I havent run into issues with runtime for some reason until now. The function will not complete even with the try() error function.
My current working code is the following:
pp <- replicate(list(), n = length(ESPN))
for (k in seq_along(ESPN)) pp[[k]] <- try(xml2::read_html(ESPN[k]), silent = TRUE)
It used to just take a while but now it never finishes.
I think the issue I am running into is due to the open connections. The script would get progressively slower and I feel it was due to the old connections. Here is a simple loop that closes out all of the connections. I will know when I run a particular report again if this is the solution but it has seemed to help so far.
for (i in seq_along(df$URLs)){function(i)
closeAllConnections(i)
}

R error: dims do not match the length of an object

I am currently trying to run some code (if you need to know the purpose to help me, ask me, but I'm trying to keep this question short). This is the code:
par<-c(a=.5,b=rep(1.3,4))
est<-rep(TRUE,length(par))
ncat<-5
Theta<-matrix(c(-6,-5.8,-5.6,-5.4,-5.2,-5,-4.8,-4.6,-4.4,-4.2,-4,-3.8,-3.6,-3.4,-3.2,-3,-2.8,-2.6,-2.4,-2.2,-2,-1.8,-1.6,-1.4,-1.2,-1,-0.8,-0.6,-0.4,-0.2,0,0.2,0.4,0.6,0.8,1,1.2,1.4,1.6,1.8,2,2.2,2.4,2.6,2.8,3,3.2,3.4,3.6,3.8,4,4.2,4.4,4.6,4.8,5,5.2,5.4,5.6,5.8,6))
p.grm<-function(par,Theta,ncat){
a<-par[1]
b<-par[2:length(par)]
z<-matrix(0,nrow(Theta),ncat)
y<-matrix(0,nrow(Theta),ncat)
y[,1]<-1
for(i in 1:ncat-1){
y[,i+1]<-(exp(a*(Theta-b[i])))/(1+exp(a*(Theta-b[i])))
}
for(i in 1:ncat-1){
z[,i]<-y[,i]-y[,i+1]
}
z[,ncat]<-y[,ncat]
z
}
However, when I try to run the code:
p.grm(par=par,Theta=Theta,ncat=ncat)
I get the following error:
Error: dims [product 61] do not match the length of object [0]
Traceback tells me that the error is occurring in the first for loop in the line:
y[,i+1]<-(exp(a*(Theta-b[i])))/(1+exp(a*(Theta-b[i])))
Could someone point me to what I'm doing wrong? When I try to run this code step by step outside of the custom p.grm function, everything seems to work fine.
It is a common mistake. When you write the for loop and you want it from 1 to ncat -1 remember to write it as for (i in 1:(ncat-1)) instead of for(i in 1:ncat-1) they are completly different.
You may also add to the function something to return return(z). Here it is the corrected code:
par<-c(a=.5,b=rep(1.3,4))
est<-rep(TRUE,length(par))
ncat<-5
Theta<-matrix(c(-6,-5.8,-5.6,-5.4,-5.2,-5,-4.8,-4.6,-4.4,-4.2,-4,-3.8,-3.6,-3.4,-3.2,-3,-2.8,-2.6,-2.4,-2.2,-2,-1.8,-1.6,-1.4,-1.2,-1,-0.8,-0.6,-0.4,-0.2,0,0.2,0.4,0.6,0.8,1,1.2,1.4,1.6,1.8,2,2.2,2.4,2.6,2.8,3,3.2,3.4,3.6,3.8,4,4.2,4.4,4.6,4.8,5,5.2,5.4,5.6,5.8,6))
p.grm<-function(par,Theta,ncat){
a<-par[1]
b<-par[2:length(par)]
z<-matrix(0,nrow(Theta),ncat)
y<-matrix(0,nrow(Theta),ncat)
y[,1]<-1
for(i in 1:(ncat-1)){
y[,i+1]<-(exp(a*(Theta-b[i])))/(1+exp(a*(Theta-b[i])))
}
for(i in 1:(ncat-1)){
z[,i]<-y[,i]-y[,i+1]
}
z[,ncat]<-y[,ncat]
return(z)
}
p.grm(par=par,Theta=Theta,ncat=ncat)

Need to Print Value of a Variable using Paste in R

I am trying to create a data frame of various error messages based on Data to be cross checked between two dataframes and storing the message in a vector in an iterative manner . I am using the following snippet for this purpose :
> for(j in 1:nrow(MySQL_Data)){ date_mysql=
> paste("MySQL_Data[",j,",1]") date_red= paste("RED_Data[",j,",1]")
> body= c() if(!date_mysql == date_red) {
> body<- append(body,paste("'There is data missing for date",date_mysql,"in",table2)) }else {
> NULL }}
My table2 variable prints as MYSQL_Data[2,1] instead of the actual value of the variable which is a date
Following is the Output :
"'There is data missing for date MySQL_Data[ 2 ,1] in Dream11_UserRegistration"
Can someone help me with the error that I am committing here..
Thanks in Advance !
Your use of paste in the definitions of data_mysql and data_red makes no sense. I’m assuming that what you actually want is this:
data_mysql = MySQL_Data[j, 1]
data_red = RED_Data[j, i]
Furthermore, you’re resetting body in every loop iteration so it will only ever hold a single element.

Error: missing value where True/False

I am trying to delete all values in a list that have the tag ".dsw". My list is a list of files using the function list.files. This is my code:
for (file in GRef) {
if (strsplit(file, "[.]")[[1]][3] == "dsw") {
#GRef=GRef[-file]
for(n in 1:length(GRef)){
if (GRef[n] == file){
GRef=GRef[-n]
}
}
}
}
Where GRef is the list of file names. I get the error listed above, but I dont understand why. I have looked at this post: Error .. missing value where TRUE/FALSE needed, but I dont think it is the same thing.
You shouldn't attempt to to modify a vector while you are looping over it. The problem is your are removing items you are then trying to extract later which is causing the missing values. It's better to identify all the items you want remove first, then remove them. For example
GRef <- c("a.file.dsw", "b.file.txt", "c.file.gif", "d.file.dsw")
exts <- sapply(strsplit(GRef, "[.]"), `[`, 3)
GRef <- GRef[exts!="dsw"]

Warning meassage: number of items to replace is not a multiple of replacement length

I got warnings when running this code.
For example, when I put
tm1<- summary(tmfit)[c(4,8,9)],
I can get the result, but I need to run this code for each $i$.
Why do I get this error?
Is there any way to do this instead of via a for loop?
Specifically, I have many regressants ($y$) with the same two regressors ($x$'s).
How I can get these results of regression analysis(to make some comparisons)?
dreg=read.csv("dayreg.csv")
fundr=read.csv("fundreturnday.csv")
num=ncol(fundr)
exr=dreg[,2]
tm=dreg[,4]
for(i in 2:num)
{
tmfit=lm(fundr[,i]~exr+tm)
tm1[i]<- summary(tmfit)[c(4,8,9)]
}
Any help is highly appreciated
Try storing your result into a list instead of a vector.
dreg=read.csv("dayreg.csv")
fundr=read.csv("fundreturnday.csv")
num=ncol(fundr)
exr=dreg[,2]
tm = list()
for(i in 2:num)
{
tmfit=lm(fundr[,i]~exr+tm)
tm1[[i]]<- summary(tmfit)[c(4,8,9)]
}
You can look at an element in the list like so
tm1[[2]]

Resources