I need help on the following code
I tried to generate a random series using while clause and met the following problem
When run codes as below
m1=0.82
set=rep(1, 1000)
while(mean(set)!=m1){
k=sample(1:1000,1)
if(set[k]!=0){set[k]=0}
}
every thing is fine, however the loop can end if I change ml as as below
m1=1-0.18
set=rep(1, 1000)
while(mean(set)!=m1){
k=sample(1:1000,1)
if(set[k]!=0){set[k]=0}
}
just change from ml=0.82 to ml=1-0.18, R can't stop looping.
Related
I have problems with my coded loop for some of my variables. I have run it firstly with one variable and it ran fine. Upon entering a different variable and different data range, the loop did not include all the outputs that it should have done according to the loop. There was no error message. The code for the loop is below:
geartype=data[179:187]
data_stats_structure_gt=as.data.frame(matrix(nrow=0,ncol=3))
names(data_stats_structure_gt)=c('Gear.Type','Man_Kendall_pvalue','Man_Kendall_tauv')
for (i in 1:ncol(geartype)){
dat=geartype[,i]
dat=as.data.frame(cbind(data$ï..Year,dat))
ds=names(geartype)[i]
names(dat)=c("Year",names(geartype)[i])
dat=dat[-which(is.na(dat[,2])==TRUE),]
if(nrow(dat)>=3){
output=as.data.frame(matrix(nrow=1,ncol=3))
names(output)=c('Gear.Type','Man_Kendall_pvalue','Man_Kendall_tauv')
output$Gear.Type=names(geartype)[i]
output$Man_Kendall_pvalue=mk.test(dat[,2])$p.value
output$Man_Kendall_tauv=mk.test(dat[,2])$estimates[3]
data_stats_structure_gt=rbind(data_stats_structure_gt,output)
}
}
I ran
mpiexec -n $nprocs julia --project myfile.jl
on a cluster, where myfile.jl has the following form
using Distributed; using Dates; using JLD2; using LaTeXStrings
#everywhere begin
using SharedArrays; using QuantumOptics; using LinearAlgebra; using Plots; using Statistics; using DifferentialEquations; using StaticArrays
#Defining some other functions and SharedArrays to be used later e.g.
MySharedArray=SharedArray{SVector{Nt,Float64}}(Np,Np)
end
#sync #distributed for pp in 1:Np^2
for jj in 1:Nj
#do some stuff with local variables
for tt in 1:Nt
#do some stuff with local variables
end
end
MySharedArray[pp]=... #using linear indexing
println("$pp finished")
end
timestr=Dates.format(Dates.now(), "yyyy-mm-dd-HH:MM:SS")
filename="MyName"*timestr
#save filename*".jld2"
#later on, some other small stuff like making and saving a figure. (This does give an error "no method matching heatmap_edges(::Surface{Array{Float64,2}}, ::Symbol)" but I think that this is a technical thing about Plots so not very related to the bigger issue here)
However, when looking at the output, there are a few issues that make me conclude that something is wrong
The "$pp finished" output is repeated many times for each value of pp. It seems that this amount is actually equal to 32=$nprocs
Despite the code not being finished, "MyName" files are generated. It should be one, but I get a dozen of them with different timestr component
EDIT: two more things that I can add
the output of the different "MyName" files is not identical, but this is expected since random numbers are used in the inner loops. There are 28 of them, a number that I don't easily recognize except that its again close to the 32 $nprocs
earlier, I wrote that the walltime was exceeded, but this turns out not to be true. The .o file ends with "BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES ... EXIT CODE :9", pretty shortly after the last output file.
$nprocs is obtained in the pbs script through
#PBS -l select=1:ncpus=32:mpiprocs=32
nprocs= `cat $PBS_NODEFILE|wc -l`
As pointed out by adamslc on the Julia discourse, the proper way to use Julia on a cluster is to either
Start a session with one core from the job script, add more with addprocs() in the Julia script itself
Use more specialized Julia packages
https://discourse.julialang.org/t/julia-distributed-redundant-iterations-appearing/57682/3
I am running a query in a loop for each store in a dataframe. Typically there are 70 or so stores so the loop repeats that many times for each complete loop.
Maybe 75% of the time this loop works all the way through with no errors.
About 25% of the time I get the following error during any one of the loop iterations:
Error in curl::curl_fetch_memory(url, handle = handle) :
Timeout was reached
Then I have to figure out which iteration bombed, and repeat the loop excluding iterations that completed successfully.
I can't find anything on the web to help me understand what is causing this seemingly random error. Perhaps it is a BQ technical issue? There does not seem to be any relation to the size of the result set it crashes on.
Here is the part of my code that does the loop...again it works all the way through most of the time. The cartesian product across IDs is intentional, as I want every combination of each Test ID with all possible Control IDs within store.
sql<-"SELECT pstore as store, max(pretrips) as pretrips FROM analytics.campaign_ids
group by 1 order by 1"
store_maxtrips<-query_exec(sql,project=project, max_pages = 1)
store_maxtrips
for (i in 1:length(store_maxtrips$store)) {
#pull back all ids shopping in same primary store as each test ID with their pre metrics
sql<-paste("SELECT a.pstore as pstore, a.id as test_id,
b.id as ctl_id,
(abs(a.zpbsales-b.zpbsales)*",wt_pb_sales,")+(abs(a.zcatsales-b.zcatsales)*",wt_cat_sales,")+
(abs(a.zsales-b.zsales)*",wt_retail_sales,")+(abs(a.ztrips-b.ztrips)*",wt_retail_trips,") as zscore
FROM analytics.campaign_ids a inner join analytics.pre_zscores b
on a.pstore=b.pstore
where a.id<>b.id and a.pstore=",store_maxtrips$store[i]," order by a.pstore, a.id, zscore")
print(paste("processing store",store_maxtrips$store[i]))
query_exec(sql,project=project,destination_table = "analytics.campaign_matches",
write_disposition = "WRITE_APPEND", max_pages = 1)
}
Solved!
It turns out I was using query_exec, but I should have been using insert_query_job since I do not want to retrieve any results. The errors were all happening in the course of R trying to retrieve results from BigQuery which I didn't want anyhow.
By using insert_query_job + wait_for(job) in my loop instead of the query_exec command, it eliminated all issues with the loop finishing.
I did also need to add a try() function to help circumvent some rare errors that still popped up with this approach. Thanks to MarkeD for this tip. So my final solution looked like this:
try(job<-insert_query_job(sql,project=project,destination_table = "analytics.campaign_matches",
write_disposition = "WRITE_APPEND"))
wait_for(job)
Thanks to everyone who commented and helped me research the issue.
I have my code running in a for loop over dates. The code takes a while to run, and there a couple of days left, but I urgently need whatever results there are. Is there a way of breaking the code/for loop, but keep whatever data has been produced up to now.
Yes. You can press "escape", examine the results and then restart your loop.
for(iii in 1:100000000) force(iii)
# now press ESC
iii
# in my case 1121673
# use this value to restart the loop later:
for(iii in 1121674:100000000) force(iii)
I have looked at other posts that appeared similar to this question but they have not helped me. This may be just my ignorance of R. Thus I decided to sign up and make my first post on stack-overflow.
I am running an R-script and would like the user to decide either to use one of the two following loops. The code to decide user input looks similar to the one below:
#Define the function
method.choice<-function() {
Method.to.use<-readline("Please enter 'New' for new method and'Old' for old method: ")
while(Method.to.use!="New" && Method.to.use!="Old"){ #Make sure selection is one of two inputs
cat("You have not entered a valid input, try again", "\n")
Method.to.use<-readline("Please enter 'New' for new method and 'Old' for old method: ")
cat("You have selected", Method.to.use, "\n")
}
return(Method.to.use)
}
#Run the function
method.choice()
Then below this I have the two possible choices:
if(Method.to.use=="New") {
for(i in 1:nrow(linelist)){...}
}
if(Method.to.use=="Old"){
for(i in 1:nrow(linelist)){...}
}
My issue is, and what I have read from other posts, is that whether I use "readline", "scan" or "ask", R does not wait for my input. Instead R will use the following lines as the input.
The only way I found that R would pause for input is if the code is all on the same line or if it is run line by line (instead of selecting all the code at once). See example from gtools using "ask":
silly <- function()
{
age <- ask("How old are you? ")
age <- as.numeric(age)
cat("In 10 years you will be", age+10, "years old!\n")
}
This runs with a pause:
silly(); paste("this is quite silly")
This does not wait for input:
silly()
paste("this is quite silly")
Any guidance would be appreciated to ensure I can still run my entire script and have it pause at readline without continuing. I am using R-studio and I have checked that interactive==TRUE.
The only other work-around I found is wrapping my entire script into one main function, which is not ideal for me. This may require me to use <<- to write to my environment.
Thank you in advance.