I just started out with Julia coming from python and am doing some experimenting.
I want to create a array that is a series of 5 arrays produced by a function inside a loop I assume I misunderstand how either the for loop or push function work. I've tried the append function but then you get a array of the elements in the coords array.
trj = []
for i in 1:n
coords = run(coords, vels)
println(coords)
push!(trj, coords)
end
println()
println(trj)
This is the output I get:
Any[[0.0806422, 0.785333, 0.942802], [0.0510982, 0.716546, 0.620131], [0.357414, 0.448247, 0.386514]]
Any[[0.265953, 0.785003, 0.898467], [0.230179, 0.729533, 0.656178], [0.260246, 0.571314, 0.302354]]
Any[[0.451263, 0.784673, 0.854133], [0.40926, 0.74252, 0.692224], [0.163079, 0.694381, 0.218195]]
Any[[0.636573, 0.784342, 0.809798], [0.588342, 0.755507, 0.72827], [0.065912, 0.817448, 0.134035]]
Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]]
Any[Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]], Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]], Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]], Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]], Any[[0.821884, 0.784012, 0.765463], [0.767423, 0.768494, 0.764317], [0.968745, 0.940515, 0.0498756]]]
As you can see the trj array is just the last version of coords times 5 instead of a series of the 5 produced coords arrays.
You only have one copy of the elements in coords, and modify those elements with your run() function. This is one reason Julia has a convention that argument-mutating functions be named with a !, as run!(coords, vels), to remind you of this. I would suggest you change run() to modify a copy of its argument and return it, but if you don't want to do that you could do:
trj = Vector{Vector{Vector{Float64}}}()
for i in 1:n
coords = run(coords, vels) # should be called run!
println(coords)
push!(trj, deepcopy(coords))
end
println()
println(trj)
deepcopy() gets you new elements in the vector, so the next call to coords() does not overwrite them.
Related
I have a dataframe called "df00" and it has the first column named as "Room type". This column has a lot of line breaks in it.
I need to make a copy of this dataframe (let's call it df00_dup) and add a duplicate of the first column in this new dataframe. The duplicated column will be named "RoomInfo". I need to remove the line breaks found in the column "RoomInfo".
Here is an extract of the first column of df00:
Room type
King Room with Balcony...
King Room with Balcony...
Deluxe Room...
Deluxe Room...
Column RoomInfo in df00_dup is, of course, the same as above.
My R codes to remove those line breaks currently stand as follows:
df00_dup <- df00
df00_dup$RoomInfo <- gsub("[\r\n]", "", df00[1])
When I run the second line from the above codes, the column RoomInfo now displays the following:
RoomInfo
c("King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
c("King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
c("King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
c("King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
All rows of the RoomInfo column is now filled with the above.
Expected output:
RoomInfo
King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
King Room with Balcony\n\n\n\n\n\n\n\n\n\n\n...
Deluxe Room\n\n\n\n\n\n\n\n\n\n\n...
Deluxe Room\n\n\n\n\n\n\n\n\n\n\n...
What is wrong with my codes? How can I remove those line breaks by maintaining the correct mapping?
Added (dput):
structure(list(`Room type` = c("King Room with Balcony\n\n\n\n\n\n\n\n\n\n2 large double beds\n\n\n\n\n\n\n\n\n50 m²BalconyGarden viewPrivate bathroom\n\n\n\nFree toiletries\n\n\n\n\nToilet\n\n\n\n\nFireplace\n\n\n\n\nBath or shower\n\n\n\n\nTowels\n\n\n\n\nLinen\n\n\n\n\nSocket near the bed\n\n\n\n\nDesk\n\n\n\n\nSeating Area\n\n\n\n\nSlippers\n\n\n\n\nMosquito net\n\n\n\n\nIroning facilities\n\n\n\n\nTea/Coffee maker\n\n\n\n\nOutdoor furniture\n\n\n\n\nOutdoor dining area\n\n\n\n\nWake-up service\n\n\n\n\nEntire unit located on ground floor\n\n\n\n\nClothes rack\n\n\n\n\nDrying rack for clothing\n\n\n\n\nToilet paper\n\n\n\n\nHand sanitiser",
"King Room with Balcony\n\n\n\n\n\n\n\n\n\n2 large double beds\n\n\n\n\n\n\n\n\n50 m²BalconyGarden viewPrivate bathroom\n\n\n\nFree toiletries\n\n\n\n\nToilet\n\n\n\n\nFireplace\n\n\n\n\nBath or shower\n\n\n\n\nTowels\n\n\n\n\nLinen\n\n\n\n\nSocket near the bed\n\n\n\n\nDesk\n\n\n\n\nSeating Area\n\n\n\n\nSlippers\n\n\n\n\nMosquito net\n\n\n\n\nIroning facilities\n\n\n\n\nTea/Coffee maker\n\n\n\n\nOutdoor furniture\n\n\n\n\nOutdoor dining area\n\n\n\n\nWake-up service\n\n\n\n\nEntire unit located on ground floor\n\n\n\n\nClothes rack\n\n\n\n\nDrying rack for clothing\n\n\n\n\nToilet paper\n\n\n\n\nHand sanitiser",
"Deluxe Family Room\n\n\n\n\n\n\n\n\n\n2 extra-large double beds\n\n\n\n\n\n\n\n\n70 m²BalconyGarden viewPrivate bathroom\n\n\n\nFree toiletries\n\n\n\n\nToilet\n\n\n\n\nSofa\n\n\n\n\nFireplace\n\n\n\n\nBath or shower\n\n\n\n\nTowels\n\n\n\n\nLinen\n\n\n\n\nSocket near the bed\n\n\n\n\nDesk\n\n\n\n\nSeating Area\n\n\n\n\nSlippers\n\n\n\n\nMosquito net\n\n\n\n\nIroning facilities\n\n\n\n\nTea/Coffee maker\n\n\n\n\nOutdoor furniture\n\n\n\n\nOutdoor dining area\n\n\n\n\nWake-up service\n\n\n\n\nEntire unit located on ground floor\n\n\n\n\nClothes rack\n\n\n\n\nDrying rack for clothing\n\n\n\n\nToilet paper\n\n\n\n\nHand sanitiser\n\n\n\nMore",
"Deluxe Family Room\n\n\n\n\n\n\n\n\n\n2 extra-large double beds\n\n\n\n\n\n\n\n\n70 m²BalconyGarden viewPrivate bathroom\n\n\n\nFree toiletries\n\n\n\n\nToilet\n\n\n\n\nSofa\n\n\n\n\nFireplace\n\n\n\n\nBath or shower\n\n\n\n\nTowels\n\n\n\n\nLinen\n\n\n\n\nSocket near the bed\n\n\n\n\nDesk\n\n\n\n\nSeating Area\n\n\n\n\nSlippers\n\n\n\n\nMosquito net\n\n\n\n\nIroning facilities\n\n\n\n\nTea/Coffee maker\n\n\n\n\nOutdoor furniture\n\n\n\n\nOutdoor dining area\n\n\n\n\nWake-up service\n\n\n\n\nEntire unit located on ground floor\n\n\n\n\nClothes rack\n\n\n\n\nDrying rack for clothing\n\n\n\n\nToilet paper\n\n\n\n\nHand sanitiser\n\n\n\nMore",
"Deluxe Family Room\n\n\n\n\n\n\n\n\n\n2 extra-large double beds\n\n\n\n\n\n\n\n\n70 m²BalconyGarden viewPrivate bathroom\n\n\n\nFree toiletries\n\n\n\n\nToilet\n\n\n\n\nSofa\n\n\n\n\nFireplace\n\n\n\n\nBath or shower\n\n\n\n\nTowels\n\n\n\n\nLinen\n\n\n\n\nSocket near the bed\n\n\n\n\nDesk\n\n\n\n\nSeating Area\n\n\n\n\nSlippers\n\n\n\n\nMosquito net\n\n\n\n\nIroning facilities\n\n\n\n\nTea/Coffee maker\n\n\n\n\nOutdoor furniture\n\n\n\n\nOutdoor dining area\n\n\n\n\nWake-up service\n\n\n\n\nEntire unit located on ground floor\n\n\n\n\nClothes rack\n\n\n\n\nDrying rack for clothing\n\n\n\n\nToilet paper\n\n\n\n\nHand sanitiser\n\n\n\nMore"
), `Price for 2 nights` = c("MUR 46,748\n\n\n\nPrice\nMUR 46,748\n\n\n\n\n+MUR 4,030 taxes and charges",
"MUR 23,374\n\n\n\nPrice\nMUR 23,374\n\n\n\n\n+MUR 2,015 taxes and charges",
"MUR 59,334\n\n\n\nPrice\nMUR 59,334\n\n\n\n\n+MUR 5,115 taxes and charges",
"MUR 89,001\n\n\n\nPrice\nMUR 89,001\n\n\n\n\n+MUR 7,672 taxes and charges",
"MUR 29,667\n\n\n\nPrice\nMUR 29,667\n\n\n\n\n+MUR 2,557 taxes and charges"
)), row.names = c(NA, -5L), class = c("tbl_df", "tbl", "data.frame"))
We would need to extract the column as a vector as df00[1] would still be a data.frame with a single column and gsub/sub expects a vector as input
df00_dup$RoomInfo <- gsub("[\r\n]", "", df00[[1]])
I am trying to make a dynamic string in Julia by inserting the value of a variable into the string. Everything worked fine until today when the value returned nothing leaving me with an error.
How do I include a nothing in a string? At least without having to go through the hassle of some if n == nothing; n = "None" thing for every variable I want to insert into a string.
function charge_summary(charges_df)
if size(charges_df)[1] > 0
n_charges = size(charges_df)[1]
total_charges = round(abs(sum(charges_df[:amount])), digits=2)
avg_charges = round(abs(mean(charges_df[:amount])), digits=2)
most_frequent_vender = first(sort(by(charges_df, :transaction_description, nrow), :x1, rev=true))[:transaction_description]
sms_text = """You have $n_charges new transactions, totaling \$$total_charges.
Your average expenditure is \$$avg_charges.
Your most frequented vender is $most_frequent_vender.
"""
return sms_text
else
return nothing
end
end
sms_msg = charge_summary(charges_df)
Returns:
ArgumentError: `nothing` should not be printed; use `show`, `repr`, or custom output instead.
string at io.jl:156 [inlined]
charge_summary(::DataFrame) at get-summary.jl:18
top-level scope at none:0
include_string(::Module, ::String, ::String, ::Int64) at eval.jl:30
(::getfield(Atom, Symbol("##105#109")){String,Int64,String})() at eval.jl:91
withpath(::getfield(Atom, Symbol("##105#109")){String,Int64,String}, ::String) at utils.jl:30
withpath at eval.jl:46 [inlined]
#104 at eval.jl:90 [inlined]
hideprompt(::getfield(Atom, Symbol("##104#108")){String,Int64,String}) at repl.jl:76
macro expansion at eval.jl:89 [inlined]
(::getfield(Atom, Symbol("##103#107")))(::Dict{String,Any}) at eval.jl:84
handlemsg(::Dict{String,Any}, ::Dict{String,Any}) at comm.jl:168
(::getfield(Atom, Symbol("##14#17")){Array{Any,1}})() at task.jl:259
Unfortunately you have to explicitly handle nothing. For example like this:
Your most frequented vender is $(something(most_frequent_vender, "None")).
The reason for this is that it is not clear how you would want nothing to be converted to a string, so you have to provide this value (in your case you wanted "None").
A shorter version would be:
Your most frequented vender is $(repr(most_frequent_vender)).
but then nothing is printed as "nothing".
Define Base.string(x::Nothing) method:
➜ ~ julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.0.3 (2018-12-20)
_/ |\__'_|_|_|\__'_| | android-termux/900b8607fb* (fork: 1550 commits, 315 days)
|__/ |
julia> Base.string(x::Nothing) = repr(x) # or just return the string "None", that's up to you.
julia> "$(nothing)"
"nothing"
julia>
Julia 1.3 update
Allow nothing to be printed #32148
Have the following string that I'd like to parse:
((K00134,K00150) K00927,K11389) (K00234,K00235)
each step is separated by a space and alternation is represented by a comma. I'm stuck in the first part of the string where there is a space inside the brackets. The desired output I'm looking for is:
[[['K00134', 'K00150'], 'K00927'], 'K11389'], ['K00234', 'K00235']
What I've got so far is a basic setup to do recursive parsing, but I'm stumped on how to code in a space separated list into the bracket expression
from pyparsing import Word, Literal, Combine, nums, \
Suppress, delimitedList, Group, Forward, ZeroOrMore
ortholog = Combine(Literal('K') + Word(nums, exact=5))
exp = Forward()
ortholog_group = Suppress('(') + Group(delimitedList(ortholog)) + Suppress(')')
atom = ortholog | ortholog_group | Group(Suppress('(') + exp + Suppress(')'))
exp <<= atom + ZeroOrMore(exp)
You are on the right track, but I think you only need one place where you include grouping with ()'s, not two.
import pyparsing as pp
LPAR,RPAR = map(pp.Suppress, "()")
ortholog = pp.Combine('K' + pp.Word(pp.nums, exact=5))
ortholog_group = pp.Forward()
ortholog_group <<= pp.Group(LPAR + pp.OneOrMore(ortholog_group | pp.delimitedList(ortholog)) + RPAR)
expr = pp.OneOrMore(ortholog_group)
tests = """\
((K00134,K00150) K00927,K11389) (K00234,K00235)
"""
expr.runTests(tests)
gives:
((K00134,K00150) K00927,K11389) (K00234,K00235)
[[['K00134', 'K00150'], 'K00927', 'K11389'], ['K00234', 'K00235']]
[0]:
[['K00134', 'K00150'], 'K00927', 'K11389']
[0]:
['K00134', 'K00150']
[1]:
K00927
[2]:
K11389
[1]:
['K00234', 'K00235']
This is not exactly what you said you were looking for:
you wanted: [[['K00134', 'K00150'], 'K00927'], 'K11389'], ['K00234', 'K00235']
I output : [[['K00134', 'K00150'], 'K00927', 'K11389'], ['K00234', 'K00235']]
I'm not sure why there is grouping in your desired output around the space-separated part (K00134,K00150) K00927. Is this your intention or a typo? If intentional, you'll need to rework the definition of ortholog_group, something that will do a delimited list of space-delimited groups in addition to the grouping at parens. The closest I could get was this:
[[[[['K00134', 'K00150']], 'K00927'], ['K11389']], [['K00234', 'K00235']]]
which required some shenanigans to group on spaces, but not group bare orthologs when grouped with other groups. Here is what it looked like:
ortholog_group <<= pp.Group(LPAR + pp.delimitedList(pp.Group(ortholog_group*(1,) & ortholog*(0,))) + RPAR) | pp.delimitedList(ortholog)
The & operator in combination with the repetition operators gives the space-delimited grouping (*(1,) is equivalent to OneOrMore, *(0,) with ZeroOrMore, but also supports *(10,) for "10 or more", or *(3,5) for "at least 3 and no more than 5"). This too is not quite exactly what you asked for, but may get you closer if indeed you need to group the space-delimited bits.
But I must say that grouping on spaces is ambiguous - or at least confusing. Should "(A,B) C D" be [[A,B],C,D] or [[A,B],C],[D] or [[A,B],[C,D]]? I think, if possible, you should permit comma-delimited lists, and perhaps space-delimited also, but require the ()'s when items should be grouped.
I am trying to accomplish a task using R to scrape data on a website.
I would like to go through each link on the following page:
http://capitol.hawaii.gov/advreports/advreport.aspx?year=2013&report=deadline&rpt_type=&measuretype=hb&title=House Bills
Select only items with Current Status showing "transmitted to the governor". For example, http://capitol.hawaii.gov/measure_indiv.aspx?billtype=HB&billnumber=17&year=2013
And then scrapping the cells within STATUS TEXT for the following clause" Passed Final Reading". For example: Passed Final Reading as amended in SD 2 with Representative(s) Fale, Jordan, Tsuji voting aye with reservations; Representative(s) Cabanilla, Morikawa, Oshiro, Tokioka voting no (4) and none excused (0).
I have tried using previous examples with packages Rcurl and XML (in R), but I don't know how to use them correctly for aspx sites. So what I would love to have is: 1. Some suggestion on how to build such a code. 2. And recommendation for how to learn the knowledge needed for performing such a task.
Thanks for any help,
Tom
require(httr)
require(XML)
basePage <- "http://capitol.hawaii.gov"
h <- handle(basePage)
GET(handle = h)
res <- GET(handle = h, path = "/advreports/advreport.aspx?year=2013&report=deadline&rpt_type=&measuretype=hb&title=House")
# parse content for "Transmitted to Governor" text
resXML <- htmlParse(content(res, as = "text"))
resTable <- getNodeSet(resXML, '//*/table[#id ="GridViewReports"]/tr/td[3]')
appRows <-sapply(resTable, xmlValue)
include <- grepl("Transmitted to Governor", appRows)
resUrls <- xpathSApply(resXML, '//*/table[#id ="GridViewReports"]/tr/td[2]//#href')
appUrls <- resUrls[include]
# look at just the first
res <- GET(handle = h, path = appUrls[1])
resXML <- htmlParse(content(res, as = "text"))
xpathSApply(resXML, '//*[text()[contains(.,"Passed Final Reading")]]', xmlValue)
[1] "Passed Final Reading as amended in SD 2 with Representative(s) Fale, Jordan,
Tsuji voting aye with reservations; Representative(s) Cabanilla, Morikawa, Oshiro,
Tokioka voting no (4) and none excused (0)."
Let package httr handle all the background work by setting up a handle.
If you want to run over all 92 links:
# get all the links returned as a list (will take sometime)
# print statement included for sanity
res <- lapply(appUrls, function(x){print(sprintf("Got url no. %d",which(appUrls%in%x)));
GET(handle = h, path = x)})
resXML <- lapply(res, function(x){htmlParse(content(x, as = "text"))})
appString <- sapply(resXML, function(x){
xpathSApply(x, '//*[text()[contains(.,"Passed Final Reading")]]', xmlValue)
})
head(appString)
> head(appString)
$href
[1] "Passed Final Reading as amended in SD 2 with Representative(s) Fale, Jordan, Tsuji voting aye with reservations; Representative(s) Cabanilla, Morikawa, Oshiro, Tokioka voting no (4) and none excused (0)."
$href
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."
[2] "Passed Final Reading as amended in CD 1 with Representative(s) Cullen, Har voting aye with reservations; Representative(s) McDermott voting no (1) and none excused (0)."
$href
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; Representative(s) Hashem, McDermott voting no (2) and none excused (0)."
$href
[1] "Passed Final Reading, as amended (CD 1). 24 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 1 Excused: Ige."
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and Representative(s) Say excused (1)."
$href
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."
[2] "Passed Final Reading as amended in CD 1 with Representative(s) Johanson voting aye with reservations; none voting no (0) and none excused (0)."
$href
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and none excused (0)."
I want to combine awk and R language. The thing is that I have a set of *.txt files in a specified directory and that I don't know the length of the header from the files. In some cases I have to skip 25 lines while in others I have to skip 27 and etc. So I want to type some awk commands to get the number of lines to skip. Once I have this value, I can begin processing the data with R.
Furthermore, in the R file I combine R an bash so my code looks like this :
!/usr/bin/env Rscript
...
argv <- commandArgs(T)
**error checking...**
import_file <- argv[1]
export_file <- argv[2]
**# your function call**
format_windpro(import_file, export_file)
Where and how can i type my awk command. Thanks!
I tried to do what you told me about awk commands and I still get an error. The program doesn't recognize my command and so I can not enter the number of lines to skip to my function. Here is my code:
**nline <- paste('$(grep -n 'm/s' import_file |awk -F":" '{print $1}')')
nline <- scan(pipe(nline),quiet=T)**
I look for the pattern m/s in the first column in order to know where I have my header text. I use R under w7.
Besides Vincent's hint of using system("awk ...", intern=TRUE), you can also use the pipe() function that is part of the usual text connections:
R> sizes <- read.table(pipe("ls -l /tmp | awk '!/^total/ {print $5}'"))
R> summary(sizes)
V1
Min. : 0
1st Qu.: 482
Median : 4096
Mean : 98746
3rd Qu.: 13952
Max. :27662342
R>
Here I am piping a command into awk and then read all the output from awk, that could also be a single line:
R> cmd <- "ls -l /tmp | awk '!/^total/ {sum = sum + $5} END {print sum}'"
R> totalsize <- scan(pipe(cmd), quiet=TRUE)
R> totalsize
[1] 116027050
R>
You can use system to run an external program from R.
system("gawk --version", intern=TRUE)