How can I easily import citations from R into endnote obtained by for instance
citation("ggplot2")
Is there a good workflow for this or do I manually have to do it?
How automated this can be will depend on what Endnote can import. It seems BibTeX import is not currently possible out of the box, and requires some extra software. See for example: http://www.lib.uts.edu.au/content/faq/how-can-i-import-bibliography-endnote-bibtex-latex-what-about-converting-other-way-endno
Read ?bibentry and in particular the argument style and the Details section. See if Endnote can import data in any of those formats? I doubt it, but I have never used Endnote.
If not, we can go the BibTeX route if you install something that allows you to import BibTeX into Endnote.
> utils:::print.bibentry(citation("ggplot2"), style = "Bibtex")
#Book{,
author = {Hadley Wickham},
title = {ggplot2: elegant graphics for data analysis},
publisher = {Springer New York},
year = {2009},
isbn = {978-0-387-98140-6},
url = {http://had.co.nz/ggplot2/book},
}
To get this out into a file for passing to an import utility, you can use capture.output()
capture.output(utils:::print.bibentry(citation("ggplot2"), style = "Bibtex"),
file = "endnote_import.bib")
Which gives a file with the following content:
$ cat endnote_import.bib
#Book{,
author = {Hadley Wickham},
title = {ggplot2: elegant graphics for data analysis},
publisher = {Springer New York},
year = {2009},
isbn = {978-0-387-98140-6},
url = {http://had.co.nz/ggplot2/book},
}
which you should be able to import with third party tools.
Related
I want to extract the table from this website https://www.rankingthebrands.com/The-Brand-Rankings.aspx?rankingID=37&year=214
Checking the source of that website, I noticed that somehow the table tag is missing. I assume that this table is a summary of multiple div classes. Is there any easy approach to convert this table to excel/csv? I badly have coding skills/experience...
Appreciate any help
There are a few way to do that. One of which (in python) is (pretty self-explanatory, I believe):
import lxml.html as lh
import csv
import requests
url = 'https://www.rankingthebrands.com/The-Brand-Rankings.aspx?rankingID=37&year=214'
req = requests.get(url)
doc = lh.fromstring(req.text)
headers = ['Position', 'Name', 'Brand Value', 'Last']
with open('brands.csv', 'a', newline='') as fp:
#note the 'a' in there - for 'append`
file = csv.writer(fp)
file.writerow(headers)
#with the headers out of the way, the heavier xpath lifting begins:
for row in doc.xpath('//div[#class="top100row"]'):
pos = row.xpath('./div[#class="pos"]//text()')[0]
name = row.xpath('.//div[#class="name"]//text()')[0]
brand_val = row.xpath('.//div[#class="weighted"]//text()')[0]
last = row.xpath('.//div[#class="lastyear"]//text()')[0]
file.writerow([pos,name,brand_val,last])
The resulting file should be at least close to what you're looking for.
I have this question. I need to download certain .csv files from a website as the title said, and i'm having troubles doing it. I'm very new on programming and especially with this topic(web scraping)
from bs4 import BeautifulSoup as BS
import requests
DOMAIN = 'https://datos.gob.ar'
URL = 'https://datos.gob.ar/dataset/cultura-mapa-cultural-espacios-culturales/'
FILETYPE = ".csv"
def get_soup(url):
return BS(requests.get(url).text, 'html.parser')
for link in get_soup(URL).find_all('a'):
file_link = link.get('href')
if FILETYPE in file_link:
print(file_link)
this code shows all avaibable .csv files but I just need to download those which end up with "biblioteca popular.csv" , "cine.csv" and "museos.csv"
Maybe it's a very simple task but I can not finding out
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/456d1087-87f9-4e27-9c9c-1d9734c7e51d/download/biblioteca_especializada.csv
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/01c6c048-dbeb-44e0-8efa-6944f73715d7/download/biblioteca_popular.csv
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/8d0b7f33-d570-4189-9961-9e907193aebc/download/casas_bicentenario.csv
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/4207def0-2ff7-41d5-9095-d42ae8207a5d/download/museos.csv
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/392ce1a8-ef11-4776-b280-6f1c7fae16ae/download/cine.csv
https://datos.cultura.gob.ar/dataset/37305de4-3cce-4d4b-9d9a-fec3ca61d09f/resource/87ebac9c-774c-4ef2-afa7-044c41ee4190/download/teatro.csv
You can extract the JavaScript object housing that info which otherwise would be loaded to where you see if by JavaScript running in the browser. You then need to do some Unicode code point cleaning and string cleaning and parse as JSON. You can use a key word list to select from desired urls.
Unicode cleaning method by #Mark Tolonen
import json
import requests
import re
URL = 'https://datos.gob.ar/dataset/cultura-mapa-cultural-espacios-culturales/'
r = requests.get(URL)
search = ["Bibliotecas Populares", "Salas de Cine", "Museos"]
s = re.sub( r'\n\s{2,}', '', re.search(r'"#graph": (\[[\s\S]+{0}[\s\S]+)}}'.format(search[0]), r.text).group(1))
data = json.loads(re.sub(r'\\"', '', re.sub(r'\\u([0-9a-fA-F]{4})',lambda m: chr(int(m.group(1),16)),s)))
for i in data:
if 'schema:name' in i:
name = i['schema:name']
if name in search:
print(name)
print(i['schema:url'])
I use the following to create a D2L exam from the "capital.Rmd" example (I converted the question to schoice)
exams2blackboard("capitals.Rmd", n =3, name = "testquiz" )
After I upload the testquiz.zip file, I notice that the correct answer must be manually chosen on the D2L platform.
I was wondering if there is a workaround.
Many Thanks,
Umut
If you want the correct solution to be selected, do not use the Import option from the Question Library or from the Quiz itself. Use the Import/Export/Copy Components under the Course Admin tab.
If you import the questions through the following steps, BrightSpace correctly picks the right solution. It’s a bit longer but seems to correctly choose the solution.
Under the Course Admin tab of your course, go to
'Import/Export/Copy Components' -> ‘Import Components’ -> Start -> (drag and drop the ZIP file)
Click ‘Advanced Options…’
This step will take a few minutes for large files; if you do not click
Advanced Options, then the import will automatically import the
questions into the 'Question Library' and will generate a Quiz with the
imported questions; you do not want this.
-> Continue -> Continue -> at this point choose 'Question Library' from the section 'Select Components to Import'
I would not choose ‘Quizzes’ because it automatically creates a quiz
and makes it available to students. It has the unfortunate side-effect
of making ALL the questions available, which means all the versions of
various dynamic questions; this is not something we want.
-> Continue -> Continue. This stage takes a few minutes for large
imports.
Now the Questions are available in the Question Library and can be used to generate new quizzes. Each question has the correct answer selected already. This works for ‘schoice’ and ‘mchoice’ versions of questions. Currently, plots are not imported, though, still trying to figure out why.
This problem is new to me. In earlier versions of Brightspace/D2L the import of single-choice and multiple-choice exercises via exams2blackboard() worked well. Possibly, D2L changed in the meantime given that neither the current release version from CRAN nor the development version from R-Forge work for you.
D2L also supports other import formats and we did play around with some of these. See the following discussions in the R/exams forum on R-Forge:
https://R-Forge.R-project.org/forum/forum.php?thread_id=33404&forum_id=4377&group_id=1337
https://R-Forge.R-project.org/forum/forum.php?thread_id=33657&forum_id=4377&group_id=1337
Notably we tried to use the XML-based QTI 2.1 format that seems to be employed by D2L internally. However, D2L apparently uses a particular custom flavor of QTI 2.1. It should be possible to reverse engineer that and improve exams2qti21() correspondingly but so far (to the best of my knowledge) no one put the time and effort into this that would be needed.
For simple single/multiple choice questions a CSV-based exchange format can also be used. I have put together a very basic exams2d2l() function that was posted in the threads above and that I'm also including below. It can set up the CSV file for a single exercise like the capitals.Rmd exercise that you use above. For plain text exercises like that it seems to work well but not for more complex elements (graphics, code, math, etc.).
exams2d2l <- function(file, dir = ".", ## n = 1L, nsamp = NULL disabled for now
name = NULL, quiet = TRUE, edir = NULL, tdir = NULL, sdir = NULL, verbose = FALSE,
resolution = 100, width = 4, height = 4, svg = FALSE,
encoding = "", converter = NULL, ...)
{
## for Rnw exercises use "ttm" converter otherwise "pandoc" converter
if(any(tolower(tools::file_ext(unlist(file))) == "rmd")) {
if(is.null(converter)) converter <- "pandoc"
} else {
if(is.null(converter)) converter <- "ttm"
}
## output directory or display on the fly
## output name processing
if(is.null(name)) name <- tools::file_path_sans_ext(basename(file))
## set up .html transformer and writer function
htmltransform <- make_exercise_transform_html(converter = converter, ...)
## create exam with HTML text
rval <- xexams(file,
driver = list(sweave = list(quiet = quiet, pdf = FALSE, png = !svg, svg = svg,
resolution = resolution, width = width, height = height, encoding = encoding),
read = NULL, transform = htmltransform, write = NULL),
dir = dir, edir = edir, tdir = tdir, sdir = sdir, verbose = verbose)
## currently: only a single exercise
rval <- rval[[1L]][[1L]]
## put together CSV
cleanup <- function(x) gsub('"', '""', paste(x, collapse = "\n"), fixed = TRUE)
rval <- c(
'NewQuestion,MC,,,',
sprintf('ID,"%s",,,', cleanup(rval$metainfo$file)),
sprintf('Title,"%s",,,', cleanup(rval$metainfo$name)),
sprintf('QuestionText,"%s",,,', cleanup(rval$question)),
sprintf('Points,%s,,,', if(is.null(rval$metainfo$points)) 1 else rval$metainfo$points),
'Difficulty,1,,,',
'Image,,,,',
paste0('Option,', ifelse(rval$metainfo$solution, 100, 0), ',"', cleanup(rval$questionlist), '",,"', cleanup(rval$solutionlist), '"'),
'Hint,,,,',
sprintf('Feedback,"%s",,,', cleanup(rval$solution))
)
writeLines(rval, file.path(dir, paste0(name, ".csv")))
invisible(rval)
}
I have .bib file (downloaded from web of science) and I want to import it into R, replace all instances of "in light of" with "CONSIDERING", and export it as a .bib file. I have not been able to find a function that can write my data back to a .bib file. WriteBib does not work because refs is a "pairlist" object, not "bibentry". Any advice on how to export a .bib file that can be imported into Mendeley? thanks for your help!
here is the code:
library(bibtex)
library(RefManageR)
refs = do_read_bib("/Users/CarrieAnn/Downloads/savedrecs (1).bib", encoding = "unknown", srcfile)
for (i in 1:length(refs)) {
refs[[i]] = gsub("in light of", "CONSIDERING", refs[[i]])
}
I think your easiest option is to treat the .bib file like a normal text file. Try this:
raw_text <- readLines("example.bib")
new_text <- gsub("in light of", "CONSIDERING", raw_text)
writeLines(new_text, con="new_example.bib")
Contents of example.bib:
% a sample bibliography file
%
#article{small,
author = {Doe, John},
title = {A small paper},
journal = {The journal of small papers},
year = 1997,
volume = {-1},
note = {in light of recent events},
}
#article{big,
author = {Smith, Jane},
title = {A big paper},
journal = {The journal of big papers},
year = 7991,
volume = {MCMXCVII},
note = {in light of what happened},
}
Output of new_example.bib:
% a sample bibliography file
%
#article{small,
author = {Doe, John},
title = {A small paper},
journal = {The journal of small papers},
year = 1997,
volume = {-1},
note = {CONSIDERING recent events},
}
#article{big,
author = {Smith, Jane},
title = {A big paper},
journal = {The journal of big papers},
year = 7991,
volume = {MCMXCVII},
note = {CONSIDERING what happened},
}
A bit of explanation:
BibEntry objects have non-standard internals and are hard to work with, outside of the functionality provided in the RefManageR package. As soon as you unclass or reduce a BibEntry object to a list, it becomes difficult to put back into a bib format, due to the object's required mix of fields and attributes. (And to make matters worse, bibtex and RefManageR don't have exactly the same internal structure, so it's hard to convert from one context to the other.)
Try changing your code like this (make sure to use read.bib function and in the loop, reference the fields in which the text you want to change appears, e.g. "note" or "title". For the example file supplied by #andrew_reece, it should work like this:
refs = read.bib("example.bib", encoding = "unknown", srcfile)
for (i in 1:length(refs)) {
refs$note[i] = gsub("in light of", "CONSIDERING", refs$note[i])
}
WriteBib(as.BibEntry(refs), "example2.bib")
However, based on your task description, I agree with #andrew_reece that treating bib files as plain text is easier (on the other hand, for larger bib files, you may actually want to exercise more control over which fields you are substituting.)
I'm trying to write a quick script to open a family document, change the parameter group of 2 specified parameters, and then close and save the document. I've done multiple tests and I am able to change the parameter groups of the specified parameters, but the changes of the groups don't save back to the family file. When I open the newly saved family, the parameter groups revert back to their original group.
This is with Revit 2017.2.
The same script, when run in RPS in Revit 2018 will do as desired.
import clr
import os
clr.AddReference('RevitAPI')
clr.AddReference('RevitAPIUI')
from Autodesk.Revit.DB import *
from Autodesk.Revit.UI import UIApplication
from System.IO import Directory, SearchOption
searchstring = "*.rfa"
dir = r"C:\Users\dboghean\Desktop\vanity\2017"
docs = []
if Directory.Exists(dir):
files = Directory.GetFiles(dir, searchstring, SearchOption.AllDirectories)
for f in files:
name, extension = os.path.splitext(f)
name2, extension2 = os.path.splitext(name)
if extension2:
os.remove(f)
else:
docs.append(f)
else:
print("Directory does not exist")
doc = __revit__.ActiveUIDocument.Document
app = __revit__.Application
uiapp = UIApplication(app)
currentPath = doc.PathName
pgGroup = BuiltInParameterGroup.PG_GRAPHICS
for i in docs:
doc = app.OpenDocumentFile(i)
paramList = [i for i in doc.FamilyManager.Parameters]
t = Transaction(doc, "test")
t.Start()
for i in paramList:
if i.Definition.Name in ["Right Sidesplash Edge line", "Left Sidesplash Edge line"]:
i.Definition.ParameterGroup = pgGroup
t.Commit()
doc.Close(True)
Any ideas?
Thanks!
I can confirm that this happens in Revit 2017. Strange!
A simple way around it is to arbitrarily rename the parameter using doc.FamilyManager.RenameParameter, then rename it back to the original name.
So in your case this would be three additional lines of code after changing the Parameter group:
originalName = i.Definition.Name
doc.FamilyManager.RenameParameter(i, "temp")
doc.FamilyManager.RenameParameter(i, originalName)
Doesnt get to the root problem, but works around it