I want to change my system's ip through R , is there any way this can be done ?
have tried below answer by Pablo Barbera but couldn't actually worked .
library(RCurl)
# check current IP address
print(getURL("http://ifconfig.me/ip"))
# proxy options
opts <- list(proxy="127.0.0.1", proxyport=8118)
# opening connection with TOR
con <- socketConnection(host="127.0.0.1",port=9051)
print(getURL("http://ifconfig.me/ip", .opts = opts))
for (i in 1:10)
{
writeLines('AUTHENTICATE \"password\"\r\nSIGNAL NEWNYM\r\n', con=con)
Sys.sleep(5)
print(getURL("http://ifconfig.me/ip", .opts = opts))
Sys.sleep(5)
}
LINK : Changing Tor identity in R
Can anybody make me understand what this code is saying and how this is wroking ?
Related
I am trying to create mapbox tiles using mapboxapi::tippecanoe() in R. Unfortunately, my work computer runs Windows 10, which greatly complicates what I am trying to do. Tippecanoe is a Unix executable, so I downloaded and installed Ubuntu and am running it on a Windows subsystem for Linux. To get tippecanoe to launch, I had to edit the source code of mapboxapi::tippecanoe() to pass arguments to WSL. I then ran into an issue where Tippecanoe would give me an error that it could not open database files. Some research on Github led me to believe that this was related the number of open files limit in Ubuntu. After a lot of digging, I was able to increase ulimit -n to 65535 for on my ubuntu terminal. As soon as I launch Ubuntu, if I type in ulimit -n, I get 65535. However, when I call `sytem2("wsl", "ulimit -n"), I get the default value of 1024. I thought this was due to the user that R was calling in Ubuntu, but running system2("wsl", "whoami") returned the username for who I increased both the hard and soft nofile limits for. I am really stumped. Apologies for not pasting a reproducible example, but I am not sure how to make one for this situation. Any help would be much appreciated. Thanks!
Well after a whole lot of tinkering, this ended up being mostly a straightforward R code issue. The ulimit issue may still have been a problem, but actually I need to fix the R code in mapboxapi::tippecanoe(). Because mapboxapi::tippecanoe()uses the system() command to call tippecanoe, not only did I need to change the call to invoke wsl through a login shell using system2("wsl", "- d Ubuntu -lc 'tippecanoe <arguments to tippecanoe>'"), but I also needed to change the paths that R sent to tippecanoe to be linux paths instead of Window paths. If anyone else is having trouble with this, here is the tweaked mapboxapi::tippecanoe() command code that actually worked for me:
tippecanoe2<-function (input, output, layer_name, min_zoom = NULL,
max_zoom = NULL, drop_rate = NULL, overwrite = TRUE, other_options = NULL,
keep_geojson = FALSE)
{
check_install <- system2("wsl", "tippecanoe -v") == 0
linux_dir<-paste(getwd(), layer_name, sep="/")#make a directory in your linux directory for the .mbtiles
parsed<-strsplit(linux_dir, split="/") #parse the windows directory path
n<-length(parsed[[1]])
dir_out<-paste("",parsed[[1]][n-1], parsed[[1]][n], sep="/") #construct the linux directory path
dir.create(linux_dir)
op<-options(useFancyQuotes = FALSE)
if (!check_install) {
rlang::abort(c("tippecanoe is not installed or cannot be found by the application you are using to run mapboxapi.",
"If you haven't installed tippecanoe, please visit https://github.com/mapbox/tippecanoe for installation instructions.",
"If you have installed tippecanoe, run `Sys.getenv('PATH')` and make sure your application can find tippecanoe. If it cannot, adjust your PATH accordingly."))
}
opts <- c()
if (!is.null(min_zoom)) {
opts <- c(opts, sprintf("-Z%s", min_zoom))
}
if (!is.null(max_zoom)) {
opts <- c(opts, sprintf("-z%s", max_zoom))
}
if (is.null(min_zoom) && is.null(max_zoom)) {
opts <- c(opts, "-zg")
}
if (!is.null(drop_rate)) {
opts <- c(opts, sprintf("-r%s", drop_rate))
}
else {
opts <- c(opts, "-as")
}
if (overwrite) {
opts <- c(opts, "-f")
}
collapsed_opts <- paste0(opts, collapse = " ")
if (!is.null(other_options)) {
extra_opts <- paste0(other_options, collapse = " ")
collapsed_opts <- paste(collapsed_opts, extra_opts)
}
dir <- linux_dir
if (any(grepl("^sf", class(input)))) {
input <- sf::st_transform(input, 4326)
if (is.null(layer_name)) {
layer_name <- stringi::stri_rand_strings(1, 6)
}
if (keep_geojson) {
outfile <- paste0(layer_name, ".geojson")
path <- file.path(dir_out, outfile)
sf::st_write(input, path, quiet = TRUE, delete_dsn = TRUE,
delete_layer = TRUE)
}
else {
tmp <- tempdir("//wsl$/Ubuntu/tmp")#Here you would need to tweak to the file path for your linux distribution's temporary directory
tempfile <- paste0(layer_name, ".geojson")
path <- file.path(tmp, tempfile)
sf::st_write(input, path, quiet = TRUE, delete_dsn = TRUE,
delete_layer = TRUE)
}
call <- sprintf("tippecanoe -o %s/%s %s %s", dir_out, output,
collapsed_opts, path)
call2<-paste("-d Ubuntu /bin/bash -lc", sQuote(call, op), sep=" ")
system2("wsl", call2)
}
else if (inherits(input, "character")) {
if (!is.null(layer_name)) {
collapsed_opts <- paste0(collapsed_opts, " -l ",
layer_name)
}
call <- sprintf("tippecanoe -o %s/%s %s %s", dir_out, output,
collapsed_opts, input)
call2<-paste("-d Ubuntu /bin/bash -lc", sQuote(call, op), sep=" ")
system2("wsl", call2)
}
}
I am trying to use R to access postgresql db on Heroku, and I found that I can use src_dbi from dplyr package.
I have dplyr properly installed, but when I try to call src_dbi I get an error message:
Error in src_dbi(db_con) : could not find function "src_dbi"
This happens right when I run:
db <- src_dbi(db_con)
after poviding the credentials:
config <- run("heroku", c("config:get", "postgres://xxxxxxxxxxxxxx
", "-a", "prjectAlpha"))
pg <- httr::parse_url(config$stdout)
dbConnect(RPostgres::Postgres(),
dbname = "xxxxxxxxxx",
host = "xxxxxxxxx.amazonaws.com",
port = 5432,
user = "xxxxxx",
password = "xxxxxxxxxxxxxxxxx",
sslmode = "require"
) -> db_con
The idea is to be able to download a table and re-upload it after making a few changes with R.
My solution to access Heroku Postgresql from R:
library(dbplyr) #in case you have an error, run: system("defaults write org.R-project.R force.LANG en_US.UTF-8") from Rails console, then restart R.
library(processx)
library(RPostgres)
library(httr)
library(tidyverse)
library(dplyr)
config <- run("heroku", c("config:get", "postgres://xxxxxxxxxxxxxx
", "-a", "prjectAlpha"))
pg <- httr::parse_url(config$stdout)
dbConnect(RPostgres::Postgres(),
dbname = "xxxxxxxxxx",
host = "xxxxxxxxx.amazonaws.com",
port = 5432,
user = "xxxxxx",
password = "xxxxxxxxxxxxxxxxx",
sslmode = "require"
) -> db_con
Once the connection is set up:
db <- src_dbi(db_con)
Once the connection is established, check for the available tables
db
Now time to retrive data
A lot of examples only show treatment directly from the collection, but one might just be interested to read the tables. I have a table called "weather_records"
weather_records_local_df <- tbl(db_con, "weather_records")
df <- collect(weather_records_local_df)
Then do what you want with the data. Hope it helps.
I would like to send a mail using SMTPS in R. Currently, non of the available packages supports sending Mails via TLS (rmail & sendmaileR) or they have a hard to install Java dependency (mailr). I tried using curl and managed to send a mail using the following code snippet:
curl --url 'smtps://mail.server.com:465' --ssl-reqd --mail-from 'mail1#example.com' --mail-rcpt 'mail2#example.com' --upload-file mail.txt --user 'user:password'
Unfortunately, I could not translate that snippet into R using the brilliant curl package. While I managed to find all options, the curl statement crashes the R session every time. Furthermore, I was not able to add the mail.txt file to the request which I created in a temporary directory. Did someone manage sending mails using the curl package? Why does the program always crash? The goal should be to send mails on all platforms.
# input variables
to <- "mail1#example.com"
from <- Sys.getenv("MAIL_USER")
password <- Sys.getenv("MAIL_PASSWORD")
server <- Sys.getenv("MAIL_SERVER")
port <- 465
subject <- "Test Mail"
message <- c("Hi there!",
"This is a test message.",
"Cheers!")
# compose email body
header <- c(paste0('From: "', from, '" <', from, '>'),
paste0('To: "', to, '" <', to, '>'),
paste0('Subject: ', subject))
body <- c(header, "", message)
# create tmp file to save mail text
mail_file <- tempfile(pattern = "mail_", fileext = ".txt")
file_con <- file(mail_file)
writeLines(body, file_con)
close(file_con)
# define curl options
handle <- curl::new_handle()
curl::handle_setopt(handle = handle,
mail_from = from,
mail_rcpt = to,
use_ssl = TRUE,
port = port,
userpwd = paste(from, password, sep = ":"))
con <- curl::curl(url = server, handle = handle)
open(con, "r")
close(con)
# delete file
unlink(mail_file)
I've been trying to download pdfs embedded in a map following this code (original one can be found here). Each pdf refers to a brazilian municipality (5,570 files).
library(XML)
library(RCurl)
url <- "http://simec.mec.gov.br/sase/sase_mapas.php?uf=RJ&tipoinfo=1"
page <- getURL(url)
parsed <- htmlParse(page)
links <- xpathSApply(parsed, path="//a", xmlGetAttr, "href")
inds <- grep("*.pdf", links)
links <- links[inds]
regex_match <- regexpr("[^/]+$", links, perl=TRUE)
destination <- regmatches(links, regex_match)
for(i in seq_along(links)){
download.file(links[i], destfile=destination[i])
Sys.sleep(runif(1, 1, 5))
}
I already used this code in other projects a few times and it worked. For this specific case, it doesn't. In fact, I've tried many things to scrape these files but it seems impossible to me. Recently, I got the following link. Then it makes possible to combine uf (state) and muncod (municipal code) to download the file, but I dont know how to include this to the code though.
http://simec.mec.gov.br/sase/sase_mapas.php?uf=MT&muncod=5100102&acao=download
Thanks in advance!
devtools::install_github("ropensci/RSelenium")
library(rvest)
library(httr)
library(RSelenium)
# connect to selenium server from within r (REPLACE SERVER ADDRESS)
rem_dr <- remoteDriver(
remoteServerAddr = "192.168.50.25", port = 4445L, browserName = "firefox"
)
rem_dr$open()
# get the two-digit state codes for brazil by scraping the below webpage
tables <- "https://en.wikipedia.org/wiki/States_of_Brazil" %>%
read_html() %>%
html_table(fill = T)
states <- tables[[4]]$Abbreviation
# for each state, we are going to go navigate to the map of that state using
# selenium, then scrape the list of possible municipality codes from the drop
# down menu present in the map
get_munip_codes <- function(state) {
url <- paste0("http://simec.mec.gov.br/sase/sase_mapas.php?uf=", state)
rem_dr$navigate(url)
# have to wait until the drop down menu loads. 8 seconds will be enough time
# for each state
Sys.sleep(8)
src <- rem_dr$getPageSource()
out <- read_html(src[[1]]) %>%
html_nodes(xpath = "//select[#id='muncod']/option[boolean(#value)]") %>%
xml_attrs("value") %>%
unlist(use.names = F)
print(state)
out
}
state_munip <- sapply(
states, get_munip_codes, USE.NAMES = TRUE, simplify = FALSE
)
# now you can download each pdf. first create a directory for each state, where
# the pdfs for that state will go:
lapply(names(state_munip), function(x) dir.create(file.path("brazil-pdfs", x)))
# ...then loop over each state/municipality code and download the pdf
lapply(
names(state_munip), function(state) {
lapply(state_munip[[state]], function(munip) {
url <- sprintf(
"http://simec.mec.gov.br/sase/sase_mapas.php?uf=%s&muncod=%s&acao=download",
state, munip
)
file <- file.path("brazil-pdfs", state, paste0(munip, ".pdf"))
this_one <- paste0("state ", state, ", munip ", munip)
tryCatch({
GET(url, write_disk(file, overwrite = TRUE))
print(paste0(this_one, " downloaded"))
},
error = function(e) {
print(paste0("couldn't download ", this_one))
try(unlink(file, force = TRUE))
}
)
})
}
)
STEPS:
Get the IP address of your windows machine (see https://www.digitalcitizen.life/find-ip-address-windows)
start selenium server docker container by running this:
docker run -d -p 4445:4444 selenium/standalone-firefox:2.53.1
start rocker/tidyverse docker container by running this:
docker run -v `pwd`/brazil-pdfs:/home/rstudio/brazil-pdfs -dp 8787:8787 rocker/tidyverse
Go into your preferred browser and enter this address: http://localhost:8787 ...This will take you to the login screen for rstudio server. login using the username "rstudio" and password "rstudio"
Copy/paste the code shown above in a new Rstudio .R document. Replace the value for remoteServerAddr with the IP address you found in step 1.
Run the code...this should write the pdfs to a directory "brazil-pdfs" that is both inside the container and mapped to your windows machine (in other words, the pdfs will show up in the brazil-pdfs dir on your local machine as well). note, it takes a while to run the code b/c there are a lot of pdfs.
I have a connection with an ftp server with the following code:
url <- "ftp://MyServer"
userpwd <- "MyUser:MyPass"
filenames <- getURL(url, userpwd = userpwd, ftp.use.epsv = FALSE, dirlistonly = TRUE, port = 22)
filen <- "MyFile.csv"
rawdata <- getURL(paste(url, filen, sep = ""), userpwd = userpwd, crlf = TRUE)
The file will be moved to an SFTP server, so I need to change the input. This new SFTP server is accessed via port 22 instead of the standard port 21. At the moment the connection fails with the following error
Error in function (type, msg, asError = TRUE) :
Failed to connect to MyServer port 21: Connection refused
It takes the wrong port, but how do I tell R to choose port 22?
You need to specify the SFTP protocol in the URL, so the line
url <- "ftp://MyServer"
should become
url <- "sftp://MyServer"
getUrl will then use the SSH port (22).