I am currently trying to upload a CSV file to a proprietary service via httr::POST(). Unfortunately the Admins are not experienced in R and can give only little support.
This is an example on how it should look like in the command line:
curl -X POST --header 'Content-Type: multipart/form-data' \
-F file=#"member-1-test.csv.gz" 'https://some/api/endpoint'
So, in the following code I just try to stick with the example (and additionally provide a token).
> library(tidyverse)
> library(httr)
# Provide some test data with characters specifically quoted
> test_file <- tibble::tribble(
~keytype, ~key, ~action, ~segment,
6,"\"https://www.google.com\"", 0, 37372818,
6,"\"https://www.sport1.de\"" , 0, 37372818
)
> data.table::fwrite(test_file, "test.csv", quote = FALSE)
> file <- upload_file(path = "C:/R/projects/DefaultWorkingDirectory/test.csv")
> res <- POST(
url = "https://some/api/endpoint",
body = list(file = file),
add_headers(.headers = c('Content-Type' = "multipart/form-data", "Authorization" = token))
)
This gives me the follwing error:
> res
Response [https://some/api/endpoint]
Date: 2020-11-18 09:35
Status: 503
Content-Type: text/plain
Size: 95 B
> content(res, encoding = "UTF8")
"upstream connect error or disconnect/reset before headers. reset reason: connection termination"
Any help or guidance on how to move forward with this issue is very much appreciated. Thanks!
Directly after posting the question it was already solved by one of the Admins :)
The issue was that the encoding needs to be set to "multipart" and that a specific content type needs to be provided which is similar to JSON but needs to be added to the "Accept"-header field.
Here the answer for any future references:
> res <- POST(
url = "https://some/api/endpoint",
body = list(
file = upload_file("test.csv")
),
add_headers(c(
"Authorization" = token,
"Accept" = "specific_format_for_the_application"
)),
encode = "multipart",
verbose()
)
I am trying to wrap a function into REST API using plumber R Package. As a Input , function takes a shapefile and returns shapefile in terms of .shp format as well as GeoJSON format after the transformation. With the help of decorator now I need to enhance it into a web service.
R File:
#* Spatial Polygon Object
#* #param pathtoshpfile:character Path to Shapefile with Name
#* #param design:character one or two
#* #post /sprayermap
function(pathtoshpfile, design = c("one", "two")) {
# library
require(rgeos)
require(sp)
require(rgdal)
require(raster)
#Importing Shapefile
a_shape <- raster::shapefile(pathtoshpfile)
if (class(a_shape) == "SpatialPolygonsDataFrame") {
if (design == "one") {
a_shape <- tryCatch (
rgeos::gBuffer(a_shape, byid = TRUE, width = 0),
error = function(err) {
return(paste("sprayer map : ", err))
}
)
sprayer_map <- tryCatch (
aggregate(a_shape, "Rx"),
error = function(err) {
return(paste("sprayer map : ", err))
}
)
sprayer_map#data$Rx <- as.integer(sprayer_map#data$Rx)
raster::shapefile(sprayer_map, filename = "field_sprayer_map", overwrite =
TRUE)
rgdal::writeOGR(
sprayer_map,
dsn = "field_sprayer_map.GeoJSON",
layer = "geojson",
driver = "GeoJSON",
overwrite_layer = TRUE
)
return(paste0("SpatialPolygonsDataFrame"))
} else {
return(paste0("design two"))
}
} else {
return(paste0("Please provide spatial polygon object !"))
}
}
Plumber Part :
library(plumber)
# 'plumber.R' is the location of the file shown above
pr("plumber.R") %>%
pr_run(port=8000)
#############################
curl "http://127.0.0.1:8000/sprayermap?pathtoshpfile=/path/to/directory/test.shp&design=one"
Based on my limited understanding on creation of APIs , I created above script and got below error.
Error: unexpected string constant in "curl "http://127.0.0.1:8000/sprayermap?
Though plain R Function script works good
Looking for guidance how the decorators can be used to convert the above function into Restful APIs where Input and output both are shapefiles.
Test Shape File: Test Shape File
I believe your function is setup correctly. The following curl calls should help you out:
curl -X POST "http://127.0.0.1:8000/sprayermap?pathtoshpfile=../Downloads/jnk/test.shp&design=one"
(ie. insert -X POST in your existing call). Alternatively, via --data:
curl --data "pathtoshpfile=../Downloads/jnk/test.shp" "http://127.0.0.1:8000/sprayermap"
On my machine, both queries return
["SpatialPolygonsDataFrame"]
and field_sprayer_map is written to disk. For the record, this works with both the current plumber CRAN version 0.4.6:
p = plumb("../Downloads/jnk/plumber.R")
p$run(port=8000)
And the upcoming v1.0.0 release candidate:
pr("../Downloads/jnk/plumber.R") %>%
pr_run(port=8000)
I would like to send a mail using SMTPS in R. Currently, non of the available packages supports sending Mails via TLS (rmail & sendmaileR) or they have a hard to install Java dependency (mailr). I tried using curl and managed to send a mail using the following code snippet:
curl --url 'smtps://mail.server.com:465' --ssl-reqd --mail-from 'mail1#example.com' --mail-rcpt 'mail2#example.com' --upload-file mail.txt --user 'user:password'
Unfortunately, I could not translate that snippet into R using the brilliant curl package. While I managed to find all options, the curl statement crashes the R session every time. Furthermore, I was not able to add the mail.txt file to the request which I created in a temporary directory. Did someone manage sending mails using the curl package? Why does the program always crash? The goal should be to send mails on all platforms.
# input variables
to <- "mail1#example.com"
from <- Sys.getenv("MAIL_USER")
password <- Sys.getenv("MAIL_PASSWORD")
server <- Sys.getenv("MAIL_SERVER")
port <- 465
subject <- "Test Mail"
message <- c("Hi there!",
"This is a test message.",
"Cheers!")
# compose email body
header <- c(paste0('From: "', from, '" <', from, '>'),
paste0('To: "', to, '" <', to, '>'),
paste0('Subject: ', subject))
body <- c(header, "", message)
# create tmp file to save mail text
mail_file <- tempfile(pattern = "mail_", fileext = ".txt")
file_con <- file(mail_file)
writeLines(body, file_con)
close(file_con)
# define curl options
handle <- curl::new_handle()
curl::handle_setopt(handle = handle,
mail_from = from,
mail_rcpt = to,
use_ssl = TRUE,
port = port,
userpwd = paste(from, password, sep = ":"))
con <- curl::curl(url = server, handle = handle)
open(con, "r")
close(con)
# delete file
unlink(mail_file)
I'm using asterisk/freepbx. Asterisk is not recording calls that are fetched from park. Here is my call flow.
A calls to Asterisk Server(AS)
Call is picked up by extension B
B does an attended transfer by dialling *2200 (200 is my default parking lot)
C dials 1 to fetch the parked call
C dials *1 to record the call.
Recording is not done.
I found a fix and it works for me. In the asterisk log i found that asterisk tries to record to an invalid file with no filename, just an extension(.wav). It executed 2 files - /var/lib/asterisk/agi-bin/parkfetch.agi and /var/lib/asterisk/bin/one_touch_record.php.
one_touch_record.php generates filename from data read from channel, like year, date, mixmonitor folder etc, but as there was no valid filename in the log, these should be null here.
$mixMonDir = getVariable($channel, "MIXMON_DIR");
$year = getVariable($channel, "YEAR");
$month = getVariable($channel, "MONTH");
$day = getVariable($channel, "DAY");
$mixMonFormat = getVariable($channel, "MIXMON_FORMAT");
$mixMonPost = getVariable($channel, "MIXMON_POST");
$astman->mixmonitor($channel, "{$mixMonDir}{$year}/{$month}/{$day}/{$callFileName}.{$mixMonFormat}", "a", $mixMonPost, rand());
So i inspected the parkfetch.agi were i found that these channel vars are copied only if REC_STATUS is "RECORDING" and in this case REC_STATUS is "INITIALIZED". So i added an OR clause ie i changed if ($rec_status == "RECORDING") to if ($rec_status == "RECORDING" || $rec_status=="INITIALIZED")
if ($channel) {
$rec_status = get_var("IMPORT($channel,REC_STATUS)");
$agi->set_variable('REC_STATUS', $rec_status);
if ($rec_status == "RECORDING" || $rec_status=="INITIALIZED") {
foreach (array('MIXMON_DIR', 'YEAR', 'MONTH', 'DAY', 'CALLFILENAME', 'MIXMON_FORMAT', 'MIXMON_POST', 'MON_FMT') as $v) {
$agi->set_variable($v, get_var("IMPORT($channel,$v)"));
}
}
}
And it worked. Now when I pressed *1 after fetching call from park, it is getting recorded.
If someone found a better solution, please leave it as a comment in my blog.
http://sachindotg.blogspot.in/2014/02/asterisk-cannot-record-call-after.html
The following function (found here) works well for messages containing ASCII characters. Can you help me modify it for multilingual messages too, because I don't know python at all?
send.email <- function(to, from, subject,
message, attachment=NULL,
username, password,
server="xxx.xxx.xxx.xxx",
confirmBeforeSend=FALSE){
# to: a list object of length 1. Using list("Recipient" = "recip#somewhere.net") will send the message to the address but
# the name will appear instead of the address.
# from: a list object of length 1. Same behavior as 'to'
# subject: Character(1) giving the subject line.
# message: Character(1) giving the body of the message
# attachment: Character(1) giving the location of the attachment
# username: character(1) giving the username. If missing and you are using Windows, R will prompt you for the username.
# password: character(1) giving the password. If missing and you are using Windows, R will prompt you for the password.
# server: character(1) giving the smtp server.
# confirmBeforeSend: Logical. If True, a dialog box appears seeking confirmation before sending the e-mail. This is to
# prevent me to send multiple updates to a collaborator while I am working interactively.
if (!is.list(to) | !is.list(from)) stop("'to' and 'from' must be lists")
if (length(from) > 1) stop("'from' must have length 1")
if (length(to) > 1) stop("'send.email' currently only supports one recipient e-mail address")
if (length(attachment) > 1) stop("'send.email' can currently send only one attachment")
if (length(message) > 1){
stop("'message' must be of length 1")
message <- paste(message, collapse="\\n\\n")
}
if (is.null(names(to))) names(to) <- to
if (is.null(names(from))) names(from) <- from
if (!is.null(attachment)) if (!file.exists(attachment)) stop(paste("'", attachment, "' does not exist!", sep=""))
if (missing(username)) username <- winDialogString("Please enter your e-mail username", "")
if (missing(password)) password <- winDialogString("Please enter your e-mail password", "")
require(rJython)
rJython <- rJython()
rJython$exec("import smtplib")
rJython$exec("import os")
rJython$exec("from email.MIMEMultipart import MIMEMultipart")
rJython$exec("from email.MIMEBase import MIMEBase")
rJython$exec("from email.MIMEText import MIMEText")
rJython$exec("from email.Utils import COMMASPACE, formatdate")
rJython$exec("from email import Encoders")
rJython$exec("import email.utils")
mail<-c(
#Email settings
paste("fromaddr = '", from, "'", sep=""),
paste("toaddrs = '", to, "'", sep=""),
"msg = MIMEMultipart()",
paste("msg.attach(MIMEText('", message, "'))", sep=""),
paste("msg['From'] = email.utils.formataddr(('", names(from), "', fromaddr))", sep=""),
paste("msg['To'] = email.utils.formataddr(('", names(to), "', toaddrs))", sep=""),
paste("msg['Subject'] = '", subject, "'", sep=""))
if (!is.null(attachment)){
mail <- c(mail,
paste("f = '", attachment, "'", sep=""),
"part=MIMEBase('application', 'octet-stream')",
"part.set_payload(open(f, 'rb').read())",
"Encoders.encode_base64(part)",
"part.add_header('Content-Disposition', 'attachment; filename=\"%s\"' % os.path.basename(f))",
"msg.attach(part)")
}
#SMTP server credentials
mail <- c(mail,
paste("username = '", username, "'", sep=""),
paste("password = '", password, "'", sep=""),
#Set SMTP server and send email, e.g., google mail SMTP server
paste("server = smtplib.SMTP('", server, "')", sep=""),
"server.ehlo()",
"server.starttls()",
"server.ehlo()",
"server.login(username,password)",
"server.sendmail(fromaddr, toaddrs, msg.as_string())",
"server.quit()")
message.details <-
paste("To: ", names(to), " (", unlist(to), ")", "\n",
"From: ", names(from), " (", unlist(from), ")", "\n",
"Using server: ", server, "\n",
"Subject: ", subject, "\n",
"With Attachments: ", attachment, "\n",
"And the message:\n", message, "\n", sep="")
if (confirmBeforeSend)
SEND <- winDialog("yesnocancel", paste("Are you sure you want to send this e-mail to ", unlist(to), "?", sep=""))
else SEND <- "YES"
if (SEND %in% "YES"){
jython.exec(rJython,mail)
cat(message.details)
}
else cat("E-mail Delivery was Canceled by the User")
}
I call it like this:
send.email(list("somebody#somecompany.com"),
list("rJythonMail#mycompany.com"),
"Δοκιμή αποστολής email με attachment",
"Με χρήση της rJython",
attachment="monthly_report.xls",
username="gd047",password="xxxxxx")
The issue is how your enclosed Python code is structured. This blog entry goes into more detail on how to properly send Unicode email, which you're using Python to manage the underling SMTP connection for. Note that according to R. David Murray, this will be fixed in later iterations of the email package in Python.
Here's the salient Python code you can retrofit into your calls in rJython, borrowing directly from the aforementioned blog post:
from smtplib import SMTP
from email.MIMEText import MIMEText
from email.Header import Header
from email.Utils import parseaddr, formataddr
def send_email(sender, recipient, subject, body):
"""Send an email.
All arguments should be Unicode strings (plain ASCII works as well).
Only the real name part of sender and recipient addresses may contain
non-ASCII characters.
The email will be properly MIME encoded and delivered though SMTP to
localhost port 25. This is easy to change if you want something different.
The charset of the email will be the first one out of US-ASCII, ISO-8859-1
and UTF-8 that can represent all the characters occurring in the email.
"""
# Header class is smart enough to try US-ASCII, then the charset we
# provide, then fall back to UTF-8.
header_charset = 'ISO-8859-1'
# We must choose the body charset manually
for body_charset in 'US-ASCII', 'ISO-8859-1', 'UTF-8':
try:
body.encode(body_charset)
except UnicodeError:
pass
else:
break
# Split real name (which is optional) and email address parts
sender_name, sender_addr = parseaddr(sender)
recipient_name, recipient_addr = parseaddr(recipient)
# We must always pass Unicode strings to Header, otherwise it will
# use RFC 2047 encoding even on plain ASCII strings.
sender_name = str(Header(unicode(sender_name), header_charset))
recipient_name = str(Header(unicode(recipient_name), header_charset))
# Make sure email addresses do not contain non-ASCII characters
sender_addr = sender_addr.encode('ascii')
recipient_addr = recipient_addr.encode('ascii')
# Create the message ('plain' stands for Content-Type: text/plain)
msg = MIMEText(body.encode(body_charset), 'plain', body_charset)
msg['From'] = formataddr((sender_name, sender_addr))
msg['To'] = formataddr((recipient_name, recipient_addr))
msg['Subject'] = Header(unicode(subject), header_charset)
# Send the message via SMTP to localhost:25
smtp = SMTP("localhost")
smtp.sendmail(sender, recipient, msg.as_string())
smtp.quit()
Once ported, this should get your code working. Hopefully, better support for RFC2047 and RFC3490 will be added soon. For now, you regrettably have to hack around the issue.