When I run this command in RStudio,
library(elastic)
connect("192.12.32.2")
The following message/output comes:
Found http or https on es_host, stripping off, see the docs
transport: http
host: 192.12.32.2
port: 9200
path: NULL
username: NULL
password: [secret]
errors: simple
headers (names): NULL
After this I ran this command,
res <- Search(index = 'epgs_20*', fields = c("ts", "ma", "pi", "tn", "Ope"),scroll="5m",search_type = "scan")
and I get the following error:
Error: 404 - IndexMissingException[[epgs_20%2A] missing].
If instead of above index = 'epgs_20*', we use index = 'epgs_2016.2.07'. It works fine but when using * it doesn't work.
How do I run the above command with index = 'epgs_20*' ?
Related
I have a few scrapes via RSelenium scheduled. Sometimes the scraping failes and i would like to know the reason.
I note that the error Messages (in red) are quite informative, but i dont know how to log them.
Lets say i provided a "non well formed URL".:
tryCatch(
expr = remDr$navigate("i.am.not.an.url"),
error = function(error){
print(error)
# write.table(error, file = ...)
}
)
This is what i get, but it doesnt give much specification on what triggered the error
<simpleError: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
class: org.openqa.selenium.WebDriverException
Further Details: run errorDetails method>
This is more informative - but i dont manage to log it.
Selenium message:Target URL i.am.not.an.url is not well-formed.
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: '9bc48e7a4511', ip: '172.17.0.4', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-1087-aws', java.version: '1.8.0_91'
Driver info: driver.version: unknown
What i tried:
Using the Error Handling Class. It includes very detailed specification of the error Messages and its meanings, but i dont manage to extract them given my current error.
errHandle = errorHandler(remDr)
errHandle$checkStatus(remDr)
errHandle$checkError(res = remDr)
Using a message handler from another #SO question:
messageHandler <- function(fun, ...) {
zz <- textConnection("foo", "w", local = TRUE)
sink(zz, type = "message")
res <- fun(...)
sink(type = "message")
close(zz)
#handle messages
list(res, messages = foo)
}
wrongURL <- function() {
remDr$navigate("mistake")
}
messageHandler(fun = wrongURL)
I found a way via errorDetails():
tryCatch(
expr = remDr$navigate("i.am.not.an.url"),
error = function(error){
return(remDr$errorDetails()$localizedMessage)
}
)
I'm working on a project that requires accessing webpages and I do this via
rvest::html_session(). For documentation and training I would like to set
up a reproducible example and have considered the following.
Use servr::httd(system.file("egwebsite", package = "<pkgname>"), daemon =
TRUE, browser = FALSE) to set up a simple HTTP server
Use rvest::html_session("http://127.0.0.1:4321") to set up the html
session.
However, the following simple example behaves differently on Linux (Debian 9)
and Windows 10. (I do not have easy access to OSx and have not tested on
that OS).
# On Windows
servr::httd(daemon = TRUE, browser = FALSE, port = 4321)
## Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321
## To stop the server, run servr::daemon_stop("94019719908480") or restart your R session
R.utils::withTimeout(
{
s <- rvest::html_session("http://127.0.0.1:4321")
},
timeout = 3,
onTimeout = "error")
s
## <session> http://127.0.0.1:4321/
## Status: 200
## Type: text/html
## Size: 2352
servr::daemon_stop()
However, on my Linux box (Debian 9) I get the following
servr::httd(daemon = TRUE, browser = FALSE, port = 4321)
## Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321
## To stop the server, run servr::daemon_stop("94019719908480") or restart your R session
R.utils::withTimeout(
{
s <- rvest::html_session("http://127.0.0.1:4321")
},
timeout = 3,
onTimeout = "error")
## Error: reached elapsed time limit
## Error in curl::curl_fetch_memory(url, handle = handle) :
## Operation was aborted by an application callback
That is, I am unable to create a html_session in the same R interactive
session that spawned the http server. If, however, I start a second R
session while the leaving the initial session running, I am able to create
the html_session without error.
What can I do so that I can create an html_session based on a servr::httd
HTTP server within the same R session on Linux?
Edit 1
If I add httr::verbose() to the html_session call I get the following when the session is created successfully. When the process hangs and fails to create the session the output stops on the last -> and none of the lines with <- are shown.
> s <- html_session("http://127.0.0.1:4321", httr::verbose())
-> GET / HTTP/1.1
-> Host: 127.0.0.1:4321
-> User-Agent: libcurl/7.52.1 r-curl/3.1 httr/1.3.1
-> Accept-Encoding: gzip, deflate
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 200 OK
<- Content-Type: text/html
<- Content-Length: 61303
<-
I have found a solution to my problem, run servr::httd in a subprocess. This solution requires the subprocess package.
First, a helper function R_binary will return the file path for the R binary on Windows or unix based OS.
R_binary <- function () {
R_exe <- ifelse (tolower(.Platform$OS.type) == "windows", "R.exe", "R")
return(file.path(R.home("bin"), R_exe))
}
Next, start R vanilla as a subprocess.
subR <- subprocess::spawn_process(R_binary(), c("--vanilla"))
Then start the HTTP server in the subprocess
subprocess::process_write(subR, 'servr::httd(".", browser = FALSE, port = 4321)\n')
## [1] 47
subprocess::process_read(subR)$stderr
## [1] "Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321"
A quick test to show that there is communication between the active R session and the HTTP server:
session <- rvest::html_session("http://127.0.0.1:4321")
session
## <session> http://127.0.0.1:4321/
## Status: 200
## Type: text/html
## Size: 1054
And finally, kill the subprocess
subprocess::process_kill(subR)
I've updated my elasticsearch to version 5.x! And tried to connect my R Studio(with elastic package). But it gives me an error Error: 404 - no such index.
I connect like this:
elastic::connect(es_host = "http://kibana.smasell.com", es_port = "9200/es", es_user = "****", es_pwd = "****")
It gives me:
transport: http
host: kibana.smasell.com
port: 9200/es
path: NULL
username: *****
password: <secret>
errors: simple
headers (names): NULL
And then try to search:
elastic::Search(index = "data_month_2016-01", size =1)
gives me
Error: 404 - no such index
Can somebody help me?
I am using openresty with lua. When I was loading torch without cutorch,cunn(GPU) it was working fine. But it is unable to load cutorch module.
Here is my nginx.conf
error_log stderr notice;
daemon off;
events{}
http {
lua_code_cache on;
#lua_package_cpath '/usr/local/cuda/include/?.so;;';
init_by_lua_file 'evaluate.lua';
server {
listen 6788;
lua_code_cache on;
lua_need_request_body on;
client_body_buffer_size 10M;
client_max_body_size 10M;
location /ff {
# only permit POST requests
if ($request_method !~ ^(POST)$ ) {
return 403;
}
content_by_lua '
local body = ngx.req.get_body_data()
if body == nil then
ngx.say("empty body")
else
local resp =FeedForward(body)
ngx.say(cjson.encode({result=resp}))
end
';
}
}
}
Here is my evaluate.lua code
-- load the required lua models
torch = require("torch")
nn = require("nn")
gm = require("image")
cutorch = require "cutorch"
cunn = require "cunn"
cjson = require("cjson")
-- model
modelPath='./model.t7'
ninputs = 3
model = nn.Sequential()
model:add(nn.Linear(ninputs,2))
-- let’s save a dummy model
-- to demonstrate the functionality
torch.save(modelPath, model)
-- load a pretrained model
net = torch.load(modelPath)
net:cuda()
net:evaluate()
-- our end point function
-- this function is called by the ngx server
-- accepts a json body
function FeedForward(json)
print("starting")
-- decode and extract field x
local data = cjson.decode(json)
local input = torch.Tensor(data.x)
local response = {}
-- example checks
if input == nil then
print("No input given")
elseif input:size(1) ~=ninputs then
print("Wrong input size")
else
-- evaluat the input and create a response
local output = net:forward(input:cuda()):float()
-- from tensor to table
for i=1,output:size(1) do
response[i] = output[i]
end
end
-- return response
return response
end
I am trying to run the model using following commands
/usr/local/openresty/nginx/sbin/nginx -p "$(pwd)" -c "nginx.conf"
It's starting up fine but when I am sending curl request like
curl -H "Content-Type: application/json" -X POST -d '{"x":[1,2,3]}' http://localhost:6788/ff
I am getting following error.
2016/09/29 12:59:59 [notice] 10355#0: *1 [lua] evaluate.lua:28: FeedForward(): starting, client: 127.0.0.1, server: , request: "POST /ff HTTP/1.1", host: "localhost:6788"
THCudaCheck FAIL file=/tmp/luarocks_cutorch-scm-1-700/cutorch/lib/THC/generic/THCStorage.cu line=40 error=3 : initialization error
2016/09/29 12:59:59 [error] 10355#0: *1 lua entry thread aborted: runtime error: unknown reason
stack traceback:
coroutine 0:
[C]: in function 'resize'
/home/ubuntu/torch/install/share/lua/5.1/cutorch/Tensor.lua:14: in function 'cuda'
/rootpath/evaluate.lua:41: in function 'FeedForward'
content_by_lua(nginx.conf:31):7: in function <content_by_lua(nginx.conf:31):1>, client: 127.0.0.1, server: , request: "POST /ff HTTP/1.1", host: "localhost:6788"
Without cutorch the model is running fine like If I remove
net:cuda()
and replace line
local output = net:forward(input:cuda()):float()
by
local output = net:forward(input):float()
It's working fine. Also I tried to run evaluate.lua using th and it's working fine there with cutorch and cunn packages.
While using elastic package in R, I'm getting an message while using connect("172.28.6.5").
Message details :
Found http or https on es_host, stripping off, see the docs .
After this when I am running the command :
res <- Search(index = 'abc_20*', fields = c("Seq_Num"),scroll="5m",search_type = "scan")
It gives me error message :
Error: 404 - IndexMissingException[[abc_20%2A] missing]
This error is only shown in my laptop.
How to resolve this issue?
Is that exactly what you did? I get no issues running that command.
library(elastic)
connect("172.28.6.5")
#> transport: http
#> host: 172.28.6.5
#> port: 9200
#> path: NULL
#> username: NULL
#> password: <secret>
#> errors: simple
#> headers (names): NULL
Looking at the source for elastic::connect(), assuming you've tried updating the package (this block traces back to April 2016)
# strip off transport if found
if (grepl("^http[s]?://", es_host)) {
message("Found http or https on es_host, stripping off, see the docs")
es_host <- sub("^http[s]?://", "", es_host)
}
(note: this is a message not an Error as you have in your question) suggests you're passing in something that regex matches to ^http[s]?://