google app engine flexible with docker 'site cannot be reached' - r

I want to server R model using App Engine.
I am following this example R with app engine, but stuck. I tried several methods but still have issues. Any guidance on this issue?
Please refer to my code
app.yaml
runtime: custom
env: flex
Dockerfile
FROM gcr.io/gcer-public/plumber-appengine
LABEL maintainer="mark"
RUN R -e "install.packages(c('plumber'), repos='http://cran.rstudio.com/')"
WORKDIR /payload/
COPY [".", "./"]
EXPOSE 8080
ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8080)"]
CMD ["schedule.R"]
schedule.R
#* #get /demoR
get_predict_length <- function(){
dataset <- iris
# create the model
model <- lm(Petal.Length ~ Petal.Width, data = dataset)
petal_width = "0.4"
#petal_width = '0.4'
# convert the input to a number
petal_width <- as.numeric(petal_width)
#create the prediction data frame
prediction_data <- data.frame(Petal.Width=petal_width)
# create the prediction
predict(model,prediction_data)
}
I deploy using 'gcloud app deploy and its successful. I get a link 'https://iris-custom-dot-my-project-name.appspot.com/'.
Final output in logs
Stackdriver logs show:
Starting server to listen on port 8080
when I click on app engine version https://iris-custom-dot-my-project-name.appspot.com/' , I get below message:
This site can’t be reached

Office network issue
In my case the real issue was, my office network blocked port 8080, so when I connected from home or in office over mobile hotspot, it worked.
In general follow below steps to resolve the issue.
1) search google 'my public IP address"
2) Add your IP in firewall rules will resolve the issue.
either use gcloud commands
https://cloud.google.com/sdk/gcloud/reference/app/firewall-rules/create
or
using GCP UI ( you can use any priority number which has not been used already)

Related

How to collect jolokia data via telegraf but just if the jolokia connection is active?

When my application is up, telegraf works fine and collects data related to jolokia since my application opens the port 11722 that telegraf uses to get the metrics. But then, when my application is down, telegraf starts to get errors since it can't connect to Jolokia. My telegraf version is 1.5.3 and this is a Production environment, so I don't have much flexibility to change the version. Is there a way to collect the jolokia metrics just when my application is up and running?
I've tried to create a script to check if jolokia was running and use with a tag that then I could use with my agent, but this didn't work:
[[inputs.exec]]
commands = ["sh /local/1/home/svcegctp/telegraf/inputs/scripts/check_jolokia.sh"]
timeout = "1s"
data_format = "influx"
name_override = "jvm_status"
[inputs.exec.tags]
running = "true"
(...)
[[inputs.jolokia2_agent]]
# Add agents URLs to query
urls = ["http://localhost:11722/jolokia"]
[inputs.jolokia2_agent.tags]
running = "true"
This is my script:
check_jolokia.sh
#!/bin/bash
if curl -s -u <username>:<password> http://localhost:11722/jolokia/version >/dev/null 2>&1; then
echo "jvm_status running=true"
else
echo "jvm_status running=false"
fi

plumberDeploy::do_provision() - Error: Size is not available in this region

I am trying to provision droplet on DigitalOcean with plumberDeploy R package.
I am running:
mydrop <- plumberDeploy::do_provision()
And getting back:
THIS ACTION COSTS YOU MONEY!
Provisioning a new server for which you will get a bill from DigitalOcean.
Using default ssh keys: work_R_laptop
Error: Size is not available in this region.
Any ideas how to troubleshoot?
it may not be a solution for the size issue, but you can specify the droplet-id or hostname as a paramter, so I can suggest that you manually create the droplet from DigitalOcean access panel and the run:
id <- plumberDeploy::do_provision("your droplet hostname", example=FALSE).
Then it should run the setup for the given droplet as expected.

Status code 500 : Container did not respond in time

I would like to run an R Shiny application via Shinyproxy locally first.
My Application is functional, after docking it, it still works with a docker command.
I created a ‘shinyproxy’ image and launched the container but I get the following error message when I click on the application name:
# Error
**Status code:** 500
**Message:** Container did not respond in time
I suspect a problem related to the different ports … notions that I don’t master.
The dockerfile of shiny app looks like :
with the following Rprofile :
local({
options(shiny.port = 3838, shiny.host = "0.0.0.0")
})
The shinyproxy application.yml looks like :
What I tried is the following command :
docker run -d -v /var/run/docker.sock:/var/run/docker.sock --net sp-net -p 8080:8080 shinyproxy_container
All help is welcome

Error while deploying R using plumber in Google App Engine Flex with Docker

What am doing?
I am trying to deploy a R model on Google App Engine Flex with docker container. My final objective is to serve model as API. I am getting errors when deploying app using plumber and docker container.
R code with plumber runs fine on my local computer using RStudio. But now I am using AI platform jupyter notebooks with R. I tested the docker locally using Docker Run image-name command and I get below message once Docker run.
Starting server to listen on port 8080
When I run the R + plumber code in my local Rstudio , I get below messages
Starting server to listen on port 8080
Running the swagger UI at http://127.0.0.1:8080/__swagger__/
After this I run gcloud app deploy ( this agains build docker image etc) , build runs for more than 15 mins and fails with error message , as shown in the end.
Details of code etc:
app.yaml
service: iris-custom
runtime: custom
env: flex
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 20
# added below to increase app_start_timeout_sec
readiness_check:
path: "/readiness_check"
check_interval_sec: 5
timeout_sec: 4
failure_threshold: 2
success_threshold: 2
app_start_timeout_sec: 900
Dockerfile
FROM gcr.io/gcer-public/plumber-appengine
# install the linux libraries needed for plumber
RUN export DEBIAN_FRONTEND=noninteractive; apt-get -y update \
&& apt-get install -y
# install plumber commented as plumber is preinstalled
#RUN R -e "install.packages(c('plumber'), repos='http://cran.rstudio.com/')"
# copy everything from the current directory into the container
WORKDIR /payload/
COPY [".", "./"]
# open port 8080 to traffic
EXPOSE 8080
# when the container starts, start the main.R script
ENTRYPOINT ["Rscript", "main.R"]
main.R
library(plumber)
r <- plumb("rest_controller.R")
r$run(port=8080, host="0.0.0.0")
rest_controller.R
#* #get /predict_petal_length
get_predict_length <- function(){
dataset <- iris
# create the model
model <- lm(Petal.Length ~ Petal.Width, data = dataset)
petal_width = "0.4"
# convert the input to a number
petal_width <- as.numeric(petal_width)
#create the prediction data frame
prediction_data <- data.frame(Petal.Width=petal_width)
# create the prediction
predict(model,prediction_data)
}
Error message:
ERROR: (gcloud.app.deploy) Error Response: [4] Your deployment has
failed to become healthy in the allotted time and therefore was rolled
back. If you believe this was an error, try adjusting the
'app_start_timeout_sec' setting in the 'readiness_check' section.
I tried a little modified code ,deployment succeeds but app engine still does not work.
issue with code link
From the Google Cloud Doku it seems like in order for your Apllication to pass it needs to return the http status code 200 (see https://cloud.google.com/appengine/docs/flexible/custom-runtimes/configuring-your-app-with-app-yaml#updated_health_checks).
But your Application returns the http status code 404 on the path you have defined for redincess check, since it doesn't exist.
readiness_check:
path: "/readiness_check"
So I would either suggest to add this path as an option to your rest_controller.R file like
#* #get /readiness_check
readiness_check<- function(){
return ("app ready")
}
or modify your app.yml so that it checks the get_predict_length enpoint instead
readiness_check:
path: "/get_predict_length"

How can I check that my Spark Cluster do work?

I have installed Spark 2.3.0 on Ubuntu 18.04 with two nodes: a master one (ip: 172.16.10.20) and a slave one (ip: 172.16.10.30). I can check that this Spark cluster looks like up and running
jps -lm | grep spark
14165 org.apache.spark.deploy.master.Master --host 172.16.10.20 --port 7077 --webui-port 8080
13701 org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://172.16.10.20:7077
I give it a try with this simple R script (using the sparklyr package):
library(sparklyr)
library(dplyr)
# Set your SPARK_HOME path
Sys.setenv(SPARK_HOME="/home/master/spark/spark-2.3.0-bin-hadoop2.7/")
config <- spark_config()
# Optionally you can modify config parameters here
sc <- spark_connect(master = "spark://172.16.10.20:7077", spark_home = Sys.getenv("SPARK_HOME"), config = config)
# Some test code, copying data to Spark cluster
iris_tbl <- copy_to(sc, iris)
src_tbls(sc)
spark_apply(iris_tbl, function(data) {
return(head(data))
})
All commands are executed, fine and smooth (but a bit slow to my taste), and the spark log is kept in a temp file. When looking into the log file I see no mention of the slave node, which makes me wonder, whether this Spark is really running in a cluster mode.
How may I check that the master-slave relation is really working?
In your case please check
172.16.10.20:8080 url and open executors tab to see the number executors running
Here is URL
http://[driverHostname]:4040 by default
http://<master-ip>:8080(webui-port)
Additional info on a monitor and inspect Spark job executions
command based status check stack question

Resources