Failed to debug grpc channelz with web tool - grpc

In this doc A short introduction to Channelz, there is a web tool called grpc-zpages conveniently serve channelz data through a web page.
However, we cannot find any TopChannels data from this web tool. Does someone meet the same issue?
To make this web tool run, there are some changes to the codes
Add the following codes to http_filters of zprox.sh
function envoy_on_response(response_handle)
filter_name = "ENVOY"
response_handle:headers():add("my_Filter", filter_name)
end
to solve the error envoy_on_response() function not found. lua filter will not hook responses
Comment network_mode: 'host' of start_docker.sh to solve the error docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings
Replace manual.GenerateAndRegisterManualResolver() with
r := manual.NewBuilderWithScheme("whatever")
r.InitialState(resolver.State{Addresses: []resolver.Address{{Addr: ":10001"}, {Addr: ":10002"}, {Addr: ":10003"}}})
// Set up a connection to the server.
conn, err := grpc.Dial(r.Scheme()+":///test.server",
grpc.WithInsecure(),
grpc.WithResolvers(r),
grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
)
under grpc 1.37.1

Please make sure you had added channelz service and reflection service to the server. If your service is a client, you need to make the client has a grpc server also. After call server and the channel is connected, the related data could be fetched.
The grpc-zpages is not maintain anymore, and it is hard to use. I suggest to use channelzcli replace.

I also had trouble using the web tool and ended up using grpcurl.
One thing that took me a while to realize is that in order to activate channelz on your grpc client you need to run a grpc server and enable channelz on your client service alongside your grpc client
Some grpcurl useful commands for Channelz:
Query for the Channelz service API, to see all available methods:
grpcurl -plaintext -msg-template localhost:50052 describe grpc.channelz.v1.Channelz
Query method template to understand what parameters it requires:
grpcurl -plaintext -msg-template localhost:50052 describe .grpc.channelz.v1.GetSocketReques
Client commands examples (make sure to use the client port):
grpcurl -plaintext localhost:50052 grpc.channelz.v1.Channelz/GetTopChannels
grpcurl -plaintext -d '{"subchannel_id":"14"}' localhost:50052 grpc.channelz.v1.Channelz/GetSubchannel
grpcurl -plaintext -d '{"socket_id":"34"}' localhost:50052 grpc.channelz.v1.Channelz/GetSocket
Server commands (make sure to use the server port):
grpcurl -plaintext localhost:50051 grpc.channelz.v1.Channelz/GetServers
grpcurl -plaintext -d '{"server_id":"2"}' localhost:50051 grpc.channelz.v1.Channelz/GetServerSockets
grpcurl -plaintext -d '{"socket_id":"21"}' localhost:50051 grpc.channelz.v1.Channelz/GetSocket

Related

How can I interact with containerd gRPC API using grpcurl?

Introduction
There is a CLI tool to interact with containerd called ctr. However I would like to achieve the same results without it using curl, after searching on google it can't be done with curl but grpcurl instead.
How can I do basic gRPC calls to containerd?
What I have tried
First I downloaded both grpcurl and api.proto.
Then I listed available services:
vagrant#ubuntu-focal:~/containerd/api/grpc/types$ grpcurl -import-path . -proto api.proto list
types.API
vagrant#ubuntu-focal:~/containerd/api/grpc/types$ grpcurl -import-path . -proto api.proto list types.API
types.API.AddProcess
types.API.CreateCheckpoint
types.API.CreateContainer
types.API.DeleteCheckpoint
types.API.Events
types.API.GetServerVersion
types.API.ListCheckpoint
types.API.Signal
types.API.State
types.API.Stats
types.API.UpdateContainer
types.API.UpdateProcess
Used protoc to create a compiled version instead but didnt seem to change anything.
Finally tried making a request to the method Stats (including containerd's UNIX socket path and -plaintext). However it seems I am messing something up, here is the error (both using --proto and --protoset:
sudo grpcurl -plaintext -protoset myservice.protoset -unix /run/containerd/containerd.sock types.API/Stats
ERROR:
Code: Unimplemented
Message: unknown service types.API
sudo grpcurl -plaintext -proto api.proto -unix /run/containerd/containerd.sock types.API.Stats
ERROR:
Code: Unimplemented
Message: unknown service types.API
Also tried with:
API/Stats
API.Stats
types.API.Stats
I cloned the wrong repo instead of containerd.
To check it working use the following request:
vagrant#ubuntu-focal:~/containerd/api/services/version/v1$ sudo grpcurl -plaintext -proto version.proto -unix /run/containerd/containerd.sock containerd.services.version.v1.Version/Version
{
"version": "1.5.5-0ubuntu3~20.04.2"
}

curl CONNECT method without path (destination hostname and port instead)

I want to test proxy server. In order to make https request, browser sends CONNECT method beforehand (e.g. like Firefox does, when proxy is specified).
I can not achieve/send the same result in curl:
Following has root slash /www.example.com:443:
curl -X CONNECT http://proxy_host:proxy_port/www.example.com:443
Following will not work (without slash):
curl -X CONNECT http://proxy_host:proxy_portwww.example.com:443
Following is not what I want:
curl -X CONNECT http://proxy_host:proxy_port/some_path
So the first line of HTTP data should be CONNECT www.example.com:443 HTTP/1.1 but not CONNECT /www.example.com:443 HTTP/1.1 like curl sends in this case.
Maybe this question also related some-how, if I would know how to not send path.
NOTE! I do not want to use curl -x http://proxy_host:proxy_port https://www.example.com, because this option/flag -x does not work with custom SSL certificates --cacert ... --key ... --cert ....
Any ideas how to send plain header data or not specify path, or specify host and port as a path?
(-X simply replaces the string in the request so of course setting it to CONNECT will not issue a proper CONNECT request and will certainly not make curl handle it correctly.)
curl will do a CONNECT by itself when connecting to a TLS server through a HTTP proxy, and even though you claim -x breaks the certificate options that is an incorrect statement. The --cacert and other options work the same even when the connection is done through a HTTP proxy.
You can also make curl do a CONNECT trough a HTTP(S) proxy for other protocols by using -p, --proxytunnel - also in combination with -x.

How to use invoke http to perform GET request in nifi?

I need to perform a get request from nifi to couchbase. The curl command is:
curl http://HOST:PORT/query/service -d "statement=select item.Date from bucket unnest bucket as item" -u USER:PASSWORD
I tried using InvokeHttp and ExecuteStreamCommand but it keeps returning errors(status code 400). The full error message is:
{ "requestID": "bff62c0b-36fd-401d-bca0-0959e0944323", "errors":
[{"code":1050,"msg":"No statement or prepared value"}], "status":
"fatal", "metrics": {"elapsedTime": "113.31µs","executionTime":
"74.321µs","resultCount": 0,"resultSize": 0,"errorCount": 1
It's important to say that I prefer that the http request will be triggered by an incoming flowfile. I tried using the processors in various of ways but non of them worked.
When I run the command from the nifi server it works fine.
Thanks for the help
the -d parameter of the curl utility forces HTTP POST command
and application/x-www-form-urlencoded mime-type.
so, in the nifi InvokeHTTP select the following parameters
HTTP Method = POST
Remote URL = <your url here>
Basic Authentication Username = <username>
Basic Authentication Password = <password>
Content-Type = application/x-www-form-urlencoded
and the body of the flow file should be
statement=select item.Date from bucket unnest bucket as item
I don't know nifi, but based on the error message, the "statement=" part of the request isn't being included, or you are not sending the request as a POST command.

What version of Apache Airflow includes the experimental API?

The airflow documentation states:
Airflow exposes an experimental Rest API. It is available through the webserver. Endpoints are available at /api/experimental/. Please note that we expect the endpoint definitions to change.
https://airflow.apache.org/api.html#experimental-rest-api
However it doesn't state in which version the API appears. We are running Airflow v1.8.0
But whenever I browse to /api/ or /api/experimental/ I get a 404 and the spinning circles.
I tried curling the same URLs but that only confirmed the same, /api/ gives me a 404:
$ curl -I -L -s http://${AIRFLOWIP}:8080/admin/ | grep HTTP
HTTP/1.1 200 OK
$ curl -I -L -s http://${AIRFLOWIP}:8080/api/ | grep HTTP
HTTP/1.1 404 NOT FOUND
We have this setting in airflow.cfg:
[api]
# How to authenticate users of the API
auth_backend = airflow.api.auth.backend.default
I don't know whether the API is only available in a later version of Airflow or if we have stood it up incorrectly.
Can someone let me know in which version of airflow we can find the experimental API?
The first experimental endpoints were added in 1.8.0, with a few more endpoints added in following releases. There is no endpoint for the root paths of /api/ and /api/experimental/ on any version, so those curls are not expected to work. However, there is a /api/experimental/test/ endpoint which you can hit to confirm the API is available.
If you're going to be using the experimental API, I think the code is the best reference at the moment.
The Airflow API is no more at the experimental phase.
Stable version here Airflow REST API.

Tensorflow Serving: Need to make HTTP request to TF Server which accepts only gRPC requests

My client is on a server where I can make only HTTP request to any server. The Tensorflow is hosted in an AWS machine which accepts the only gRPC requests. Looking for some leads to make this communication happen?
EDIT: 12 th June 2018
TF officially releases REST API for serving
https://www.tensorflow.org/serving/api_rest
They use this particular example: half_plus_three
Server:
$ tensorflow_model_server --rest_api_port=8501 \
--model_name=half_plus_three \
--model_base_path=$(pwd)/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_three/
Client:
$ curl -d '{"instances": [1.0,2.0,5.0]}' -X POST http://localhost:8501/v1/models/half_plus_three:predict
{
"predictions": [3.5, 4.0, 5.5]
}
Which language/platform is your server running?
OTOH, TF is adding REST support now.

Resources