How can I interact with containerd gRPC API using grpcurl? - grpc

Introduction
There is a CLI tool to interact with containerd called ctr. However I would like to achieve the same results without it using curl, after searching on google it can't be done with curl but grpcurl instead.
How can I do basic gRPC calls to containerd?
What I have tried
First I downloaded both grpcurl and api.proto.
Then I listed available services:
vagrant#ubuntu-focal:~/containerd/api/grpc/types$ grpcurl -import-path . -proto api.proto list
types.API
vagrant#ubuntu-focal:~/containerd/api/grpc/types$ grpcurl -import-path . -proto api.proto list types.API
types.API.AddProcess
types.API.CreateCheckpoint
types.API.CreateContainer
types.API.DeleteCheckpoint
types.API.Events
types.API.GetServerVersion
types.API.ListCheckpoint
types.API.Signal
types.API.State
types.API.Stats
types.API.UpdateContainer
types.API.UpdateProcess
Used protoc to create a compiled version instead but didnt seem to change anything.
Finally tried making a request to the method Stats (including containerd's UNIX socket path and -plaintext). However it seems I am messing something up, here is the error (both using --proto and --protoset:
sudo grpcurl -plaintext -protoset myservice.protoset -unix /run/containerd/containerd.sock types.API/Stats
ERROR:
Code: Unimplemented
Message: unknown service types.API
sudo grpcurl -plaintext -proto api.proto -unix /run/containerd/containerd.sock types.API.Stats
ERROR:
Code: Unimplemented
Message: unknown service types.API
Also tried with:
API/Stats
API.Stats
types.API.Stats

I cloned the wrong repo instead of containerd.
To check it working use the following request:
vagrant#ubuntu-focal:~/containerd/api/services/version/v1$ sudo grpcurl -plaintext -proto version.proto -unix /run/containerd/containerd.sock containerd.services.version.v1.Version/Version
{
"version": "1.5.5-0ubuntu3~20.04.2"
}

Related

Setting up the EFI Shell in Qemu to allow for Http requests

I am developing an UEFI App that will need to perform a GET request through http.
As a start up point, I want to make sure my setup is working properly so that the http requests can actually go through.
To that end, I spent the last few days trying to make the http command work in the EFI Shell launched inside QEMU.
I can get the ping command to work properly, but calling:
http httpbin.org/get
Always returns: 
Unable to open http protocol on `eth0` - Unsupported
Unable to download the file `/get` on `eth0` - Unsupported
This is my startup.nsh script to configure the EFI Shell's interface:
connect
ifconfig -r eth0
ifconfig -s eth0 dhcp
ifconfig -l eth0
These were my different attempts at invoking Qemu properly:
-netdev user,id=mynet0,hostfwd=tcp::8080-:80 -device e1000,netdev=mynet0 \
        -netdev user,id=user.0 -device e1000,netdev=user.0 \
        -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32 \
       
And following this guide I tried to setup a tap, albeit without luck, I'd launch qemu with the following configuration:
-netdev tap,id=mynet0,ifname=tap0,script=no,downscript=no -device e1000,netdev=mynet0,mac=52:55:00:d1:55:01 \
Do you have any clue what step am I missing?
Where do you believe I could be failing in making eth0 supported?
Is the tap crucial?
Are you able to make this setup work on your side?
Update:
Very good suggestion #MiSimon, I hadn't realized that the HttpDxe driver wasn't being built with the OvmfPkg.
I have now added its INF to OvmfPkgX64.dsc and OvmfPkgX64.fdf.
Although, running drivers is displaying a duplicate entry:
0000000A D - - 1 - HttpDxe HttpDxe
0000000A ? - - - - HttpDxe HttpDxe
With respect to calling the http command, the error has progressed to:
Downloading 'http://httpbin.org/get'
Unable to download the file '/get' on 'eth0' - Unsupported
The debug log shows:
HttpNotify: Event - 0, EventStatus - Unsupported
Error: Could not retrieve the host address from DNS server.
The tool requires nearly all network drivers to be loaded.
Make sure your image contains the following drivers:
SnpDxe
MnpDxe
ArpDxe
Ip4Dxe/Ip6Dxe
Dhcp4Dxe/Dhcp6Dxe
Udp4Dxe/Udp6Dxe
DnsDxe
TcpDxe
HttpDxe
HttpUtilitiesDxe
All of them are can be found in EDK2 inside the NetworkPkg.

How to get the status code within the response in grpcurl?

I use grpcurl for grpc request. I have the following request:
grpcurl -plaintext -import-path C:/Users/username -proto file.proto <localhost:port > service_name/methode
The mentioned request will return the following response without the status code. How can I get the status code within the response?
{
"id": "0000000000000",
"name": "AAAAAAAAAA"
}
You cannot (directly).
Error codes are part of the gRPC protocol metadata not the (user-defined) message(s).
With gRPCurl, you've a couple of ways to get the error code.
In Bash, you can use command's exit status:
grpcurl ...
echo $?
See:
Need documentation for gRPC error codes
See gRPC canonical error codes
Or you can use gRPCurl's -format-error flag.
-format-error
When a non-zero status is returned, format the response using the
value set by the -format flag .

Failed to debug grpc channelz with web tool

In this doc A short introduction to Channelz, there is a web tool called grpc-zpages conveniently serve channelz data through a web page.
However, we cannot find any TopChannels data from this web tool. Does someone meet the same issue?
To make this web tool run, there are some changes to the codes
Add the following codes to http_filters of zprox.sh
function envoy_on_response(response_handle)
filter_name = "ENVOY"
response_handle:headers():add("my_Filter", filter_name)
end
to solve the error envoy_on_response() function not found. lua filter will not hook responses
Comment network_mode: 'host' of start_docker.sh to solve the error docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings
Replace manual.GenerateAndRegisterManualResolver() with
r := manual.NewBuilderWithScheme("whatever")
r.InitialState(resolver.State{Addresses: []resolver.Address{{Addr: ":10001"}, {Addr: ":10002"}, {Addr: ":10003"}}})
// Set up a connection to the server.
conn, err := grpc.Dial(r.Scheme()+":///test.server",
grpc.WithInsecure(),
grpc.WithResolvers(r),
grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
)
under grpc 1.37.1
Please make sure you had added channelz service and reflection service to the server. If your service is a client, you need to make the client has a grpc server also. After call server and the channel is connected, the related data could be fetched.
The grpc-zpages is not maintain anymore, and it is hard to use. I suggest to use channelzcli replace.
I also had trouble using the web tool and ended up using grpcurl.
One thing that took me a while to realize is that in order to activate channelz on your grpc client you need to run a grpc server and enable channelz on your client service alongside your grpc client
Some grpcurl useful commands for Channelz:
Query for the Channelz service API, to see all available methods:
grpcurl -plaintext -msg-template localhost:50052 describe grpc.channelz.v1.Channelz
Query method template to understand what parameters it requires:
grpcurl -plaintext -msg-template localhost:50052 describe .grpc.channelz.v1.GetSocketReques
Client commands examples (make sure to use the client port):
grpcurl -plaintext localhost:50052 grpc.channelz.v1.Channelz/GetTopChannels
grpcurl -plaintext -d '{"subchannel_id":"14"}' localhost:50052 grpc.channelz.v1.Channelz/GetSubchannel
grpcurl -plaintext -d '{"socket_id":"34"}' localhost:50052 grpc.channelz.v1.Channelz/GetSocket
Server commands (make sure to use the server port):
grpcurl -plaintext localhost:50051 grpc.channelz.v1.Channelz/GetServers
grpcurl -plaintext -d '{"server_id":"2"}' localhost:50051 grpc.channelz.v1.Channelz/GetServerSockets
grpcurl -plaintext -d '{"socket_id":"21"}' localhost:50051 grpc.channelz.v1.Channelz/GetSocket

Source file not available anymore

I'm facing something new in my experience. An updated Yocto environment fails to build while fetching MariaDB:
WARNING: mariadb-5.5.64-r0 do_fetch: Failed to fetch URL https://downloads.mariadb.org/f/mariadb-5.5.64/source/mariadb-5.5.64.tar.gz, attempting MIRRORS if available
ERROR: mariadb-5.5.64-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-BIEhFRRkoA"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/sysroots-uninative/x86_64-linux/usr/bin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/openembedded-core/scripts:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot-native/usr/bin/arm-openstlinux_eglfs-linux-gnueabi:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot/usr/bin/crossscripts:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot-native/usr/sbin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot-native/usr/bin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot-native/sbin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/recipe-sysroot-native/bin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/openembedded-core/bitbake/bin:/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/hosttools"; export HOME="/home/osboxes"; /usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate -P /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/downloads 'https://downloads.mariadb.org/f/mariadb-5.5.64/source/mariadb-5.5.64.tar.gz' --progress=dot -v failed with exit code 8, output:
--2019-11-03 12:10:45-- https://downloads.mariadb.org/f/mariadb-5.5.64/source/mariadb-5.5.64.tar.gz
Resolving downloads.mariadb.org (downloads.mariadb.org)... 116.203.207.31, 2a01:4f8:c2c:b04e::1
Connecting to downloads.mariadb.org (downloads.mariadb.org)|116.203.207.31|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: http://ftp.hosteurope.de/mirror/mariadb.org//mariadb-5.5.64/source/mariadb-5.5.64.tar.gz [following]
--2019-11-03 12:10:46-- http://ftp.hosteurope.de/mirror/mariadb.org//mariadb-5.5.64/source/mariadb-5.5.64.tar.gz
Resolving ftp.hosteurope.de (ftp.hosteurope.de)... 80.237.136.138, 2a01:488:10:1::50ed:888a
Connecting to ftp.hosteurope.de (ftp.hosteurope.de)|80.237.136.138|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-11-03 12:10:46 ERROR 404: Not Found.
ERROR: mariadb-5.5.64-r0 do_fetch: Fetcher failure for URL: 'https://downloads.mariadb.org/f/mariadb-5.5.64/source/mariadb-5.5.64.tar.gz'. Unable to fetch URL from any source.
ERROR: mariadb-5.5.64-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/build-openstlinuxeglfs-stm32mp1/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-openstlinux_eglfs-linux-gnueabi/mariadb/5.5.64-r0/temp/log.do_fetch.11382
ERROR: Task (/local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/meta-openembedded/meta-oe/recipes-dbs/mysql/mariadb_5.5.64.bb:do_fetch) failed with exit code '1'
Now I see the URLs are hardcoded into: layers/meta-openembedded/meta-oe/recipes-dbs/mysql/mariadb.inc:
SUMMARY = "A robust, scalable, and reliable SQL server"
HOMEPAGE = "http://mariadb.org"
SECTION = "libs"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
SRC_URI = "https://downloads.mariadb.org/f/${BP}/source/${BP}.tar.gz \
file://fix-cmake-module-path.patch \
file://remove-bad-path.patch \
file://fix-mysqlclient-r-version.patch \
file://my.cnf \
file://mysqld.service \
file://install_db.service \
file://install_db \
file://mysql-systemd-start \
file://configure.cmake-fix-valgrind.patch \
file://fix-a-building-failure.patch \
file://change-cc-to-cc-version.patch \
file://0001-disable-ucontext-on-musl.patch \
"
Hence, I tried to retrieve the same file (mariadb-5.5.64.tar.gz) on the Internet and I found this:
https://downloads.mariadb.org/f/mariadb-5.5.64/source/mariadb-5.5.64.tar.gz/from/http%3A//ftp.hosteurope.de/mirror/archive.mariadb.org/?serve
that seems very close to the one above, but the working one has another path as suffix.
I don't understand if they (the mariadb maintainers) changed the URLs or there is an error in the meta-openembedded recipe.
The maintainers of FOSS software, in this case the MariaDB Foundation, usually don't make promises that download URLS will remain unchanged. MariaDB uses a scheme to help downloaders find a nearby mirror site, and that's what you found.
Those mirrorfinders rely on stuff like http redirect operations. Browsers are good at handling those operations. Downloading code built into other programs may not be.
You may find it helpful to edit that URL in that recipe to point to the exact ftp file you need. It appears to be in this ftp folder.
http://ftp.hosteurope.de/mirror/archive.mariadb.org/mariadb-5.5.64/source/

jfrog artifactory after 6.2.0 upgrade using get [RELEASE] no longer works?

After upgrading to Artifactory Pro 6.2.0 using [RELEASE] for the version in the requested path to get the latest Maven release artifact no longer seems to work.
$ wget --no-check-certificate -N --user=reader --password=****** -P . https://artifactory.***.com/artifactory/libs-release-local/envision/tools/envision-buildtools/\[RELEASE\]/envision-buildtools-\[RELEASE\].tgz
Warning: wildcards not supported in HTTP.
--2018-08-14 10:45:59-- https://artifactory.****.com/artifactory/libs-release-local/envision/tools/envision-buildtools/[RELEASE]/envision-buildtools-[RELEASE].tgz
Resolving artifactory.***.com... 10.***.**.**
Connecting to artifactory.***.com|10.***.**.**|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2018-08-14 10:45:59 ERROR 400: Bad Request.
Is there any work-around or fix for this?
In the end this was a Tomcat issue - not an Artifactory bug. We had to update the server.xml to add the relaxedPathChars and relaxedQueryChars attributes as shown below.
<Connector port="8081" sendReasonPhrase="true" relaxedPathChars='[]' relaxedQueryChars='[]'/>

Resources