proto grror when generating gateway protoc-gen-grpc-gateway: program not found or is not executable - grpc

I have found similar posts but they are rather old and they sadly did not solve my issue.
I am trying to add a Gateway for my GoLang GRPC services
PROJ_PATH=${CURDIR}
.PHONY: proto
proto: ## Generate protobuf code
# Compile proto files inside the project.
protoc api.proto --proto_path=${PROJ_PATH}/proto --go_out=. --go-grpc_out=. \
--grpc-gateway_out . \
--grpc-gateway_opt generate_unbound_methods=true \
--openapiv2_out . \
--openapiv2_opt logtostderr=true \
--openapiv2_opt generate_unbound_methods=true
I add the two dependencies
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2
But I still get his error
protoc-gen-grpc-gateway: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--grpc-gateway_out: protoc-gen-grpc-gateway: Plugin failed with status code 1.
make: *** [proto] Error 1
I had to remove the import and gateway options in the GRPC file because this error so it looks like I have the wrong plugin but reading here tells me otherwise.
google/api/annotations.proto: File not found.
api.proto:5:1: Import "google/api/annotations.proto" was not found or had errors.
make: *** [proto] Error 1
syntax = "proto3";
option go_package = "pkg/api";
import "google/api/annotations.proto"; <---- had to remove
service ApiService {
rpc Test(TestRequest) returns (TestResponse){
option (google.api.http) = { <---- had to remove
get: "/v1/test"
body: "*"
};
}
}
Any advice would be greatly appreciated.
---- solved with ------
go install \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway#latest \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2#latest
then add the path to the proto command
--plugin=protoc-gen-grpc-gateway=${GOPATH}/bin/protoc-gen-grpc-gateway \
full command...
protoc api.proto --proto_path=${PROJ_PATH}/proto --go_out=. --go-grpc_out=. \
--grpc-gateway_out . \
--grpc-gateway_opt generate_unbound_methods=true \
--plugin=protoc-gen-grpc-gateway=${GOPATH}/bin/protoc-gen-grpc-gateway \
--openapiv2_out . \
--openapiv2_opt logtostderr=true \
--openapiv2_opt generate_unbound_methods=true

I had to install the binary like so
go install \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway#latest \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2#latest
Then update the proto command with the path.
--plugin=protoc-gen-grpc-gateway=${GOPATH}/bin/protoc-gen-grpc-gateway \
Full proto command
PROJ_PATH=${CURDIR}
.PHONY: proto
proto: ## Generate protobuf code
# Compile proto files inside the project.
protoc api.proto --proto_path=${PROJ_PATH}/proto --go_out=. --go-grpc_out=. \
--grpc-gateway_out . \
--grpc-gateway_opt generate_unbound_methods=true \
--plugin=protoc-gen-grpc-gateway=${GOPATH}/bin/protoc-gen-grpc-gateway \
--openapiv2_out . \
--openapiv2_opt logtostderr=true \
--openapiv2_opt generate_unbound_methods=true

Related

Can I completely customize the flags for ./configure in a SPEC file?

My rpmbuild log tells me all the flags used when calling configure:
./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu \
--program-prefix= \
--disable-dependency-tracking \
--prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--sysconfdir=/etc \
--datadir=/usr/share \
--includedir=/usr/include \
--libdir=/usr/lib64 \
--libexecdir=/usr/libexec \
--localstatedir=/var \
--sharedstatedir=/var/lib \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--prefix=/opt/custom/SENSOR/Qt-5.15.2 \
--confirm-license \
--opensource
My problem is that the 'build' and 'host' flags (plus several others) are unknown commands for this particular configure script. How can I take complete control of the call to configure in my SPEC file? It's obviously not enough to add new flags to the %configure scriptlet, I need to remove flags that rpmbuild is adding by default.
It looks like the answer is to call configure directly instead of using the scriptlet. I.e., replace %configure with:
./configure --prefix=/opt/custom/SENSOR -confirm-license -opensource

How do you include bluetooth in a yocto linux system

Does any know of any good instructions explaining how to implement bluetooth into a yocto build.
I've added bluetooth into the MACHINE variable within the machine.conf and bluez to CORE_IMAGE_EXTRA_INSTALL in the local.conf located in my build directory. After building a new image, bluetoothd won't start because this condition is not met ConditionPathIsDirectory=/sys/class/bluetooth which is correct because that location doesn't exist on my system.
I'm new to yocto so any guidance would be greatly appreciated.
EDIT
So the /conf/bblayers has the following content:
BBLAYERS ?= " \
${BSPDIR}/poky/meta \
${BSPDIR}/poky/meta-poky \
${BSPDIR}/poky/meta-yocto-bsp \
${BSPDIR}/meta-atmel \
${BSPDIR}/meta-openembedded/meta-oe \
${BSPDIR}/meta-openembedded/meta-networking \
${BSPDIR}/meta-openembedded/meta-python \
${BSPDIR}/meta-openembedded/meta-multimedia \
${BSPDIR}/meta-swupdate \
${BSPDIR}/meta-custom \
${BSPDIR}/meta-openembedded/meta-initramfs \
The local.conf contains:
CORE_IMAGE_EXTRA_INSTALL += "\
bluez5 \
rpm \
valgrind \
"
DISTRO_FEATURES_append = " systemd bluetooth"
DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VURTUAL-RUNTIME_initscripts = "systemd-compat-units"
The .inc file in my custom layer contains:
#IMAGE_FEATURES += "ssh-server-openssh package-management"
IMAGE_INSTALL = "\
packagegroup-core-boot \
packagegroup-core-full-cmdline \
packagegroup-base-bluetooth \
kernel-modules \
libgpiod \
ntp \
sqlite3 \
openssl \
mtd-utils \
${CORE_IMAGE_EXTRA_INSTALL} \
"
inherit core-image
The machine conf file contains:
MACHINE_FEATURES = "apm ext2 ext3 pcmcia usbhost usbgadget"
Running
bitbake -e packagegroup-base | grep ^ADD_BT
Returns
ADD_BT="packagegroup-base-bluetooth"
bitbake -e <image-name> | grep ^IMAGE_INSTALL=
Returns
IMAGE_INSTALL=" packagegroup-core-boot packagegroup-core-full-cmdline packagegroup-base-bluetooth kernel-modules libgpiod ntp sqlite3 openssl mtd-utils apg-st-databases bluez5 rpm valgrind "
bitbake -e packagegroup-base-extended | grep ^RDEPENDS_packagegroup-base-extended=
returns
RDEPENDS_packagegroup-base-extended=" packagegroup-base packagegroup-base-wifi packagegroup-base-bluetooth "
First of all, MACHINE is dedicated to have one value which is the configuration file for your board, and you can not add bluetooth to it.
Yocto has an automatic bluetooth feature that handles adding bluetooth into your build. But it only there for boards that have a builtin bluetooth module.
For more info, check the machine features in the official Yocto doc here.
Let me now explain how bluetooth is integrated:
Basically, you only need some kernel defconfig configuration and bluez5 utility.
If you add the following line to local.conf or your custom machine file:
DISTRO_FEATURES_append = " bluetooth"
it will take effect if you build image that is based on core-image class, like core-image-minimal, because it contains a packagegroup-base-bluetooth as follows:
SUMMARY_packagegroup-base-bluetooth = "Bluetooth support"
RDEPENDS_packagegroup-base-bluetooth = "\
bluez5 \
"
RRECOMMENDS_packagegroup-base-bluetooth = "\
kernel-module-bluetooth \
kernel-module-l2cap \
kernel-module-rfcomm \
kernel-module-hci-vhci \
kernel-module-bnep \
kernel-module-hidp \
kernel-module-hci-uart \
kernel-module-sco \
${#bb.utils.contains('COMBINED_FEATURES', 'usbhost', 'kernel-module-hci-usb', '',d)} \
${#bb.utils.contains('COMBINED_FEATURES', 'pcmcia', 'kernel-module-bluetooth3c-cs', '',d)} \
${#bb.utils.contains('COMBINED_FEATURES', 'pcmcia', 'kernel-module-bluecard-cs', '',d)} \
${#bb.utils.contains('COMBINED_FEATURES', 'pcmcia', 'kernel-module-bluetoothuart-cs', '',d)} \
${#bb.utils.contains('COMBINED_FEATURES', 'pcmcia', 'kernel-module-dtl1-cs', '',d)} \
"
It will add packagegroup-base-bluetooth only in two conditions:
If bluetooth in DISTRO_FEATRUES and not in MACHINE_FEATURES and one of (pcmcia, pci or usbhost) in MACHINE_FEATURES
If bluetooth in COMBINED_FEATURES.
For more info on COMBINED_FEATURES check this link.
So, you just need to make true one of the conditions.
You can dig in more to understand more details, check:
poky/meta/classes/core-image.bbclass
poky/meta/recipes-core/packagegroups/packagegroup-base.bb
NOTE
If your bluetooth module has a dedicated kernel driver for example, you need to activate it in the kernel configuration via a fragment file.
Or create a module recipe for it.
EDIT
In order to add bluetooth just:
DISTRO_FEATURES_append = " bluetooth"
after that, you can check that bluetooth package is added. Here are details:
packagegroup-base-extended is always added to images based on core-image.bbclass and here is its content:
RDEPENDS_packagegroup-base-extended = "\
packagegroup-base \
${ADD_WIFI} \
${ADD_BT} \
${ADD_3G} \
${ADD_NFC} \
"
You see the ADD_BT variable? It is by default empty, and it is filled with packagegroup-base-bluetooth only if the first condition is true, and here is the detail from the recipe:
if "bluetooth" in distro_features and not "bluetooth" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
d.setVar("ADD_BT", "packagegroup-base-bluetooth")
Now, to check that it is added, run:
bitbake -e packagegroup-base | grep ^ADD_BT
you should see: ADD_BT="packagegroup-base-bluetooth"
Also, you can check that packagegroup-base-extended is added to your image:
bitbake -e <your_image> | grep ^IMAGE_INSTALL=
Also, you can check if packagegroup-base-extended has packaged bluetooth package with it:
bitbake -e packagegroup-base-extended | grep ^RDEPENDS_packagegroup-base-extended=
EDIT2:
Your configuration looks okay, specially when packagegroup-base-bluetooth is shipped with in IMAGE_INSTALL.
I have some comments that you need to consider anyways:
local.conf:
Remove bluez5 from CORE_IMAGE_EXTRA_INSTALL
Change:
DISTRO_FEATURES_append = " systemd bluetooth"
DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VURTUAL-RUNTIME_initscripts = "systemd-compat-units"
with:
INIT_MANAGER = "systemd"
DISTRO_FEATURES_append = " bluetooth"
In your custom image file:
Put inherit core-image before IMAGE_INSTALL, because IMAGE_INSTALL of core-image will not take effect because it is ?=. So:
inherit core-image
IMAGE_INSTALL += "\
packagegroup-core-full-cmdline \
kernel-modules \
libgpiod \
ntp \
sqlite3 \
openssl \
mtd-utils \
"
Now, the only part that mention bluetooth is :
DISTRO_FEATURES_append = " bluetooth"
Now, check again for the variable tests.
This is not as simple as adding bluetooth as a DISTRO_FEATURE. Adding this only adds the systemd service and other user-space tools to the build. In fact, what you really need is your Linux kernel to support Bluetooth (i.e. add the necessary drivers and configuration to the Linux Kconfig). It can be done by means of enabling CONFIG_BT through menuconfig or directly on your config file. Additionally, you might want to include kernel drivers/plugins for the specific Bluetooth hardware you are using.
For /sys/class/bluetooth to exist, the kernel module bluetooth.ko, which is only autoloaded by systemd-udev if it actually finds a working Bluetooth hardware device.
I suggest you start by adding support for Bluetooth on your Linux kernel, recompiling and reflashing. Then all your user-space tools should start working if your BT hardware is able to be brought up.
This is a good resource that may help you: https://wiki.archlinux.org/title/bluetooth#Troubleshooting

Protoc does not export the TS file version of *_grpc_pb.js?

I am new to setting up the gRPC web based client side. Our backend is already up and running on Go with gRPC. I am testing out what it's like converting the .proto file into TS. I am successfully able to generate some of the files, however, I am missing the TypeScript "Service" file.
I pretty much followed the instructions from the grpc_tools_node_protoc_ts site.
Setup a script to generate files for 1) the service and 2) the client model:
PROTOC_GEN_TS_PATH="./node_modules/.bin/protoc-gen-ts"
GRPC_TOOLS_NODE_PROTOC_PLUGIN="./node_modules/.bin/grpc_tools_node_protoc_plugin"
GRPC_TOOLS_NODE_PROTOC="./node_modules/.bin/grpc_tools_node_protoc"
OUT_DIR="./_protos_/proto/"
# JavaScript code generating
${GRPC_TOOLS_NODE_PROTOC} \
--plugin=protoc-gen-grpc="${GRPC_TOOLS_NODE_PROTOC_PLUGIN}" \
--js_out=import_style=commonjs,binary:"${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
${GRPC_TOOLS_NODE_PROTOC} \
--plugin=protoc-gen-ts="${PROTOC_GEN_TS_PATH}" \
--ts_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
What I get on the output is missing the *_grpc_pb.d.ts. I am under the impression I need this? 🤷🏻‍♂️
I have also tried adding the service option to the flag:
--ts_out="service=grpc-web:${OUT_DIR}" \
This now generates a *_pb_service.d.ts output file, still without the *_grpc_pb.d.ts file. I was reading the docs more and am thinking this service=grpc-web is actually the option I need since we're not running a node server.
Does this seem right? This is what I have now:
# Note the ts_out flag "service=grpc-node":
# This does generate the *_grpc_pb.d.ts but not the service files
protoc \
--plugin="protoc-gen-ts=${PROTOC_GEN_TS_PATH}" \
--plugin=protoc-gen-grpc=${GRPC_TOOLS_NODE_PROTOC_PLUGIN} \
--js_out="import_style=commonjs,binary:${OUT_DIR}" \
--ts_out="service=grpc-node:${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
# Note the ts_out flag "service=grpc-web":
# This does generate the service files, but not the *_grpc_pb.d.ts file
protoc \
--plugin="protoc-gen-ts=${PROTOC_GEN_TS_PATH}" \
--plugin=protoc-gen-grpc=${GRPC_TOOLS_NODE_PROTOC_PLUGIN} \
--js_out="import_style=commonjs,binary:${OUT_DIR}" \
--ts_out="service=grpc-web:${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto

Getting No FileSystem for Schema WASB . Hdinsight Map Reduce

I am running a simple map reduce job in Azure HDInsight,below is the command that we are running:
java -jar WordCount201.jar wasb://hexhadoopcluster-2019-05-15t07-01-07-193z#hexanikahdinsight.blob.core.windows.net/hexa/CustData.csv wasb://hexhadoopcluster-2019-05-15t07-01-07-193z#hexanikahdinsight.blob.core.windows.net/hexa
Getting the below error :
java.io.IOException: No FileSystem for scheme: wasb
For Java use jdk1.8 and below POM org.apache.hadoop hadoop-mapreduce-examples2.7.3scope>provided org.apache.hadoophadoop-mapreduce-client-common2.7.3providedjdk.toolsjdk.toolsorg.apache.hadoophadoop-common2.7.3provided
WASB is a wrapper around HDFS file system. I am not sure you can use it in normal java program. Do you have any reference / link which you referred to?
You can try to get the https equivalent of the custData.csv file.Below is an example of Spark job I am able to submit on HDInsight cluster using WASB
spark-submit \
--class com.nileshgule.movielens.MovieRatingAnalysis \
--master yarn \
--deploy-mode cluster \
--executor-memory 1g \
--name MoviesCsvReader \
--conf "spark.app.id=MovieRatingAnalysis" \
wasb://hd-spark-cluster-2019#hdsparkclusterstorage.blob.core.windows.net/learning-spark-1.0.jar \
wasb://hd-spark-cluster-2019#hdsparkclusterstorage.blob.core.windows.net/ml-latest/ratings.csv \
wasb://hd-spark-cluster-2019#hdsparkclusterstorage.blob.core.windows.net/ml-latest/movies.csv
And here is an example of passing the same files using their equivalent https URI
spark-submit \
--class com.nileshgule.movielens.MovieRatingAnalysis \
--master yarn \
--deploy-mode cluster \
--executor-memory 1g \
--name MoviesCsvReader \
--conf "spark.app.id=MovieRatingAnalysis" \
https://hdsparkclusterstorage.blob.core.windows.net/hd-spark-cluster-2019/learning-spark-1.0.jar \
https://hdsparkclusterstorage.blob.core.windows.net/hd-spark-cluster-2019/ml-latest/ratings.csv \
https://hdsparkclusterstorage.blob.core.windows.net/hd-spark-cluster-2019/ml-latest/movies.csv
For hadoop job kindly run the jar from root user . Once you login to HDinsight run the command sudo su - . And the create a folder and place the jar to that folder and run the jar .

How to solve the CSScomb error in PhpStorm?

I wanted to install on PhpStorm 8.0.1 CSScomb.js
I do everything as it is written on a page on github. Established CSScomb globally and locally (so sure). Prescribed ways and...
...run and out this error:
Error running CSScomb: Can not run program "C: \ Users \ Kanat \
AppData \ Roaming \ npm \ node_modules \ csscomb \ bin \ csscomb" (in
directory "D: \ OpenServer \ domains \ LPDevplate \ src \ scss \
modules"): CreateProcess error = 193% 1 is not a valid Win32
application
Someone faced with this error and help solve it?
the same error "Error running 'CSScomb': Cannot run program"

Resources