How to make proxy http to tcp to send only raw "body" using envoy? - http

I want to send http request to envoy and after that envoy send only body to tcp server
My steps:
Create tcp listen port nc -l 13370
deploy envoy in docker on same host (:10000 port)
Execute command curl -d "Hello" -X POST localhost:10000
I use this envoy settings
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
http_filters:
- name: lua_filter_cutting_head
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
default_source_code:
inline_string:
function envoy_on_request(request_handle)
request_handle:headers():remove("host")
request_handle:headers():remove("user-agent")
request_handle:headers():remove("x-envoy-expected-rq-timeout-ms")
end
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/"
route:
cluster: cluster_0
clusters:
- name: cluster_0
type: STATIC
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 192.168.0.23
port_value: 13370
Client --(http)-- envoy --(tcp)-- tcp_server (receive only "Hello)
So i want to see on my tcp_server only "Hello", but i see
POST / HTTP/1.1
accept: */*
content-length: 5
content-type: application/x-www-form-urlencoded
x-forwarded-proto: http
x-request-id: 8cc7670c-faf5-4ca8-aa97-b3674d81c28a
x-envoy-expected-rq-timeout-ms: 15000
Hello

Working config
admin:
address:
socket_address:
protocol: TCP
address: 127.0.0.1
port_value: 9902
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains:
- "*"
routes:
- match:
prefix: "/"
headers:
- name: ":method"
string_match:
exact: "POST"
route:
cluster: service
upgrade_configs:
- upgrade_type: CONNECT
connect_config:
allow_post: true
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
http2_protocol_options:
allow_connect: true
clusters:
- name: service
connect_timeout: 0.25s
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 192.168.0.31
port_value: 13370
```

Related

error initializing configuration envoy.yaml Failed to load pkcs12 from /etc/cert.p12

This is what i want to reach with envoy as proxy for https traffic:
I got the certificate (p12) from the java developer to load it with envoy, then make developer happy by hitting envoy without certificate (on port 8080). Check diagram above also.
In order to do that this is my envoy.yaml
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: app
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: api-example-https
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: api-example-https
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: api-example-https
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: api.example.com
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificates:
- pkcs12:
filename: /etc/cert.p12
Unfortunately, i am getting this error:
[2022-11-01 22:17:48.788][1][critical][main] [source/server/server.cc:117] error initializing configuration '/etc/envoy.yaml': Failed to load pkcs12 from /etc/cert.p12
[2022-11-01 22:17:48.788][1][info][main] [source/server/server.cc:961] exiting
Failed to load pkcs12 from /etc/cert.p12
What am i missing ? 3 hours troubleshooting but no way :(
It turns out that the certificate has to be used alongside a password.
And i verified that by using curl on the external service directly:
curl --cert-type P12 --cert 'cert.p12:MyPass' https://api.example.com
This means that envoy.yaml must include the configuration of a password.
Hence, the config file should looks like:
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: app
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: api-example-https
http_filters:
- name: envoy.filters.http.router
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: api-example-https
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: api-example-https
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: api.example.com
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificates:
- pkcs12:
filename: /etc/cert.p1
password:
inline_string: MyPass ### 🔴🔴 Fixed By adding this 🔴🔴
REF: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/common.proto#envoy-v3-api-field-extensions-transport-sockets-tls-v3-tlscertificate-pkcs12
Use password to specify the password to unprotect the PKCS12 data, if necessary.

Envoy Proxy with GRPC Server Streaming throwing UNAVAILABLE: upstream request timeout

We are having GRPC client and GRPC server with service side streaming support.
rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse);
GRPC server is running behind the Envoy proxy with GRPC configuration.
The problem is when we connect the GRPC client to Envoy Proxy -> Grpc Server we are getting the below exception. The code perfectly fine when we connect the GRPC client directly to GRPC server without Envoy Proxy.
io.grpc.StatusRuntimeException: UNAVAILABLE: upstream request timeout
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:478)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The sample envoy.yaml configuration is as follows for reference. Any help on this front will be very helpful.
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: X.X.X.X
port_value: 443
ipv4Compat: true
filter_chains:
- filter_chain_match: {}
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_params:
cipher_suites:
- ECDHE-ECDSA-AES128-GCM-SHA256
- ECDHE-RSA-AES128-GCM-SHA256
- ECDHE-ECDSA-AES128-SHA
- ECDHE-RSA-AES128-SHA
- AES128-GCM-SHA256
- AES128-SHA
- ECDHE-ECDSA-AES256-GCM-SHA384
- ECDHE-RSA-AES256-GCM-SHA384
- ECDHE-ECDSA-AES256-SHA
- ECDHE-RSA-AES256-SHA
- AES256-GCM-SHA384
- AES256-SHA
ecdh_curves:
- P-256
tls_certificates:
- certificate_chain:
filename: "/home/.tomcat_cert.pem"
private_key:
filename: "/home/.tomcat_key.pem"
validation_context:
trust_chain_verification: ACCEPT_UNTRUSTED
alpn_protocols:
- h2
require_client_certificate: false
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
http_filters:
- name: envoy.filters.http.router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/api.ApiService"
route:
cluster: grpc-server
idle_timeout: 0s
max_stream_duration:
grpc_timeout_header_max: 35s
- match:
prefix: "/site"
route:
cluster: site_router
clusters:
- name: site_router
type: static
# Comment out the following line to test on v6 networks
lb_policy: round_robin
connect_timeout: 25s
http2_protocol_options: {}
load_assignment:
cluster_name: site_router
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 7880
- name: grpc-server
type: static
# Comment out the following line to test on v6 networks
lb_policy: round_robin
connect_timeout: 25s
http2_protocol_options: {}
load_assignment:
cluster_name: grpc-server
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 7879
Please note that This works fine for unary gRPC calls as shown below.
rpc SayHello(HelloRequest) returns (HelloResponse);
But Not For Server Streaming
rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse);
Providing timeout:0s inside envoy.yaml solve my problem.
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: X.X.X.X.
port_value: 443
ipv4Compat: true
filter_chains:
- filter_chain_match: {}
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_params:
cipher_suites:
- ECDHE-ECDSA-AES128-GCM-SHA256
- ECDHE-RSA-AES128-GCM-SHA256
- ECDHE-ECDSA-AES128-SHA
- ECDHE-RSA-AES128-SHA
- AES128-GCM-SHA256
- AES128-SHA
- ECDHE-ECDSA-AES256-GCM-SHA384
- ECDHE-RSA-AES256-GCM-SHA384
- ECDHE-ECDSA-AES256-SHA
- ECDHE-RSA-AES256-SHA
- AES256-GCM-SHA384
- AES256-SHA
ecdh_curves:
- P-256
tls_certificates:
- certificate_chain:
filename: "/home/.tomcat_cert.pem"
private_key:
filename: "/home/.tomcat_key.pem"
validation_context:
trust_chain_verification: ACCEPT_UNTRUSTED
alpn_protocols:
- h2
require_client_certificate: false
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
stream_idle_timeout: 0s
http_filters:
- name: envoy.filters.http.router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/api.ApiService"
route:
cluster: grpc-server
timeout: 0s
- match:
prefix: "/policy"
route:
cluster: site_router
timeout: 30s
clusters:
- name: site_router
type: static
# Comment out the following line to test on v6 networks
lb_policy: round_robin
connect_timeout: 25s
http2_protocol_options: {}
load_assignment:
cluster_name: site_router
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 7880
- name: grpc-server
type: static
# Comment out the following line to test on v6 networks
lb_policy: round_robin
connect_timeout: 25s
http2_protocol_options: {}
load_assignment:
cluster_name: grpc-server
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 7879

Envoy proxy does not translate my gRPC request to HTTP1.1 request - using the reverse filter bridge

I'm using the grpc_http1_reverse_bridge_filter from Envoy to convert my gRPC requests to HTTP1.1. The HTTP1.1 requests will be served by Hoverfly which will have simulations uploaded beforehand.
gRPC ---> Envoy (grpc ~~ HTTPS) ---> Hoverfly [HTTP(S)]
But, it doesn't seem like Envoy is correctly translating my request to HTTP1.1. From the server-proxy logs I see:
invalid http2: Remote peer returned unexpected data while we expected SETTINGS frame. Perhaps, peer does not support HTTP/2 properly
Is there something I'm missing in my Envoy config?
admin:
address:
socket_address:
address: 0.0.0.0
port_value: 9901
layered_runtime:
layers:
- name: disable_apple_dns
static_layer:
envoy.restart_features.use_apple_api_for_dns_lookups: false
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 8811
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
route_config:
name: local_route
virtual_hosts:
- name: local_service #backend
domains:
- "*"
routes:
- match:
prefix: "/"
#grpc: {}
route:
host_rewrite_literal: hoverfly
cluster: backend_http_service_hoverfly
timeout: 60.00s
http_filters: # use the reverse bridge filter
- name: envoy.filters.http.grpc_http1_reverse_bridge
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.grpc_http1_reverse_bridge.v3.FilterConfig
content_type: application/json #application/grpc+proto
withhold_grpc_frames: true
- name: envoy.filters.http.router
clusters:
- name: backend_http_service_hoverfly
type: STRICT_DNS
lb_policy: ROUND_ROBIN
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"#type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http2_protocol_options: {}
load_assignment:
cluster_name: backend_http_service_hoverfly
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: # Re-route to hoverfly (this is docker alias)
address: hoverfly #127.0.0.1 #kv-backend-service
port_value: 8500 #8081

Envoy Proxy: Is it possible to create an basic auth with .htpasswd like in nginx?

I have an nginx.conf with basic auth like this:
location / {
auth_basic "Prometheus";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://localhost:9090/;
}
Is it possible to create an basic auth config for envoy like in nginx?
This is my envoy.yaml:
admin:
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/"
route:
cluster: service_envoyproxy_io
http_filters:
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_params:
tls_minimum_protocol_version: TLSv1_2
tls_maximum_protocol_version: TLSv1_3
cipher_suites: "[ECDHE-RSA-AES256-GCM-SHA384|ECDHE-RSA-AES128-GCM-SHA256]"
tls_certificates:
- certificate_chain: { filename: "/etc/ssl/prometheus.crt" }
private_key: { filename: "/etc/ssl/prometheus.key" }
clusters:
- name: service_envoyproxy_io
connect_timeout: 30s
type: STATIC
# Comment out the following line to test on v6 networks
# dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: service_envoyproxy_io
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 0.0.0.0
port_value: 9090
I found basic_auth module in envoy-Doku: Envoy Basic Auth Module, but the strings are hardcoded....not the best way in my opinion!
Many thanks for help!

Update HOST header in Envoy proxy

Hello people I have an architecture where I want to route my requests from Envoy Proxy to nginx proxy. as we know that Nginx works on HOST header matching to route its request. I'm having trouble setting the host header while forwarding the request from Envoy to Nginx. I have the following config
LDS.config
resources:
- "#type": type.googleapis.com/envoy.config.listener.v3.Listener
name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8443
filter_chains:
- filters:
name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: HTTP1
stat_prefix: ingress_http
http_filters:
- name: envoy.router
route_config:
name: local_route
virtual_hosts:
- name: app
domains:
- "some.domain.com"
routes:
- match:
prefix: "/"
route:
cluster: test-cluster
host_rewrite_literal: "mydns.com"
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
certificate_chain: { filename: "/etc/ssl/merged_certificate.crt" }
private_key: { filename: "/etc/ssl/key.key" }
CDS.config
resources:
- "#type": type.googleapis.com/envoy.config.cluster.v3.Cluster
name: test-cluster
connect_timeout: 30s
type: static
load_assignment:
cluster_name: test-cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: my_private_ip_address
port_value: 443
I have tried the auto_host_rewrite but found that it does not works with static cluster type. but it does not updates with host_literal too.
it turns out that the config is working exactly as expected. I was looking at wrong header to verify it in the NGINX access logs. lesson learned.

Resources