I am using RabbitMQ server for send message in my Symfony2 application.
I have used OldSoundRabbitMqBundle for this.
After successful installation of RabbitMq server on my application server it is working fine.
But when I install RabbitMQ server on different machine and try to connect it from my application server it is not connecting.
I have given the connection config as follows:
old_sound_rabbit_mq:
connections:
default:
host: myrabbitserverIp
port: 80
user: 'test'
password: 'test'
vhost: '/'
lazy: false
producers:
messages:
connection: default
exchange_options: {name: 'messages', type: direct}
consumers:
messages:
connection: default
exchange_options: {name: 'messages', type: direct}
queue_options: {name: 'messages'}
callback: message.amqp_consumer
Do I need to change any configuration for RabbitMq server?
Related
I'm trying to config envoy as rest api gateway with multiple grpc servers and have a problem with routing. The only way to match endpoint to grpc cluster, that i've found is to match via request header (http request /first must be resolved by first cluster, /second - by second):
...
routes:
- match:
prefix: "/"
headers:
- name: x-service
exact_match: "first"
route:
cluster: first
- match:
prefix: "/"
headers:
- name: x-service
exact_match: "second"
route:
cluster: second
...
But, in this case i need to set custom header 'x-service' at the client (frontend). This looks like a bad idea, 'couse frontend shouldn't know anything about backend infrastructure.
Is there any other way to match http route with grpc service? Or, can i set such headers somewhere in envoy?
The Envoy configuration pasted below registers a HTTP listener on port 51051 that proxies to helloworld.Greeter service in the cluster grpc1 on port 50051 and bookstore.Bookstore service in the cluster grpc2 on port 50052 by using the gRPC route as the match prefix.
This ensures clean segregation of responsibilities and isolation since the client will not need to inject custom HTTP headers to make multi-gRPC cluster routing work.
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener1
address:
socket_address: { address: 0.0.0.0, port_value: 51051 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
access_log:
- name: envoy.access_loggers.file
typed_config:
"#type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: /dev/stdout
stat_prefix: grpc_json
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
# NOTE: by default, matching happens based on the gRPC route, and not on the incoming request path.
# Reference: https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/grpc_json_transcoder_filter#route-configs-for-transcoded-requests
- match: { prefix: "/helloworld.Greeter" }
route: { cluster: grpc1, timeout: 60s }
- match: { prefix: "/bookstore.Bookstore" }
route: { cluster: grpc2, timeout: 60s }
clusters:
- name: grpc1
connect_timeout: 1.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
dns_lookup_family: V4_ONLY
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"#type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http2_protocol_options: {}
load_assignment:
cluster_name: grpc1
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 50051
- name: grpc2
connect_timeout: 1.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
dns_lookup_family: V4_ONLY
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"#type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http2_protocol_options: {}
load_assignment:
cluster_name: grpc2
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 50052
https://github.com/envoyproxy/envoy/blob/main/test/proto/helloworld.proto
syntax = "proto3";
package helloworld;
import "google/api/annotations.proto";
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello(HelloRequest) returns (HelloReply) {
option (google.api.http) = {
get: "/say"
};
}
}
https://github.com/envoyproxy/envoy/blob/main/test/proto/bookstore.proto
syntax = "proto3";
package bookstore;
import "google/api/annotations.proto";
import "google/api/httpbody.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/struct.proto";
// A simple Bookstore API.
//
// The API manages shelves and books resources. Shelves contain books.
service Bookstore {
// Returns a list of all shelves in the bookstore.
rpc ListShelves(google.protobuf.Empty) returns (ListShelvesResponse) {
option (google.api.http) = {
get: "/shelves"
};
}
...
I have a GRPC Web client and a GRPC Server and I am using envoy proxy from the conversion of HTTP 1.1 to HTTP2.
My server creation Logic uses TLS. The code is as follows:
var opts []grpc.ServerOption
creds, err := credentials.NewServerTLSFromFile("cert/server.crt", "cert/server.key")
if err != nil {
log.Fatalf("Failed to generate credentials %v", err)
}
opts = []grpc.ServerOption{grpc.Creds(creds)}
server := grpc.NewServer(opts...)
I am calling the From my react client as follows:
const client = new LiveClient('http://localhost:8080')
const request = new GetLiveRequest()
request.setApi(1)
request.setTrackkey(trackKey)
// on success response
const stream = client.getLive(request, {})
stream.on('data', response => {
console.log(response);
}
The envoy.yaml is as follows:
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 8080 }
filter_chains:
- filters:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route:
cluster: greeter_service
max_grpc_timeout: 0s
cors:
allow_origin:
- "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
max_age: "1728000"
expose_headers: custom-header-1,grpc-status,grpc-message
http_filters:
- name: envoy.grpc_web
- name: envoy.cors
- name: envoy.router
tls_context:
common_tls_context:
alpn_protocols: "h2"
tls_certificates:
- certificate_chain:
filename: "/etc/server.crt"
private_key:
filename: "/etc/server.key"
clusters:
- name: greeter_service
connect_timeout: 0.25s
type: logical_dns
http2_protocol_options: {}
lb_policy: round_robin
hosts: [{ socket_address: { address: app, port_value: 3000 }}]
The Dockerfile for envoy is as follows:
FROM envoyproxy/envoy:36f39c746eb7d03b762099b206403935b11972d8
COPY ./envoy.yaml /etc/envoy/envoy.yaml
ADD ./cert/server.crt /etc/server.crt
ADD ./cert/server.key /etc/server.key
ADD ./cert/server.csr /etc/server.csr
WORKDIR /etc/envoy
CMD /usr/local/bin/envoy -c /etc/envoy/envoy.yaml
I am getting the following error when:
{code: 2, message: "Http response at 400 or 500 level"}
But when I remove the SSL authentication from backend server. It is working fine. I have also created a grpc client and TLS is working fine with it.
I am unable to find what is going wrong in my envoy configuration for TLS.
On further investigation in am getting following in envoy logs.
TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
The TLS certificates are working fine if I use it with envoy by directly using a GRPC client.
Your backend is already talking HTTPS through. So you don't need to configure tls_context in the envoy's config. And you have to use tcp_proxy instead of http_connection_manager Here. You also need to configure transport_socket for the proxy of your TLS backend Here.
static_resources:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/config/listener/v3/listener.proto#config-listener-v3-listener
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto#extensions-filters-network-http-connection-manager-v3-httpconnectionmanager
"#type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
access_log:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/access_loggers/file/v3/file.proto
#
# You can also configure this extension with the qualified
# name envoy.access_loggers.http_grpc
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/access_loggers/grpc/v3/als.proto
- name: envoy.access_loggers.file
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/access_loggers/file/v3/file.proto#extensions-access-loggers-file-v3-fileaccesslog
"#type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
# Console output
path: /dev/stdout
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains:
- "*"
routes:
- match:
prefix: /
grpc:
route:
cluster: greeter_service
cors:
allow_origin_string_match:
- prefix: "*"
allow_methods: GET, PUT, DELETE, POST, OPTIONS
# custom-header-1 is just an example. the grpc-web
# repository was missing grpc-status-details-bin header
# which used in a richer error model.
# https://grpc.io/docs/guides/error/#richer-error-model
allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,grpc-status-details-bin,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout,authorization
expose_headers: grpc-status-details-bin,grpc-status,grpc-message,authorization
max_age: "1728000"
http_filters:
- name: envoy.filters.http.grpc_web
# This line is optional, but adds clarity to the configuration.
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/filters/http/grpc_web/v3/grpc_web.proto
"#type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
- name: envoy.filters.http.cors
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/filters/http/cors/v3/cors.proto
"#type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
- name: envoy.filters.http.router
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/filters/http/router/v3/router.proto
"#type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/transport_sockets/tls/v3/tls.proto#extensions-transport-sockets-tls-v3-downstreamtlscontext
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
# Certificate must be PEM-encoded
filename: /etc/fullchain.pem
private_key:
filename: /etc/privkey.pem
clusters:
# https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/config/cluster/v3/cluster.proto#config-cluster-v3-cluster
- name: greeter_service
type: LOGICAL_DNS
connect_timeout: 0.25s
lb_policy: round_robin
load_assignment:
cluster_name: greeter_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: app
port_value: 3000
http2_protocol_options: {} # Force HTTP/2
# Your grpc server communicates over TLS. You must configure the transport
# socket. If you care about the overhead, you should configure the grpc
# server to listen without TLS. If you need to listen to grpc-web and grpc
# over HTTP/2 both you can also proxy your TCP traffic with the envoy.
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"#type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
Have you tried calling HTTPS from client?
const client = new LiveClient('https://localhost:8080')
Without this I'm getting Http response at 400 or 500 level as well.
I'm trying to connect to cosmos db through the gremlin console 3.3.4, following this the remote_secure.yaml is as follows:
hosts: [*****.gremlin.cosmosdb.azure.com]
port: 443
username: /dbs/sample-database/colls/sample-collection
password: ******
connectionPool: {
enableSsl: true}
{ className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { serializeResultToString: true }}:
but when I run :remote connect tinkerpop.server conf/remote-secure.yaml
I get the following error
==>Error during 'connect' - Can't construct a java object
for tag:yaml.org,2002:org.apache.tinkerpop.gremlin.driver.Settings;
exception=Keys must be scalars but found:
<org.yaml.snakeyaml.nodes.MappingNode (tag=tag:yaml.org,2002:map,
values={ key=<org.yaml.snakeyaml.nodes.ScalarNode (tag=tag:yaml.org,2002:str, value=className)>;
value=<NodeTuple
keyNode=<org.yaml.snakeyaml.nodes.ScalarNode (tag=tag:yaml.org,2002:str,
value=className)>; valueNode=<org.yaml.snakeyaml.nodes.ScalarNode
(tag=tag:yaml.org,2002:str,
value=org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0)
>> }{ key=<org.yaml.snakeyaml.nodes.ScalarNode
(tag=tag:yaml.org,2002:str, value=config)>; value=828088650 })>
in 'reader', line 27, column 1:
hosts: [*****.gremlin.cosm ...
Any ideas what I am doing wrong?
Looks like your configuration is mangled. You are missing the serializer key on that last line:
hosts: [*****.gremlin.cosmosdb.azure.com]
port: 443
username: /dbs/sample-database/colls/sample-collection
password: ******
connectionPool: {
enableSsl: true}
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { serializeResultToString: true }}
I'm trying to implement RabbitMQ with https://github.com/php-amqplib/RabbitMqBundle and Symfony2 framework.
I've managed to make the thing work with 1 producer and 1 consumer but the problem is when i use multiple consumers.
This is my configuration:
old_sound_rabbit_mq:
connections:
default:
host: 'localhost'
port: 5672
user: 'guest'
password: 'guest'
vhost: '/'
lazy: false
connection_timeout: 3
read_write_timeout: 3
# requires php-amqplib v2.4.1+ and PHP5.4+
keepalive: false
# requires php-amqplib v2.4.1+
heartbeat: 0
#requires php_sockets.dll
# use_socket: true # default false
producers:
soccer_team_stat:
connection: default
exchange_options: {name: 'soccer_team_stat_ex', type: direct}
queue_options: {name: 'soccer_team_stat_qu'}
soccer_team_stat_form:
connection: default
exchange_options: {name: 'soccer_team_stat_ex', type: direct}
queue_options: {name: 'soccer_team_stat_form_qu'}
consumers:
soccer_team_stat:
connection: default
exchange_options: {name: 'soccer_team_stat_ex', type: direct}
queue_options: {name: 'soccer_team_stat_qu'}
callback: myapp.soccer_team_stat.consume
soccer_team_stat_form:
connection: default
exchange_options: {name: 'soccer_team_stat_ex', type: direct}
queue_options: {name: 'soccer_team_stat_form_qu'}
callback: myapp.soccer_team_stat_form.consume
Service definitions:
<services>
<service class="MyApp\EtlBundle\Producers\SoccerTeamStatProducer" id="myapp.soccer_team_stat.produce">
<argument type="service" id="old_sound_rabbit_mq.soccer_team_stat_producer"/>
</service>
<service class="MyApp\EtlBundle\Producers\SoccerTeamStatProducer" id="myapp.soccer_team_stat_form.produce">
<argument type="service" id="old_sound_rabbit_mq.soccer_team_stat_producer"/>
</service>
<service class="MyApp\EtlBundle\Consumers\SoccerTeamStatConsumer" id="myapp.soccer_team_stat.consume">
<argument type="service" id="service_container"/>
</service>
<service class="MyApp\EtlBundle\Consumers\SoccerTeamStatFormConsumer" id="myapp.soccer_team_stat_form.consume">
<argument type="service" id="service_container"/>
</service>
</services>
And on php app/console rabbitmq:consumer -d soccer_team_stat_form i get:
[Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException]
You have requested a non-existent service
"old_sound_rabbit_mq.soccer_team_stat_form_consumer".
I tried various combinations includin using multiple_consumers configuration key but with no success. What i'm i missing?
If neither of routing_key and binding_key are set, direct exchange will behave like fanout and send the messages to all the queues it knows so based on what I'm seeing from your configurations, you are better off using fanout so do like below.
old_sound_rabbit_mq:
connections:
default:
host: %rabbit_mq_host%
port: %rabbit_mq_port%
user: %rabbit_mq_user%
password: %rabbit_mq_pswd%
vhost: /
lazy: true
producers:
soccer_team_stat:
connection: default
exchange_options: { name: 'soccer_team_stat_ex', type: fanout }
soccer_team_stat_form:
connection: default
exchange_options: { name: 'soccer_team_stat_form_ex', type: fanout }
consumers:
soccer_team_stat:
connection: default
exchange_options: { name: 'soccer_team_stat_ex', type: fanout }
queue_options: { name: 'soccer_team_stat_qu' }
callback: myapp.soccer_team_stat.consume
soccer_team_stat_form:
connection: default
exchange_options: { name: 'soccer_team_stat_form_ex', type: fanout }
queue_options: { name: 'soccer_team_stat_form_qu' }
callback: myapp.soccer_team_stat_form.consume
This RabbitMQ fanout example with symfony including 2 Producer & 2 Exchange & 2 Queue & N Worker & 2 Consumer is the full example (actually the full answer to your question/already made version of what you want to do) that shows how things are done within symfony apps. I would suggest you to follow the pattern used there. Very easy to follow and maintain. If you want more examples, just search for RabbitMQ keyword in that blog.
I m actually trying to create a simple blog using ghost, and I m facing a problem when starting in production envrionnement.
I m having the v0.7.1 and here's my config file (production part)
production: {
url: 'http://<my-public-ip>',
mail: {},
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost.db')
},
debug: false
},
server: {
host: '127.0.0.1',
port: '2368'
}
}
The fact is that when I try to access my public IP on a browser, I cant get anything at all on the screen(404 not found), even if I try on the 2368 port.
My firewall rules are well set.
what am I doing wrong ?
In the server object the host should be 0.0.0.0
server: {
host: '0.0.0.0',
port: '2368'
}
In the server object change the host .
host: '127.0.0.1', --> host: '0.0.0.0'
now start the ghost server by
npm start --production