We have tried to get the error logs in JSON format as we can do for access logs. But we are getting error there.
Can we not format the error logs we are getting from NGINX server to JSON format as we can do for access logs?
As described here
http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format
json log format is available in nginx
log_format logger-json escape=json '{
"source": "nginx",
"time": $msec,
"resp_body_size": $body_bytes_sent,
"host": "$http_host",
"address": "$remote_addr",
"request_length": $request_length,
"method": "$request_method",
"uri": "$request_uri",
"status": $status,
"user_agent": "$http_user_agent",
"resp_time": $request_time,
"upstream_addr": "$upstream_addr"
}';
At the moment is not possible to format the error logs as JSON.
Related
gRPC services (developed in springboot) deployed as docker container on AWS linux (ec2). Started the docker image with port forwarding -p6565:6565.
Now when directly hit via BloomRPC on laptop, it worked : ec2.IP:6565 Package.Service.Method
Configured service & route in Kong:
{
"host": "ec2.IP",
"created_at": 1588403433,
"connect_timeout": 60000,
"id": "e657d8df-6247-458a-a8e8-bec00c41e03c",
"protocol": "grpc",
"name": "aws-grpc1",
"read_timeout": 60000,
"port": 6565,
"path": null,
"updated_at": 1588403433,
"retries": 5,
"write_timeout": 60000,
"tags": null,
"client_certificate": null
}
Route:
{
"strip_path": false,
"path_handling": "v0",
"updated_at": 1588403452,
"destinations": null,
"headers": null,
"protocols": [
"grpc",
"grpcs"
],
"created_at": 1588403452,
"snis": null,
"service": {
"id": "e657d8df-6247-458a-a8e8-bec00c41e03c"
},
"name": "aws-grpc1-route1",
"methods": null,
"preserve_host": false,
"regex_priority": 0,
"paths": [
"/grpc2"
],
"sources": null,
"id": "5739297e-3be7-4a0d-8afb-cfa8ed01cec2",
"https_redirect_status_code": 426,
"hosts": null,
"tags": null
}
Now hitting it via grpcurl -> its not working:
grpcurl -v -d "{}" -insecure ec2.ip:8443 package.service.pingMethod
Error invoking method "package.service.ping": target server does not expose service "package.service"
Here is kong config which looks related:
"proxy_listen": [
"0.0.0.0:8000 reuseport backlog=16384",
"0.0.0.0:8443 **http2** ssl reuseport backlog=16384"
],
So here are queries:
(1) can 8000 also be configured for https as insecure -> via passing a env KONG_PROXY_LISTEN variable at time of kong-container start by
-e "KONG_PROXY_LISTEN=0.0.0.0:8000 http2, 0.0.0.0:8443 http2 ssl"
Is this good to do?
(2) How to enable server side reflection? OR what is use of /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo ?
You need to expose HTTP2 Proxy Listener for Kong.
You can refer to this one: https://konghq.com/blog/manage-grpc-services-kong/
In short, you need to add env variable details for KONG_PROXY_LISTEN like so:
-e "KONG_PROXY_LISTEN=0.0.0.0:8000 http2, 0.0.0.0:8443 http2 ssl, 0.0.0.0:9080 http2, 0.0.0.0:9081 http2 ssl"
Note: apparently Kong uses the ports 9080 for HTTP2 and 9081 for HTTP2 SSL. But I think this can be changed.
And also expose those 9080 and 9081 ports like so, this is example for docker run command:
-p 127.0.0.1:9080:9080 \
-p 127.0.0.1:9081:9081
And use the 9080 port in grpcurl when you try to request, like so:
grpcurl -v -d '{"name": "Ken"}' -plaintext localhost:9080 facade.GreetingService/SayHello
More updates:
gRPC deployed behind kong.ingress is working fine:
grpcurl -v -d "{\"greeting\":\"111\"}" -insecure acfb0xxxxx.elb.us-east-2.amazonaws.com:443 hello.HelloService.SayHello
Response:
Resolved method descriptor:
rpc SayHello ( .hello.HelloRequest ) returns ( .hello.HelloResponse );
Request metadata to send:
(empty)
Response headers received:
content-type: application/grpc
date: Sat, 02 May 2020 07:00:17 GMT
server: openresty
trailer: Grpc-Status
trailer: Grpc-Message
trailer: Grpc-Status-Details-Bin
via: kong/2.0.3
x-kong-proxy-latency: 1
x-kong-upstream-latency: 9
Response contents:
{
"reply": "hello 111"
}
Response trailers received:
(empty)
Sent 1 request and received 1 response
when configured on kong-API-gateway, it is not working:
grpcurl -v -d "{\"greeting\":\"111\"}" -insecure kong.ce-gateway.ip:8443 hello.HelloService.SayHello
Error invoking method "hello.HelloService.SayHello": failed to query for service descriptor "hello.HelloService": rpc error: code = Internal desc = An invalid response was received from the upstream server
Http2 is now enabled by default for Kong, but if you are having issues, a good place to start is to inspect the proxy_listeners section of the global config. In my case, I found that http2 was only enabled for the SSL port, and not for the non SSL. A good way to see the global config is to send a GET request to the root url of the admin api, for example GET http://localhost:8001/.
I am trying to use lets encrypt with docker in order to put my website in https.
I use docker with nginx proxy and nginx companion. I have set up everything correctly regarding documentation. My containers are running.
Now, i have an issue with lets encrypt here is the debug file provided :
{
"identifier": {
"type": "dns",
"value": "jack-world.com"
},
"status": "invalid",
"expires": "2017-12-20T18:42:39Z",
"challenges": [
{
"type": "tls-sni-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/G_0PYv_VpnEEUbV1PUjpJZyOIeP6b0zPxXeAlyYXclE/2728472678",
"token": "fXuUQ77koLDDTuAqEgeqQA1q_DHinF2wanQReSrgIdk"
},
{
"type": "dns-01",
"status": "pending",
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/G_0PYv_VpnEEUbV1PUjpJZyOIeP6b0zPxXeAlyYXclE/2728472680",
"token": "iab5h37N-Io6lzfi8-DKmccXsF8_Y5Ws_RYCcwzREBw"
},
{
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:acme:error:unauthorized",
"detail": "The key authorization file from the server did not match this challenge [fnFwM8VZXXjIkSOci-z5_w4W2mN8oOIXA_d74gScLo0.K6eBCVMCFTPDy-GGls8jpd0O75tW9kFA9tsX7dEU_Zw] != [fnFwM8VZXXjIkSOci-z5_w4W2mN8oOIXA_d74gScLo0.4E3VCTFsySjUrqnCg0ooULx-3kbdPBygi0aWkvg5Gd8]",
"status": 403
},
"uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/G_0PYv_VpnEEUbV1PUjpJZyOIeP6b0zPxXeAlyYXclE/2728472682",
"token": "fnFwM8VZXXjIkSOci-z5_w4W2mN8oOIXA_d74gScLo0",
"keyAuthorization": "fnFwM8VZXXjIkSOci-z5_w4W2mN8oOIXA_d74gScLo0.K6eBCVMCFTPDy-GGls8jpd0O75tW9kFA9tsX7dEU_Zw",
"validationRecord": [
{
"url": "http://jack-world.com/.well-known/acme-challenge/fnFwM8VZXXjIkSOci-z5_w4W2mN8oOIXA_d74gScLo0",
"hostname": "jack-world.com",
"port": "80",
"addressesResolved": [
"149.202.73.189",
"2001:41d0:301::21"
],
"addressUsed": "2001:41d0:301::21",
"addressesTried": []
}
]
}
],
"combinations": [
[
0
],
[
1
],
[
2
]
]
}
Here is logs from companion :
argos#jackworld:~/JackProxy$ sudo docker exec jackproxy_nginx-proxy-companion_1 /app/force_renew -v --help
/etc/nginx/certs/jack-world.com /app
Creating/renewal jack-world.com certificates... (jack-world.com)
2017-12-13 19:03:34,715:INFO:simp_le:1538: Retrieving Let's Encrypt latest Terms of Service.
2017-12-13 19:03:36,629:INFO:simp_le:1455: Generating new certificate private key
2017-12-13 19:03:37,221:ERROR:simp_le:1421: CA marked some of the authorizations as invalid, which likely means it could not access http://example.com/.well-known/acme-challenge/X. Did you set correct path in -d example.com:path or --default_root? Are all your domains accessible from the internet? Please check your domains' DNS entries, your host's network/firewall setup and your webserver config. If a domain's DNS entry has both A and AAAA fields set up, some CAs such as Let's Encrypt will perform the challenge validation over IPv6. If you haven't setup correct CAA fields or if your DNS provider does not support CAA, validation attempts after september 8, 2017 will fail. Failing authorizations: https://acme-v01.api.letsencrypt.org/acme/authz/Xw790v5P8mgdjsh-A-_wvwcmAFRIu-6UxlT2l5I7JB8
Challenge validation has failed, see error log.
Debugging tips: -v improves output verbosity. Help is available under --help.
/app
I need some help to figure out why http-01 is invalid, and if this is the only issue.
Thanks by advance
I set my CORS header for Firebase Storage with the following cors.json and gsutil
[
{
"origin": ["*"],
"method": ["GET", "OPTIONS"],
"responseHeader": ["Range", "Content-Type"],
"maxAgeSeconds": 3600
}
]
But I still get this error:
MLHttpRequest cannot load http://url_to.csv. Request header field
Range is not allowed by Access-Control-Allow-Headers in preflight
response.
I also tried replacing Range with Content-Range but it didn't have any effect.
New to Firebase hosting. I deployed few folders containing images to Firebase hosting.
Today I am facing below issue. I have no clue about it. Any help on this is highly appreciated.
ERROR:
=== Deploying to 'admob-app-id-4307XXXXXX'...
i deploying database, hosting
+ database: rules ready to deploy.
i hosting: preparing public directory for upload...
**Error: Server Error. read ECONNRESET**
C:\Users\NAME\Documents\Projects\Website\firebase>firebase deploy
**Error: HTTP Error: 500, An unknown error occurred. Please contact support.**
Snippet from debug log:
[debug] ----------------------------------------------------------------------
[debug] Command: C:\Program Files\nodejs\node.exe C:\Users\NAME\AppData\Roaming\npm\node_modules\firebase-tools\bin\firebase deploy
[debug] CLI Version: 3.4.0
[debug] Platform: win32
[debug] Node Version: v6.10.0
[debug] Time: Mon Mar 13 2017 14:07:38 GMT+0530 (India Standard Time)
[debug] ----------------------------------------------------------------------
[debug]
[debug] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase"]
[debug] >>> HTTP REQUEST GET https://admin.firebase.com/v1/projects/admob-app-id-XXXXXXXXXX
Mon Mar 13 2017 14:07:38 GMT+0530 (India Standard Time)
[debug] <<< HTTP RESPONSE 200
[debug] >>> HTTP REQUEST GET https://admin.firebase.com/v1/database/admob-app-id-XXXXXXXXXX/tokens
Mon Mar 13 2017 14:07:40 GMT+0530 (India Standard Time)
[debug] <<< HTTP RESPONSE 500
[debug] <<< HTTP RESPONSE BODY
[error]
[error] Error: HTTP Error: 500, An unknown error occurred. Please contact support.
[debug] Error Context: {
"body": {
"error": {
"code": "UNKNOWN_ERROR",
"message": "An unknown error occurred. Please contact support."
}
},
"response": {
"statusCode": 500,
"body": {
"error": {
"code": "UNKNOWN_ERROR",
"message": "An unknown error occurred. Please contact support."
}
},
"headers": {
"server": "nginx",
"date": "Mon, 13 Mar 2017 08:37:34 GMT",
"content-type": "application/json; charset=utf-8",
"content-length": "97",
"connection": "close",
"x-content-type-options": "nosniff"
},
"request": {
"uri": {
"protocol": "https:",
"slashes": true,
"auth": null,
"host": "admin.firebase.com",
"port": 443,
"hostname": "admin.firebase.com",
"hash": null,
"search": null,
"query": null,
"pathname": "/v1/database/admob-app-id-XXXXXXXXXX/tokens",
"path": "/v1/database/admob-app-id-XXXXXXXXXX/tokens",
"href": "https://admin.firebase.com/v1/database/admob-app-id-XXXXXXXXXX/tokens"
},
"method": "GET"
}
}
}
Much thanks!
I just encountered this issue while deploying to Firebase, the solution is what lead me to this post. After making sure that my internet connection was good, i simply deployed successfully.
Make sure your internet is fine, then redeploy!
GoodLuck!
What's the total size of all those folders.
Error: Server Error. read ECONNRESET
looks like network issues. Or maybe you are getting a reset packet after tripping an undocumented limit in Firebase.
Firebase probably uploads a tar or zip of the files. Why don't you try adding half the amount of folders and images at first, and if that works..
ps. I doubt firebase does a diff and only uploads what's NOT there. So you probably can't incrementally add files to get around the limit. All the above method shows you is how much you can get away with before there's a reset.
Try going through a VPN, it worked for me.
I need to use couple of iframe for a page hosted with firebase, but its giving me X-Frame-Options error, one of the iframe is for gallery hosted on picasa, and anohter ifrmae for contact form(because i couldnt sent email via firebase :()
here is error
Refused to display 'https://get.google.com/albumarchive/pwa/11111/album/1111?source=pwa#slideshow/1111' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'.
jquery.min.js:2 Uncaught DOMException: Failed to read the 'contentDocument' property from 'HTMLIFrameElement': Blocked a frame with origin "https://demodomain.com" from accessing a cross-origin frame.
i did this with firebase.json but didnt worked
"headers": [
{
"source": "**/*",
"headers": [
{"key": "X-Content-Type-Options", "value": "nosniff"},
{"key": "X-Frame-Options", "value": "ALLOW"},
{"key": "X-UA-Compatible", "value": "ie=edge"},
{"key": "X-XSS-Protection", "value": "1; mode=block"}
]
}
]
you have the right idea you're just setting the wrong value. ALLOW is not an acceptable value for the X-Frame-Options header. You can set the ALLOW-FROM value and then specify which uri you want to allow to be able to embed. Check out some more documentation below.
FIX:
"headers": [{
"source": "**/*",
"headers": [
{"key": "X-Content-Type-Options", "value": "nosniff"},
{"key": "X-Frame-Options", "value": "ALLOW-FROM https://get.google.com"},
{"key": "X-UA-Compatible", "value": "ie=edge"},
{"key": "X-XSS-Protection", "value": "1; mode=block"}
]
}]
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options