I'm using Serilog with the Serilog.Formatting.Json.JsonFormatter formatter in a .NET Core app in GKE. I am logging to Console, which is read by a GKE Logging agent. The GKE logging agent expects a "severity" property at the top level of the Log Event: GCP Cloud Logging LogEntry docs
Because of this, all of my logs show up in GCP Logging with severity "Info", as the Serilog Level is found in the jsonPayload property of the LogEntry in GCP. Here is an example LogEntry as seen in Cloud Logging:
{
insertId: "1cu507tg3by7sr1"
jsonPayload: {
Properties: {
SpanId: "|a85df301-4585ee48ea1bc1d1."
ParentId: ""
ConnectionId: "0HM64G0TCF3RI"
RequestPath: "/health/live"
RequestId: "0HM64G0TCF3RI:00000001"
TraceId: "a85df301-4585ee48ea1bc1d1"
SourceContext: "CorrelationId.CorrelationIdMiddleware"
EventId: {2}
}
Level: "Information"
Timestamp: "2021-02-03T17:40:28.9343987+00:00"
MessageTemplate: "No correlation ID was found in the request headers"
}
resource: {2}
timestamp: "2021-02-03T17:40:28.934566174Z"
severity: "INFO"
labels: {3}
logName: "projects/ah-cxp-common-gke-np-946/logs/stdout"
receiveTimestamp: "2021-02-03T17:40:32.020942737Z"
}
My first thought was to add a "Severity" property using an Enricher:
class SeverityEnricher : ILogEventEnricher
{
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
logEvent.AddOrUpdateProperty(
propertyFactory.CreateProperty("Severity", LogEventLevel.Error));
}
}
The generated log looks like this in GCP, and is still tagged as Info:
{
insertId: "wqxvyhg43lbwf2"
jsonPayload: {
MessageTemplate: "test error!"
Level: "Error"
Properties: {
severity: "Error"
}
Timestamp: "2021-02-03T18:25:32.6238842+00:00"
}
resource: {2}
timestamp: "2021-02-03T18:25:32.623981268Z"
severity: "INFO"
labels: {3}
logName: "projects/ah-cxp-common-gke-np-946/logs/stdout"
receiveTimestamp: "2021-02-03T18:25:41.029632785Z"
}
Is there any way in Serilog to add the "severity" property at the same level as "jsonPayload" instead of inside it? I suspect GCP would then pick it up and log the error type appropriately.
As a last resort I could probably use a GCP Logging sink, but my current setup is much more convenient and performant with the GKE Logging Agent already existing.
Here's a relevant Stack Overflow post with no information or advice past what I already have, which is not enough to solve this: https://stackoverflow.com/questions/57215700
I found the following information detailing the severity of each SeriLog to Stackdriver log level, the next table might also help you
Serilog
Stackdriver
Verbose
Debug
Debug
Debug
Information
Info
Warning
Warning
Error
Error
Fatal
Critical
The complete information can be found at the following link
https://github.com/manigandham/serilog-sinks-googlecloudlogging#log-level-mapping
I think this code could help you to make Stackdriver recognize the severity of the logs given by SeriLogs.
private static LogSeverity TranslateSeverity(LogEventLevel level) => level switch
{
LogEventLevel.Verbose => LogSeverity.Debug,
LogEventLevel.Debug => LogSeverity.Debug,
LogEventLevel.Information => LogSeverity.Info,
LogEventLevel.Warning => LogSeverity.Warning,
LogEventLevel.Error => LogSeverity.Error,
LogEventLevel.Fatal => LogSeverity.Critical,
_ => LogSeverity.Default
};
I will leave the link to the complete code here
https://github.com/manigandham/serilog-sinks-googlecloudlogging/blob/master/src/Serilog.Sinks.GoogleCloudLogging/GoogleCloudLoggingSink.cs#L251
Greetings!
I am using telegram.php to connect my bot. When I use sendmessage all of thing is ok in my logs but I do not receive anything from the bot.
When I check my log there is a problem like this:
ok: False
curl_error_code: 51
curl_error: SSL: no alternative certificate subject name matches target host name 'api.telegram.org'
I donit know what to do to fix it.
I don't know this telegram bot, but I see that it uses GuzzleHttp.
During the initialization it doesn't accept any configuration Request::initialize()
public static function initialize(Telegram $telegram)
{
if (!($telegram instanceof Telegram)) {
throw new TelegramException('Invalid Telegram pointer!');
}
self::$telegram = $telegram;
self::setClient(new Client(['base_uri' => self::$api_base_uri]));
}
you should check its documentation. I see that there are a lot of setters which makes you able to overwrite the default settings.
What you need is to set the the \GuzzleHttp\RequestOptions::VERIFY to false in the client config:
$this->client = new \GuzzleHttp\Client([
'base_uri' => 'someAccessPoint',
\GuzzleHttp\RequestOptions::HEADERS => [
'User-Agent' => 'some-special-agent',
],
'defaults' => [
\GuzzleHttp\RequestOptions::CONNECT_TIMEOUT => 5,
\GuzzleHttp\RequestOptions::ALLOW_REDIRECTS => true,
],
\GuzzleHttp\RequestOptions::VERIFY => false,
]);
For fix this problem copy this Url to browser and set webhook:
https://api.telegram.org/botTOKEN/setWebhook?url=https://yourwebsite.com
Solution 2 of The Error
Let’s follow these simple steps:
Download this bundle of root certificates: https://curl.haxx.se/ca/cacert.pem
Put in any location of your server.
Open php.ini and add this line:
curl.cainfo = "[the_location]\cacert.pem"
Restart your webserver.
That’s it. 🙂
I'm new to kibana.I am working with data migration from MySQL to elasticsearch.How can i do this? Is using jdbc input plugin is the only process??
Here is the logstash.conf file where i specified the input and output:
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/kibana"
jdbc_user => "xxx"
jdbc_password => "xxxxx"
jdbc_driver_library => "/root/mysql-connector-java-5.1.30/mysql-connector-java-5.1.30-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
statement => "SELECT * FROM datalog"
}
}
output {
elasticsearch {
"hosts" => "localhost:9200"
}
stdout { codec => rubydebug }
}
after running the above file by using ./logstash -f logstash.conf we are getting the below error:
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
I'm trying to monitor a symfony app with the ELK stack.
I'm shipping my logs to logstash with the following configuration :
monolog:
handlers:
main:
type: gelf
publisher:
hostname: elk-host
port: 10514
formatter: monolog.formatter.gelf_message
level: INFO
On kibana, I see that I reiceive the logs but the message is encoded in a strange way; here is an example of what kibana displays :
x\x9CMP\xC1n\x830\f\xFD\u0015+\xA7V\xAAB\xA1\f(\xD7j;Nڴ\xDD\"Ui0`)\u0004D\(\x9A\xF6\xEF\v\x9B\xD6\xEDf\xBFg\xFB\xF9\xBD\u000F1\xE1\xE8\xA9w\xA2\u0014\xB1܋\x9Dh{ϡ\u0019\xFA\x915Y\xCF^\xDA\xDEh\e\u0018\xDF\u0006\xECܡ\xF7\xBA\xC10\xF2\x8A5\x8E\xE8\f\xB9\u0006\xB8EP\xC2\xF4#*\u0001xct\xEBQ\xB8,#\xEC\xC1\xE9\u000EaSaM\u000E\xAB\u0015l\x90\x9F\u0003\xB6\xD9n\x81
Here is my monolog configuration file :
input {
gelf {
codec => "json"
}
syslog {
port => 10514
type => "syslog"
}
}
filter {
}
output {
elasticsearch {}
}
I tried to add an encoding option (charset => "UTF-8") but it was not better.
Also why are my logs displayed as "syslog" type instead of "gelf" that I specified in monolog config ?
Your sending GELF (JSON) output to a SYSLOG listener, you need to change to send it to the GELF port rather than the SYSLOG port
I am using mrt add accounts-ui-bootstrap-dropdown and mrt add accounts-password to get a simple login page running on my app.
The accounts users gives me a nice hash containing ids, createdAt, emails, etc.
If I wanted to add other fields in this hash so I can make use of them later, how would I do that? For example, I want then to also enter their given name and surname:
"given_name": "John", "surname": "Doe"
Users are special objects in meteor ; you don't want to add fields in the user but in the users profile.
From the doc :
By default the server publishes username, emails, and profile.
If you want to add properties like surname when you create the account, you should use in the Account.onCreateUser server-side hook : http://docs.meteor.com/#accounts_oncreateuser
Accounts.onCreateUser(function(options, user) {
//pass the surname in the options
user.profile['surname'] = options.surname
return user
}
If you want to update a user after, you can do it from the client that way :
Meteor.users.update({_id:Meteor.user()._id}, { $set: {what you want to update} });
By default, the users base will allow that (the current user may update itself). If you don't trust your users and want to ensure that everything is properly update, you can also forbid any updates from the client and make them via a Meteor.call() and proceed to the checkings server-side. But this would be sad.
Edit :
As said in the comments, adding options via the standard account-ui won't be possible. You'll only be able to update the user after the registration. To add options when you subscribe, you'll have to make you own form.
I won't insult you by writing html markup, but here is what you want to have after the submit event (and after the various checking) :
var options = {
username: $('input#username')[0].value,
emails: [{
address: $('input#email')[0].value,
verified: false
}],
password: $('input#password')[0].value,
profile: {
surname: $('input#surname')
},
};
Accounts.createUser( options , function(err){
if( err ) $('div#errors').html( err.message );
});
You only need the account-base package ; not the account-ui.
Login with the social networks is cake :
Meteor.loginWithFacebook({
requestPermissions: ['email', 'user_birthday', 'user_location']
}, function(error){loginCallBack(error);});
About the answer ram1 made :
This is not the way meteor works. You do not "POST" a form. You want all your client / server communication done via the websocket. The equivalent of what you are talking about is making a "Meteor.call('myserverfunction', myarguments, mycallback)" of a server method from the client and you pass the arguments you want the server to use.
But this is not the way you will get the best of meteor. There is the philosophy you want to work with :
you have datas in your local mini mongo you got from the server
you update locally those datas in your base / view
meteor do his magic to transmit those updates to the server
there the server can answer : ok, updates saved, this is seamless for you. Or answer : nop ! reverse the changes (and you can implement an error notification system)
(it can answer no because you don't have the permission to update this field, because this update break a rule you did set up...)
All you do is setting permissions and controls on the databases server-side. That way, when an honest client make an update, he sees the result instantly ; way before it has been pushed to the server and send to the other clients. This is latency compensation, one of the seven principles of meteor.
If you modify a data via Meteor.call, you will do that :
send an update to the server
the server checks and update the base
the server send the update to the clients (including you)
your local base updates and your view update => you see your update
=> this is what you had in yesterday app ; meteor allow you to build a today app. Don't apply the old recipes :)
The accepted answer has the HOW right, but the WHERE is outdated information. (Yes, this would be better as a comment on the answer, but I can't do that yet.)
From the Meteor 1.2 documentation:
The best way to store your custom data onto the Meteor.users collection is to add a new uniquely-named top-level field on the user document.
And regarding using Meteor.user.profile to store custom information:
🔗Don’t use profile
There’s a tempting existing field called profile that is added by
default when a new user registers. This field was historically
intended to be used as a scratch pad for user-specific data - maybe
their image avatar, name, intro text, etc. Because of this, the
profile field on every user is automatically writeable by that user
from the client. It’s also automatically published to the client for
that particular user.
Basically, it's probably fine to store basic information such as name, address, dob, etc in the profile field, but not a good idea to store anything beyond that as it will, by default, be writeable by the client and vulnerable to malicious users.
I had the same problem and managed to do it only with Accounts.createUser:
Accounts.createUser({
email: email,
password: password,
profile: {
givenName: 'John',
surname: 'Doe',
gender: 'M'
}
}
Thats very simple way and it works. Just add your desired variables in the profile section and it should be ready. Hope it helps someone.
I ended up using https://atmospherejs.com/joshowens/accounts-entry which offers an extraSignUpFields config option.
From the documentation (https://github.com/ianmartorell/meteor-accounts-ui-bootstrap-3/blob/master/README.md):
Custom signup options
You can define additional input fields to appear in the signup form, and you can decide wether to save these values to the profile object of the user document or not. Specify an array of fields using Accounts.ui.config like so:
Accounts.ui.config({
requestPermissions: {},
extraSignupFields: [{
fieldName: 'first-name',
fieldLabel: 'First name',
inputType: 'text',
visible: true,
validate: function(value, errorFunction) {
if (!value) {
errorFunction("Please write your first name");
return false;
} else {
return true;
}
}
}, {
fieldName: 'last-name',
fieldLabel: 'Last name',
inputType: 'text',
visible: true,
}, {
fieldName: 'gender',
showFieldLabel: false, // If true, fieldLabel will be shown before radio group
fieldLabel: 'Gender',
inputType: 'radio',
radioLayout: 'vertical', // It can be 'inline' or 'vertical'
data: [{ // Array of radio options, all properties are required
id: 1, // id suffix of the radio element
label: 'Male', // label for the radio element
value: 'm' // value of the radio element, this will be saved.
}, {
id: 2,
label: 'Female',
value: 'f',
checked: 'checked'
}],
visible: true
}, {
fieldName: 'country',
fieldLabel: 'Country',
inputType: 'select',
showFieldLabel: true,
empty: 'Please select your country of residence',
data: [{
id: 1,
label: 'United States',
value: 'us'
}, {
id: 2,
label: 'Spain',
value: 'es',
}],
visible: true
}, {
fieldName: 'terms',
fieldLabel: 'I accept the terms and conditions',
inputType: 'checkbox',
visible: true,
saveToProfile: false,
validate: function(value, errorFunction) {
if (value) {
return true;
} else {
errorFunction('You must accept the terms and conditions.');
return false;
}
}
}]
});
The official Meteor Guide provides a comprehensive answer with an example code:
The best way to store your custom data onto the Meteor.users collection is to add a new uniquely-named top-level field on the user document.
https://guide.meteor.com/accounts.html#custom-user-data