I see the same non smtp requests in my maillog and I can't figure out, what is missing so those requests can be blocked.
fail2ban is setup and I have configured a jail.local file. How can I block those 6 requests:
Feb 10 10:58:57 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /system_api.php HTTP/1.1
Feb 10 10:58:57 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /c/version.js HTTP/1.1
Feb 10 10:58:58 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /streaming/clients_live.php HTTP/1.1
Feb 10 10:58:58 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /stalker_portal/c/version.js HTTP/1.1
Feb 10 10:58:58 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /client_area/ HTTP/1.1
Feb 10 10:58:59 host postfix/submission/smtpd[5167]: warning: non-SMTP command from unknown[161.35.7.72]: GET /stalker_portal/c/ HTTP/1.1
The jail.local for postfix config is shown below:
[postfix]
# To use another modes set filter parameter "mode" in jail.local:
enabeld = true
mode = more
port = smtp,ssmtp,submission
logpath = %(postfix_log)s
backend = %(postfix_backend)s
bantime = 1000
findtime = 10000
maxretry = 3
maxmatches = %(maxretry)s
bantime.increment = true
bantime.rndtime = 1000
bantime.factor = 1
bantime.formula = ban.Time * (1<<(ban.Count if ban.Count<20 else 20)) * banFactor
bantime.multipliers = 1 2 4 8 16 32 64
bantime.overalljails = true
It looks like the postfix default rules are not blocking / banning those non-smtp requests. What am I missing here?
the missing part was the "HOST" in the filter, so fail2ban knows which IP to ban.
the filter now looks like this:
[Definition]
failregex = ^.*\[<HOST>].*system\_api
^.*\[<HOST>].*c\/version\.js
^.*\[<HOST>].*streaming\/clients
^.*\[<HOST>].*client\_area
^.*\[<HOST>].*stalker\_portal
datepattern = ^[^\[]*\[({DATE})
{^LN-BEG}
ignoreregex =
Why don't try to define fail regex in your conf file for instance :
failregex = 161.35.7.72.*GET.*/system_api.php.*
For more output check this
I know this is a bit old and hopefully resolved, but your rule is not enabled. (You've misspelled it as "enabeld".)
Related
I'm working on a project that requires accessing webpages and I do this via
rvest::html_session(). For documentation and training I would like to set
up a reproducible example and have considered the following.
Use servr::httd(system.file("egwebsite", package = "<pkgname>"), daemon =
TRUE, browser = FALSE) to set up a simple HTTP server
Use rvest::html_session("http://127.0.0.1:4321") to set up the html
session.
However, the following simple example behaves differently on Linux (Debian 9)
and Windows 10. (I do not have easy access to OSx and have not tested on
that OS).
# On Windows
servr::httd(daemon = TRUE, browser = FALSE, port = 4321)
## Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321
## To stop the server, run servr::daemon_stop("94019719908480") or restart your R session
R.utils::withTimeout(
{
s <- rvest::html_session("http://127.0.0.1:4321")
},
timeout = 3,
onTimeout = "error")
s
## <session> http://127.0.0.1:4321/
## Status: 200
## Type: text/html
## Size: 2352
servr::daemon_stop()
However, on my Linux box (Debian 9) I get the following
servr::httd(daemon = TRUE, browser = FALSE, port = 4321)
## Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321
## To stop the server, run servr::daemon_stop("94019719908480") or restart your R session
R.utils::withTimeout(
{
s <- rvest::html_session("http://127.0.0.1:4321")
},
timeout = 3,
onTimeout = "error")
## Error: reached elapsed time limit
## Error in curl::curl_fetch_memory(url, handle = handle) :
## Operation was aborted by an application callback
That is, I am unable to create a html_session in the same R interactive
session that spawned the http server. If, however, I start a second R
session while the leaving the initial session running, I am able to create
the html_session without error.
What can I do so that I can create an html_session based on a servr::httd
HTTP server within the same R session on Linux?
Edit 1
If I add httr::verbose() to the html_session call I get the following when the session is created successfully. When the process hangs and fails to create the session the output stops on the last -> and none of the lines with <- are shown.
> s <- html_session("http://127.0.0.1:4321", httr::verbose())
-> GET / HTTP/1.1
-> Host: 127.0.0.1:4321
-> User-Agent: libcurl/7.52.1 r-curl/3.1 httr/1.3.1
-> Accept-Encoding: gzip, deflate
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 200 OK
<- Content-Type: text/html
<- Content-Length: 61303
<-
I have found a solution to my problem, run servr::httd in a subprocess. This solution requires the subprocess package.
First, a helper function R_binary will return the file path for the R binary on Windows or unix based OS.
R_binary <- function () {
R_exe <- ifelse (tolower(.Platform$OS.type) == "windows", "R.exe", "R")
return(file.path(R.home("bin"), R_exe))
}
Next, start R vanilla as a subprocess.
subR <- subprocess::spawn_process(R_binary(), c("--vanilla"))
Then start the HTTP server in the subprocess
subprocess::process_write(subR, 'servr::httd(".", browser = FALSE, port = 4321)\n')
## [1] 47
subprocess::process_read(subR)$stderr
## [1] "Serving the directory /home/dewittpe/so/my-servr-question at http://127.0.0.1:4321"
A quick test to show that there is communication between the active R session and the HTTP server:
session <- rvest::html_session("http://127.0.0.1:4321")
session
## <session> http://127.0.0.1:4321/
## Status: 200
## Type: text/html
## Size: 1054
And finally, kill the subprocess
subprocess::process_kill(subR)
I am stuck for some time trying to set up an email server. I've been mostly successful in my setup, but I am currently stuck at the following impasse: every time I send an email (regardless of queue size), my message will get stuck in the active queue for ~ 5 minutes, after which it will always be sent and reach it's final destination.
postqueue -p
(shows only one mail in the queue)
postqueue -f
... has no effect
I am running postfix 3.1.0, and dovecot 2.2.22 on Ubuntu 16.04. Also, I used iRedMail scripts for the installation. I can see from /var/log/mail.log that the mail is being stored in a queue by postfix, and I'm assuming there's another setting to set the time at which it will be processed. Any help would be most appreciated.
/var/log/mail.log
Jan 9 14:35:28 mail postfix[26070]: name_mask: all
Jan 9 14:35:28 mail postfix[26070]: inet_addr_local: configured 2 IPv4 addresses
Jan 9 14:35:28 mail postfix[26070]: inet_addr_local: configured 3 IPv6 addresses
Jan 9 14:35:28 mail postfix/postfix-script[26075]: error: unknown command: ''
Jan 9 14:35:28 mail postfix/postfix-script[26076]: fatal: usage: postfix start (or stop, reload, abort, flush, check, status, set-permissions, upgrade-configuration)
Jan 9 14:35:49 mail postfix[26085]: fatal: usage: postfix [-c config_dir] [-Dv] command
Jan 9 14:35:58 mail postfix[26090]: fatal: usage: postfix [-c config_dir] [-Dv] command
Jan 9 14:40:17 mail postfix/submission/smtpd[26282]: connect from mail.example.com[127.0.0.1]
Jan 9 14:40:17 mail postfix/submission/smtpd[26282]: Anonymous TLS connection established from mail.example.com[127.0.0.1]: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
Jan 9 14:40:17 mail postfix/submission/smtpd[26282]: 87ABB42362: client=mail.example.com[127.0.0.1], sasl_method=PLAIN, sasl_username=jamal#example.com
Jan 9 14:40:17 mail postfix/cleanup[26288]: 87ABB42362: message-id=<0130a1e7b419fdf0f31ce406618aef97#example.com>
Jan 9 14:40:17 mail postfix/qmgr[28474]: 87ABB42362: from=, size=1845, nrcpt=1 (queue active)
Jan 9 14:40:17 mail postfix/submission/smtpd[26282]: disconnect from mail.example.com[127.0.0.1] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Jan 9 14:45:05 mail amavis[10642]: (10642-08) (!)ClamAV-clamd av-scanner FAILED: run_av error: ask_daemon_internal: Exceeded allowed time at (eval 104) line 611.\n
Jan 9 14:45:05 mail amavis[10642]: (10642-08) (!)WARN: all primary virus scanners failed, considering backups
Jan 9 14:45:05 mail amavis[10642]: (10642-08) (!!)AV: ALL VIRUS SCANNERS FAILED
Jan 9 14:45:07 mail postfix/postqueue[26446]: fatal: usage: postqueue -f | postqueue -i queueid | postqueue -j | postqueue -p | postqueue -s site
Jan 9 14:45:08 mail postfix/10025/smtpd[26452]: connect from mail.example.com[127.0.0.1]
Jan 9 14:45:08 mail postfix/10025/smtpd[26452]: 60ADA423BA: client=mail.example.com[127.0.0.1]
Jan 9 14:45:08 mail postfix/cleanup[26454]: 60ADA423BA: message-id=<0130a1e7b419fdf0f31ce406618aef97#example.com>
Jan 9 14:45:08 mail postfix/10025/smtpd[26452]: disconnect from mail.example.com[127.0.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5
Jan 9 14:45:08 mail postfix/qmgr[28474]: 60ADA423BA: from=, size=2920, nrcpt=1 (queue active)
Jan 9 14:45:08 mail amavis[10642]: (10642-08) Passed UNCHECKED {RelayedInternal}, ORIGINATING/MYNETS LOCAL [127.0.0.1]:33748 -> , Queue-ID: 87ABB42362, Message-ID: <0130a1e7b419fdf0f31ce406618aef97#example.com>, mail_id: FFIMhQZ3j_4G, Hits: 0.204, size: 1845, queued_as: 60ADA423BA, dkim_new=dkim:example.com, 290782 ms, Tests: [ALL_TRUSTED=-1,HTML_MESSAGE=0.001,TVD_RCVD_SINGLE=1.213,T_RP_MATCHES_RCVD=-0.01]
Jan 9 14:45:08 mail postfix/amavis/smtp[26293]: 87ABB42362: to=, relay=127.0.0.1[127.0.0.1]:10026, delay=291, delays=0.16/0.03/0.04/291, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 60ADA423BA)
Jan 9 14:45:08 mail postfix/qmgr[28474]: 87ABB42362: removed
Jan 9 14:45:09 mail postfix/smtp[26455]: Untrusted TLS connection established to cornellprod-mail-onmicrosoft-com.mail.eo.outlook.com[216.32.181.106]:25: TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)
Jan 9 14:45:09 mail postfix/smtp[26455]: 60ADA423BA: to=, relay=cornellprod-mail-onmicrosoft-com.mail.eo.outlook.com[216.32.181.106]:25, delay=1.6, delays=0.02/0.03/0.64/0.89, dsn=2.6.0, status=sent (250 2.6.0 <0130a1e7b419fdf0f31ce406618aef97#example.com> [InternalId=132499741090511, Hostname=BN3PR0401MB1284.namprd04.prod.outlook.com] 10597 bytes in 0.347, 29.819 KB/sec Queued mail for delivery)
Jan 9 14:45:09 mail postfix/qmgr[28474]: 60ADA423BA: removed
Jan 9 14:45:20 mail postfix/postqueue[26468]: fatal: Flush service is not configured for destination "active"
/etc/postfix/main.cf
# --------------------
# INSTALL-TIME CONFIGURATION INFORMATION
#
# location of the Postfix queue. Default is /var/spool/postfix. queue_directory = /var/spool/postfix
# location of all postXXX commands. Default is /usr/sbin. command_directory = /usr/sbin
# location of all Postfix daemon programs (i.e. programs listed in the
# master.cf file). This directory must be owned by root.
# Default is /usr/libexec/postfix daemon_directory = /usr/lib/postfix/sbin
# location of Postfix-writable data files (caches, random numbers).
# This directory must be owned by the mail_owner account (see below).
# Default is /var/lib/postfix. data_directory = /var/lib/postfix
# owner of the Postfix queue and of most Postfix daemon processes.
# Specify the name of a user account THAT DOES NOT SHARE ITS USER OR GROUP ID
# WITH OTHER ACCOUNTS AND THAT OWNS NO OTHER FILES OR PROCESSES ON THE SYSTEM.
# In particular, don't specify nobody or daemon. PLEASE USE A DEDICATED USER.
# Default is postfix. mail_owner = postfix
# The following parameters are used when installing a new Postfix version.
#
# sendmail_path: The full pathname of the Postfix sendmail command.
# This is the Sendmail-compatible mail posting interface.
# sendmail_path = /usr/sbin/sendmail
# newaliases_path: The full pathname of the Postfix newaliases command.
# This is the Sendmail-compatible command to build alias databases.
# newaliases_path = /usr/bin/newaliases
# full pathname of the Postfix mailq command. This is the Sendmail-compatible
# mail queue listing command. mailq_path = /usr/bin/mailq
# group for mail submission and queue management commands.
# This must be a group name with a numerical group ID that is not shared with
# other accounts, not even with the Postfix account. setgid_group = postdrop
# debugger_command =
PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
ddd $daemon_directory/$process_name $process_id & sleep 5
debug_peer_level = 2
# --------------------
# CUSTOM SETTINGS
#
# SMTP server response code when recipient or domain not found. unknown_local_recipient_reject_code = 550
# Do not notify local user. biff = no
# Disable the rewriting of "site!user" into "user#site". swap_bangpath = no
# Disable the rewriting of the form "user%domain" to "user#domain". allow_percent_hack = no
# Allow recipient address start with '-'. allow_min_user = no
# Disable the SMTP VRFY command. This stops some techniques used to
# harvest email addresses. disable_vrfy_command = yes
# Enable both IPv4 and/or IPv6: ipv4, ipv6, all. inet_protocols = all
# Enable all network interfaces. inet_interfaces = all
#
# TLS settings.
#
# SSL key, certificate, CA
# smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/cert.pem
smtpd_tls_CAfile = /etc/letsencrypt/live/mail.example.com/chain.pem
#
# Disable SSLv2, SSLv3
# smtpd_tls_protocols = !SSLv2 !SSLv3 smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3 smtp_tls_protocols = !SSLv2 !SSLv3
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3 lmtp_tls_protocols =
!SSLv2 !SSLv3 lmtp_tls_mandatory_protocols = !SSLv2 !SSLv3
#
# Fix 'The Logjam Attack'.
# smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CDC3-SHA, KRB5-DE5,
CBC3-SHA smtpd_tls_dh512_param_file = /etc/ssl/dh512_param.pem
smtpd_tls_dh1024_param_file = /etc/ssl/dh2048_param.pem
tls_random_source = dev:/dev/urandom
smtpd_tls_loglevel = 1
# Opportunistic TLS: announce STARTTLS support to remote SMTP clients, but do
# not require that clients use TLS encryption. smtpd_tls_security_level = may
# Produce Received: message headers that include information about the
# protocol and cipher used, as well as the remote SMTP client CommonName and
# client certificate issuer CommonName.
# This is disabled by default, as the information may be modified in transit
# through other mail servers. Only information that was recorded by the final
# destination can be trusted.
#smtpd_tls_received_header = yes
# Opportunistic TLS, used when Postfix sends email to remote SMTP server.
# Use TLS if this is supported by the remote SMTP server, otherwise use
# plaintext.
# References:
# - http://www.postfix.org/TLS_README.html#client_tls_may
# - http://www.postfix.org/postconf.5.html#smtp_tls_security_level smtp_tls_security_level = may
# Use the same CA file as smtpd. smtp_tls_CAfile = $smtpd_tls_CAfile smtp_tls_note_starttls_offer = yes
# Enable long, non-repeating, queue IDs (queue file names).
# The benefit of non-repeating names is simpler logfile analysis and easier
# queue migration (there is no need to run "postsuper" to change queue file
# names that don't match their message file inode number).
#enable_long_queue_ids = yes
# Reject unlisted sender and recipient smtpd_reject_unlisted_recipient = yes smtpd_reject_unlisted_sender = yes
# Header and body checks with PCRE table header_checks = pcre:/etc/postfix/header_checks body_checks =
pcre:/etc/postfix/body_checks.pcre
# A mechanism to transform commands from remote SMTP clients.
# This is a last-resort tool to work around client commands that break
# interoperability with the Postfix SMTP server. Other uses involve fault
# injection to test Postfix's handling of invalid commands.
# Requires Postfix-2.7+.
#smtpd_command_filter = pcre:/etc/postfix/command_filter.pcre
# HELO restriction smtpd_helo_required = yes smtpd_helo_restrictions =
permit_mynetworks
permit_sasl_authenticated
check_helo_access pcre:/etc/postfix/helo_access.pcre
reject_non_fqdn_helo_hostname
reject_unknown_helo_hostname
# Sender restrictions smtpd_sender_restrictions =
reject_unknown_sender_domain
reject_non_fqdn_sender
reject_unlisted_sender
permit_mynetworks
permit_sasl_authenticated
check_sender_access pcre:/etc/postfix/sender_access.pcre
# Recipient restrictions smtpd_recipient_restrictions =
reject_unknown_recipient_domain
reject_non_fqdn_recipient
reject_unlisted_recipient
check_policy_service inet:127.0.0.1:7777
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
# END-OF-MESSAGE restrictions smtpd_end_of_data_restrictions =
check_policy_service inet:127.0.0.1:7777
# Data restrictions smtpd_data_restrictions = reject_unauth_pipelining
proxy_read_maps = $canonical_maps $lmtp_generic_maps
$local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps
$recipient_canonical_maps $relay_domains $relay_recipient_maps
$relocated_maps $sender_bcc_maps $sender_canonical_maps
$smtp_generic_maps $smtpd_sender_login_maps $transport_maps
$virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains
$virtual_mailbox_maps $smtpd_sender_restrictions
$sender_dependent_relayhost_maps
# Avoid duplicate recipient messages. Default is 'yes'. enable_original_recipient = no
# Virtual support. virtual_minimum_uid = 2000 virtual_uid_maps = static:2000 virtual_gid_maps = static:2000 virtual_mailbox_base =
/var/vmail
# Do not set virtual_alias_domains. virtual_alias_domains =
#
# Enable SASL authentication on port 25 and force TLS-encrypted SASL authentication.
# WARNING: NOT RECOMMENDED to enable smtp auth on port 25, all end users should
# be forced to submit email through port 587 instead.
#
#smtpd_sasl_auth_enable = yes
#smtpd_sasl_security_options = noanonymous
#smtpd_tls_auth_only = yes
# hostname myhostname = mail.example.com myorigin = mail.example.com mydomain = mail.example.com
# trusted SMTP clients which are allowed to relay mail through Postfix.
#
# Note: additional IP addresses/networks listed in mynetworks should be listed
# in iRedAPD setting 'MYNETWORKS' (in /opt/iredapd/settings.py) too.
# for example:
#
# MYNETWORKS = ['xx.xx.xx.xx', 'xx.xx.xx.0/24', ...]
# mynetworks = 127.0.0.1 [::1]
# Accepted local emails mydestination = $myhostname, localhost, localhost.localdomain
alias_maps = hash:/etc/postfix/aliases alias_database =
hash:/etc/postfix/aliases
# Default message_size_limit. message_size_limit = 15728640
# The set of characters that can separate a user name from its extension
# (example: user+foo), or a .forward file name from its extension (example:
# .forward+foo).
# Postfix 2.11 and later supports multiple characters. recipient_delimiter = +
# The time after which the sender receives a copy of the message headers of
# mail that is still queued. Default setting is disabled (0h) by Postfix.
#delay_warning_time = 1h compatibility_level = 2
#
# Lookup virtual mail accounts
# transport_maps =
proxy:mysql:/etc/postfix/mysql/transport_maps_user.cf
proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf
sender_dependent_relayhost_maps =
proxy:mysql:/etc/postfix/mysql/sender_dependent_relayhost_maps.cf
# Lookup table with the SASL login names that own the sender (MAIL FROM) addresses. smtpd_sender_login_maps =
proxy:mysql:/etc/postfix/mysql/sender_login_maps.cf
virtual_mailbox_domains =
proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf
relay_domains =
$mydestination
proxy:mysql:/etc/postfix/mysql/relay_domains.cf
virtual_mailbox_maps =
proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_alias_maps =
proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf
proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf
proxy:mysql:/etc/postfix/mysql/catchall_maps.cf
proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf
sender_bcc_maps =
proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_user.cf
proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_domain.cf
recipient_bcc_maps =
proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_user.cf
proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_domain.cf
#
# Postscreen
# postscreen_greet_action = enforce postscreen_blacklist_action = enforce postscreen_dnsbl_action = enforce postscreen_dnsbl_threshold =
2 postscreen_dnsbl_sites =
zen.spamhaus.org=127.0.0.[2..11]*3
b.barracudacentral.org=127.0.0.[2..11]*2
postscreen_dnsbl_reply_map =
texthash:/etc/postfix/postscreen_dnsbl_reply postscreen_access_list =
permit_mynetworks cidr:/etc/postfix/postscreen_access.cidr
# Require Postfix-2.11+ postscreen_dnsbl_whitelist_threshold = -2
#
# Dovecot SASL support.
# smtpd_sasl_type = dovecot smtpd_sasl_path = private/dovecot-auth virtual_transport = dovecot dovecot_destination_recipient_limit = 1
#
# Amavisd + SpamAssassin + ClamAV
# content_filter = smtp-amavis:[127.0.0.1]:10024
# Concurrency per recipient limit. smtp-amavis_destination_recipient_limit = 1
/etc/dovecot/dovecot.conf
# More details about Dovecot settings: # -
http://wiki2.dovecot.org/ # - http://wiki2.dovecot.org/Variables
# Listen addresses. # - '*' means all available IPv4 addresses. #
- '[::]' means all available IPv6 addresses. # Listen on all available addresses by default listen = * [::]
#base_dir = /var/run/dovecot mail_plugins = quota mailbox_alias acl
mail_log notify
# Enabled mail protocols. protocols = pop3 imap sieve lmtp
# User/group who owns the message files: mail_uid = 2000 mail_gid =
2000
# Assign uid to virtual users. first_valid_uid = 2000 last_valid_uid
= 2000
# Logging. Reference: http://wiki2.dovecot.org/Logging # # Use
syslog syslog_facility = local5 # Log file path if we use internal
log system #log_path = /var/log/dovecot/dovecot.log
# Debug #mail_debug = yes #auth_verbose = yes #auth_debug = yes
#auth_debug_passwords = yes # Possible values: no, plain, sha1.
#auth_verbose_passwords = no
# SSL: Global settings. # Refer to wiki site for per protocol, ip,
server name SSL settings: #
http://wiki2.dovecot.org/SSL/DovecotConfiguration ssl_protocols =
!SSLv2 !SSLv3 ssl = required verbose_ssl = no #ssl_ca =
# Fix 'The Logjam Attack' ssl_cipher_list =
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5
# Dovecot 2.2.6 or greater: # Specify the wanted DH parameters
length ssl_dh_parameters_length = 2048 ssl_prefer_server_ciphers = yes
# With disable_plaintext_auth=yes AND ssl=required, STARTTLS is
mandatory. # Set disable_plaintext_auth=no AND ssl=yes to allow plain
password transmitted # insecurely. disable_plaintext_auth = yes
# Allow plain text password per IP address/net #remote
192.168.0.0/24 { # disable_plaintext_auth = no #}
# Mail location and mailbox format. mail_location =
maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/
# Authentication related settings. # Append this domain name if
client gives empty realm. #auth_default_realm = example.com
# Authentication mechanisms. auth_mechanisms = PLAIN LOGIN
# Limits the number of users that can be logging in at the same time.
# Default is 100. This can be overridden by process_limit = in #
service [protocol] block. # e.g. # protocol imap-login { #
... # process_limit = 500 # }
#default_process_limit = 100
service auth {
unix_listener /var/spool/postfix/private/dovecot-auth {
user = postfix
group = postfix
mode = 0666
}
unix_listener auth-master {
user = vmail
group = vmail
mode = 0666
}
unix_listener auth-userdb {
user = vmail
group = vmail
mode = 0660
} }
# LMTP server (Local Mail Transfer Protocol). # Reference:
http://wiki2.dovecot.org/LMTP service lmtp {
user = vmail
\# For higher volume sites, it may be desirable to increase the number of
\# active listener processes. A range of 5 to 20 is probably good for most
\# sites.
process_min_avail = 5
\# Logging.
\# Require 'log_path =' in 'protocol lmtp {}' block.
executable = lmtp -L
\# Listening on socket file and TCP
unix_listener /var/spool/postfix/private/dovecot-lmtp {
user = postfix
group = postfix
mode = 0600
}
inet_listener lmtp {
\# Listen on localhost (ipv4)
address = 127.0.0.1
port = 24
} }
# Virtual mail accounts. userdb {
args = /etc/dovecot/dovecot-mysql.conf
driver = sql } passdb {
args = /etc/dovecot/dovecot-mysql.conf
driver = sql }
auth_master_user_separator = * passdb {
driver = passwd-file
args = /etc/dovecot/dovecot-master-users
master = yes }
plugin {
# Quota configuration.
# Reference: http://wiki2.dovecot.org/Quota/Configuration
quota = dict:user::proxy::quotadict
quota_rule = *:storage=1G
#quota_rule2 = *:messages=0
#quota_rule3 = Trash:storage=1G
#quota_rule4 = Junk:ignore
\# Quota warning.
\#
\# If user suddenly receives a huge mail and the quota jumps from
\# 85% to 95%, only the 95% script is executed.
\#
\# Only the command for the first exceeded limit is executed, so configure
\# the highest limit first.
quota_warning = storage=100%% quota-warning 100 %u
quota_warning2 = storage=95%% quota-warning 95 %u
quota_warning3 = storage=90%% quota-warning 90 %u
quota_warning4 = storage=85%% quota-warning 85 %u
\# allow user to become max 10% (or 50 MB) over quota
quota_grace = 10%%
\#quota_grace = 50 M
\# Custom Quota Exceeded Message.
\# You can specify the message directly or read the message from a file.
\#quota_exceeded_message = Quota exceeded, please try again later.
\#quota_exceeded_message = </path/to/quota_exceeded_message.txt
\# Plugin: expire.
\#expire = Trash 7 Trash/* 7 Junk 30
\#expire_dict = proxy::expire
\# ACL and share folder
acl = vfile
acl_shared_dict = proxy::acl
\# By default Dovecot doesn't allow using the IMAP "anyone" or
\# "authenticated" identifier, because it would be an easy way to spam
\# other users in the system. If you wish to allow it,
\#acl_anyone = allow
\# Pigeonhole managesieve service.
\# Reference: http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
\# Per-user sieve settings.
sieve_dir = %Lh/sieve
sieve = %Lh/sieve/dovecot.sieve
\# Global sieve settings.
sieve_global_dir = /var/vmail/sieve
\# Note: if user has personal sieve script, global sieve rules defined in
\# sieve_default will be ignored. Please use sieve_before or
\# sieve_after instead.
\#sieve_default =
sieve_before = /var/vmail/sieve/dovecot.sieve
\#sieve_after =
\# The maximum number of redirect actions that can be performed during a
\# single script execution.
\# The meaning of 0 differs based on your version. For pigeonhole-0.3.0 and
\# beyond this means that redirect is prohibited. For older versions,
\# however, this means that the number of redirects is unlimited.
sieve_max_redirects = 30
\# Reference: http://wiki2.dovecot.org/Plugins/MailboxAlias
mailbox_alias_old = Sent
mailbox_alias_new = Sent Messages
mailbox_alias_old2 = Sent
mailbox_alias_new2 = Sent Items
\# Events to log. `autoexpunge` is included in `expunge`
\# Defined in https://github.com/dovecot/core/blob/master/src/plugins/mail-log/mail-log-plugin.c
mail_log_events = delete undelete expunge mailbox_delete mailbox_rename
mail_log_fields = uid box msgid size from subject }
service quota-warning {
executable = script /usr/local/bin/dovecot-quota-warning.sh
unix_listener quota-warning {
user = vmail
group = vmail
mode = 0660
} }
service dict {
unix_listener dict {
mode = 0660
user = vmail
group = vmail
} }
dict {
#expire = db:/var/lib/dovecot/expire/expire.db
quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf
acl = mysql:/etc/dovecot/dovecot-share-folder.conf }
protocol lda {
# Reference: http://wiki2.dovecot.org/LDA
mail_plugins = $mail_plugins sieve
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
postmaster_address = root
\# Log file path if we use internal log system
\#log_path = /var/log/dovecot/sieve.log }
protocol lmtp {
# Log file path if we use internal log system
#log_path = /var/log/dovecot/lmtp.log
\# Plugins
mail_plugins = quota sieve
postmaster_address = postmaster
\# Address extension delivery
lmtp_save_to_detail_mailbox = yes
recipient_delimiter = + }
protocol imap {
mail_plugins = $mail_plugins imap_quota imap_acl
imap_client_workarounds = tb-extra-mailbox-sep
\# Maximum number of IMAP connections allowed for a user from each IP address.
\# NOTE: The username is compared case-sensitively.
\# Default is 10.
\# Increase it to avoid issue like below:
\# "Maximum number of concurrent IMAP connections exceeded"
mail_max_userip_connections = 30 }
protocol pop3 {
mail_plugins = $mail_plugins
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_uidl_format = %08Xu%08Xv
\# Maximum number of IMAP connections allowed for a user from each IP address.
\# NOTE: The username is compared case-sensitively.
\# Default is 10.
mail_max_userip_connections = 30
\# POP3 logout format string:
\# %i - total number of bytes read from client
\# %o - total number of bytes sent to client
\# %t - number of TOP commands
\# %p - number of bytes sent to client as a result of TOP command
\# %r - number of RETR commands
\# %b - number of bytes sent to client as a result of RETR command
\# %d - number of deleted messages
\# %m - number of messages (before deletion)
\# %s - mailbox size in bytes (before deletion)
\# Default format doesn't have 'in=%i, out=%o'.
\#pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, in=%i, out=%o }
# Login processes. Refer to Dovecot wiki for more details: #
http://wiki2.dovecot.org/LoginProcess service imap-login {
#inet_listener imap {
# port = 143
#}
#inet_listener imaps {
# port = 993
# ssl = yes
#}
service_count = 1
\# To avoid startup latency for new client connections, set process_min_avail
\# to higher than zero. That many idling processes are always kept around
\# waiting for new connections.
\#process_min_avail = 0
\# number of simultaneous IMAP connections
process_limit = 500
\# vsz_limit should be fine at its default 64MB value
\#vsz_limit = 64M }
service pop3-login {
#inet_listener pop3 {
# port = 110
#}
#inet_listener pop3s {
# port = 995
# ssl = yes
#}
service_count = 1
\# number of simultaneous POP3 connections
\#process_limit = 500 }
service managesieve-login {
inet_listener sieve {
# Listen on localhost (ipv4)
address = 127.0.0.1
port = 4190
} }
namespace {
type = private
separator = /
prefix =
inbox = yes
\# Refer to document for more details about alias mailbox:
\# http://wiki2.dovecot.org/MailboxSettings
\#
\# Sent
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
auto = no
special_use = \Sent
}
mailbox "Sent Items" {
auto = no
special_use = \Sent
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
\# Trash
mailbox Trash {
auto = subscribe
special_use = \Trash
}
mailbox "Deleted Messages" {
auto = no
special_use = \Trash
}
\# Junk
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Spam {
auto = no
special_use = \Junk
}
mailbox "Junk E-mail" {
auto = no
special_use = \Junk
}
\# Archive
mailbox Archive {
auto = no
special_use = \Archive
}
mailbox Archives {
auto = no
special_use = \Archive
} }
namespace {
type = shared
separator = /
prefix = Shared/%%u/
location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%Ld/%%Ln
\# this namespace should handle its own subscriptions or not.
subscriptions = yes
list = children }
# Public mailboxes. # Refer to Dovecot wiki page for more details:
# http://wiki2.dovecot.org/SharedMailboxes/Public #namespace { #
type = public # separator = / # prefix = Public/ #
location =
maildir:/var/vmail/public:CONTROL=%Lh/Maildir/public:INDEXPVT=%Lh/Maildir/public
# # # Allow users to subscribe to the public folders. #
subscriptions = yes #}
It looks like the ClamAV hangs and times out after 5 minutes. Would that explain things? I find it hard to read your configurations.
I seem to be having an issue with our riak-kv db v2.2.3, we are currently running 5 node cluster, out of nowhere last week suddenly requests to our db started returning 500 internal server error for all/almost every request (also might be valuable to mention our db has nearly doubled in size over the past 2 weeks.) At first, we thought the issue was in the code making the requests however after sshing into one of the nodes in the cluster and attempting a simple request to list buckets we saw this:
Command:
curl -i http://localhost:8098/buckets?buckets=true
Response:
HTTP/1.1 500 Internal Server Error
Vary: Accept-Encoding
Server: MochiWeb/1.1 WebMachine/1.10.9 (cafe not found)
Date: Sun, 10 Dec 2017 06:21:20 GMT
Content-Type: text/html
Content-Length: 1193
<html><head><title>500 Internal Server Error</title></head><body>
<h1>Internal Server Error</h1>The server encountered an error while
processing this request:<br><pre>{error,
{error,
{badmatch,{error,mailbox_overload}},
[{riak_kv_wm_buckets,produce_bucket_list,2,
[{file,"src/riak_kv_wm_buckets.erl"},{line,225}]},
{webmachine_resource,resource_call,3,
[{file,"src/webmachine_resource.erl"},{line,186}]},
{webmachine_resource,do,3,
[{file,"src/webmachine_resource.erl"},{line,142}]},
{webmachine_decision_core,resource_call,1,
[{file,"src/webmachine_decision_core.erl"},{line,48}]},
{webmachine_decision_core,decision,1,
[{file,"src/webmachine_decision_core.erl"},{line,562}]},
{webmachine_decision_core,handle_request,2,
[{file,"src/webmachine_decision_core.erl"},{line,33}]},
{webmachine_mochiweb,loop,2,
[{file,"src/webmachine_mochiweb.erl"},{line,72}]},
{mochiweb_http,headers,5,
[{file,"src/mochiweb_http.erl"},{line,105}]}]}}</pre><P><HR>
<ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>
After some more investigation into the issue, I pulled the logs on one of the riak nodes and saw this:
2017-12-10 03:54:58.654 [error]
<0.24342.271>#yz_solrq_helper:send_solr_ops_for_entries:301 Updating a
batch of Solr operations failed for index <<"attachment">> with error
{error,{other,{ok,"500",[{"Content-Type","application/json;
charset=UTF-8"},{"Transfer-Encoding","chunked"}],<<"
{\"responseHeader\":{\"status\":500,\"QTime\":1}, error": {
"msg ": "Exception writing document id 1*default*Attachment*07d8e24-dc32s-11e7-q9640-a7f8b4edb446*623 to the index; possible analysis error.",
"trace": ""org.apache.solr.common.So lrException: Exception writing document id 1*default*Attachment*07d8e24-dc32s-11e7-q9640-a7f8b4edb446*623 to the index; possible analysis error.
at org.apache.solr.update. DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:169)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at or g.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd (DistributedUpdateProcessor.java:952)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:692)
at org.apache.so lr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:141)
at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLo ader.java:106)
at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:68)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java :99)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequ est(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilte r.java:777)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFi lter.java:207)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(Servle tHandler.java:455)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHand ler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(Contex tHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHa ndler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedH andler.java:135)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at org.eclipse.jetty.server.handler.HandlerCo llection.handle(HandlerCollection.java:154)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.han dle(Server.java:368)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.BlockingHttpConnec tion.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at org.eclipse.jetty.s erver.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.ecli pse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclips e.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java :608)
at org.eclip..." >>
}
}
}
After some googling some people said this error is
Caused by: java.lang.OutOfMemoryError: Java heap space
So I modified my riak config from:
search.solr.jvm_options = -d64 -Xms4g -Xmx4g -XX:+UseStringCache -XX:+UseCompressedOops
To:
search.solr.jvm_options = -d64 -Xms8g -Xmx8g -XX:+UseStringCache -XX:+UseCompressedOops
However I am still getting the same error any help is much appreciated.
I'm trying to execute a store procedure on a SQL Server 2008.
After about 30 seconds it ends in
SQLSTATE[HY000]: General error: 20003 Adaptive Server connection timed
out [20003]
I've checked if on the SQL Server remote connections are allowed and they are with a timeout of 600 seconds (default).
This is my config.yml
doctrine:
dbal:
default_connection: default
connections:
default: ...
mssql:
driver_class: \Lsw\DoctrinePdoDblib\Doctrine\DBAL\Driver\PDODblib\Driver
host: mssql_freetds
port: "%stage_database_port%"
dbname: "%stage_database_name%"
user: "%stage_database_user%"
password: "%stage_database_password%"
charset: UTF8
options:
timeout: 600 // I don't know if it's correct but it doesn't work even without this
And this is my code
$connection = $this->getDoctrine()->getManager('mssql')
->getConnection();
$stmt = $connection->prepare("Exec SP_MyStoreProcedure ?, ?");
$stmt->bindValue(1, $sd->format("Y-m-d") /* This is a date */);
$stmt->bindValue(2, $ed->format("Y-m-d") /* This is a date */);
$stmt->execute();
[EDIT]
So, i've found that i should configure time out on freetds configuration because of LswDoctrinePdoDblib. I'v edited the /etc/freetds/freetds.conf file but it still ends up connection after 30 seconds
/etc/freetds/freetds.conf
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
; tds version = 4.2
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
timeout = 600
connect timeout = 600
[mssql_freetds]
host = 192.168.0.xx
port = 1433
timeout = 600
tds version = 8.0
client charset = UTF-8
text size = 20971520
[EDIT 2]
Nothing worked for timeout except this:
doctrine:
dbal:
default_connection: default
connections:
default: ...
mssql:
driver_class: \Lsw\DoctrinePdoDblib\Doctrine\DBAL\Driver\PDODblib\Driver
host: mssql_freetds
port: "%stage_database_port%"
dbname: "%stage_database_name%"
user: "%stage_database_user%"
password: "%stage_database_password%"
charset: UTF8
options:
2: 600 # 2 is the equivalent of \PDO::ATTR_TIMEOUT
But the store procedure should end in 17seconds, it waited 10 minutes ending with the same result. I don't know why. It seems to be a FREETDS/SQL SERVER bug or something like that...
[EDIT 3]
Found that the problem was a "NOT IN" clause in the SQL of the stored procedure. I had to replace it with a LEFT JOIN combined with a IS NULL clause in WHERE statement. There's also a "IN" clause but it works. I don't know if it's a FREETDS or a LswDoctrinePdoDblib issue.
I have installed an instance of openTSDB 2.0 for testing, but the server is rejecting all attempts to insert metrics.
These two are from the documentation:
$ telnet localhost 8020
Trying ::1...
Connected to localhost.
Escape character is '^]'.
put sys.cpu.nice 1346846400 50 hostname=test1
put: unknown metric: No such name for 'metrics': 'sys.cpu.nice'
put http.hits 1234567890 34877 host=A
put: unknown metric: No such name for 'metrics': 'http.hits'
My hbase server appears to be configured correctly:
hbase(main):012:0> list
TABLE
tsdb
tsdb-meta
tsdb-tree
tsdb-uid
4 row(s) in 0.0130 seconds
=> ["tsdb", "tsdb-meta", "tsdb-tree", "tsdb-uid"]
I have tried with the server set auto create metrics both ways:
tsd.core.auto_create_metrics = true
tsd.core.meta.enable_realtime_ts = true
tsd.core.meta.enable_realtime_uid = true
Any suggestions about what I have done wrong would be greatly appreciated.
Try rest api, it takes json as input on port 4242:
curl -i -H "Content-Type: application/json" -X POST -d '{"metric": "sys.cpu.nice", "timestamp": 1346846400,"value": 18, "tags": { "host": "web01"}}' http://localhost:4242/api/put/?details
If you want to do it with code:
import requests
import json
def SendTSDBMetrics(metrics):
response = requests.post(url=tsdburi, data=metrics,headers=headers)
print response.text # print what was inserted
metric = 'sys.cpu.nice'
metrics = []
metrics.append({'metric':metric, 'timestamp':time.now(), 'value':18, 'tags':{"device_id":"1"}})
metrics.append({'metric':metric, 'timestamp':time.now(), 'value':100, 'tags':{"device_id":"1"}})
SendTSDBMetrics(json.dumps(metrics))