I am running a FreeBSD server and I have been sent a warning that spam has been sent from my server. I do not have it set as an open relay and I have customized the sendmail configuration. I'd like to know who is sending what email along with their username, email subject line as well as a summary of how much mail they have been sending. I would like to run a report on a log similar to how it is done when processing Apache server logs.
What are my options?
One idea is to alias sendmail to be a custom script, which simply cats the sendmail arguments to the end of a log before calling sendmail in the usual manner.
You can also monitor all system calls to write and read functions by executing:
ps auxw | grep sendmail | awk '{print"-p " $2}' | xargs strace -s 256 -f 2>&1 | grep -E $'#|(([0-9]+\.){3}[0-9]+)' | tee -a "/var/log/sendmail-logs.log"
This will give you direct access to the information, you cannot go deeper I think.
Can you give some sample logs? I think you're best bet would be to look through them with either grep or cut to get the source/destinations that are being sent too. Also, you could write a Perl script to automate it once you have the correct regex. This would be the best option.
If FreeBSD have default config, you have only one way to handle output mail, check what sending through you sendmail system in /etc/mail.
All output mail must be logged by /var/log/maillog
Related
My goal is to be able to read new messages from a gmail account via a linux server. I guess I could do this via IMAP or something, but I'd like to avoid that complexity if possible given that gmail has this nice feed set up:
https://mail.google.com/mail/feed/atom/
The only issue is that I'm not sure how to authenticate the call to pull this. Is this possible?
A good starting point should be:
curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | sed -n "s/<title>\(.*\)<\/title.*name>\(.*\)<\/name>.*/\2 - \1/p"
Checks the Gmail ATOM feed for your account, parses it and outputs a list of unread messages.
Also, see this thread: http://www.commandlinefu.com/commands/view/3380/check-your-unread-gmail-from-the-command-line
OTOH, I would recommend using mutt and IMAP.
This is my first question on stackoverflow!
I want to have a unix script that will run grep on the console output. Here is what my script does:
1. Telnet into a remote server (I have done this part successfully)
2. On successful login, the remote server displays outputs information on the console. I need to run grep on that console output (need help with this)
So, I need a script to run grep on the output appearing on the console.
Any thoughts??
Thanks,
Puneet
Use SSH instead. It's more secure and far easier to script.
ssh remoteusername#remotehost:/path/to/remote/script | grep 'something'
with appropriate key setup, it won't even prompt you for a password.
Have you tried I/O redirection? You could either do
$ your-command > output.txt
and then run grep on that file, or just directly pipe the output through grep like so
$ your-command | grep ...
See this article or google around for similar. There are probably thousands of good articles about this around the web.
Instead of telnet, I would suggest using netcat (nc). You could then pass your login credentials via standard input and grep the standard output (nc prints anything sent by the server on standard output).
nc <host> <port> <auth.txt | grep 'string'
What you want to do is probably using a pipe. You can probably see it in the above answers it's the | sign you see in the command. It may be difficult to locate on your keyboard, depending on the layout. (I have to admit it is not very often used).
Pipes will redirect the output of one command. Instead of sending it to the console, they will send it as an input of another command.
cmd1 | grep foo is equivalent to running grep foo on the output of cmd1 (you can replace cmd1 by your netstat command).
One last thing is that you can have as many pipes as you want. For instance on my machine I can run ls -ltr | tail -1 | awk '{print $9}' | grep foo to look for the word foo in the last modified file.
I have some applications, and standard Unix tools sending their output to named-pipes in Solaris, however named pipes can only be read from the local storage (on Solaris), so I can't access them from over the network or place the pipes on an NFS storage for networked access to their output.
Which got me wondering if there was an analogous way to forward the output of command-line tools directly to sockets, say something like:
mksocket mysocket:12345
vmstat 1 > mysocket 2>&1
Netcat is great for this. Here's a page with some common examples.
Usage for your case might look something like this:
Server listens for a connection, then sends output to it:
server$ my_script | nc -l 7777
Remote client connects to server on port 7777, receives data, saves to a log file:
client$ nc server 7777 >> /var/log/archive
netcat (also known as nc) is exactly what you're looking for. It's getting to be reasonably standard, but not available on all systems.
socat seems to be a beefed-up version of netcat, with lots more features, but less commonly available.
On Linux, you can also use /dev/tcp/<host>/<port>. See the Advanced Bash-Scripting Guide for more information.
netcat will help establish a pipe over the network.
You may want to use one of:
ssh: secure (encrypted), already installed out-of-the-box on Solaris - but you have to set up a keypair for non-interactive sessions
e.g. vmstat 2>&1 | ssh -i private.key oss#remote.node "cat >vmstat.out"
netcat: simple to set up - but insecure and open to attacks
see http://www.debian-administration.org/articles/58 etc.
Everyone is on the right track with netcat. But I want to add that if you are piping into nc and expecting a response, you will need to use the -q <seconds> option. From the manual:
-q seconds
after EOF on stdin, wait the specified number of seconds and then quit. If seconds is negative, wait forever.
For instance, if you want to interact with your SSH Agent you can do something like this:
echo -en '\x00\x00\x00\x01\x0b' | nc -q 1 -U $SSH_AUTH_SOCK | strings
A more complete example is at https://gist.github.com/RichardBronosky/514dbbcd20a9ed77661fc3db9d1f93e4
* I stole this from https://ptspts.blogspot.com/2010/06/how-to-use-ssh-agent-programmatically.html
I am currently creating an overnight job that calls a Unix script which in turn creates and transfers a file using ftp. I would like to check all possible return codes. The man page for ftp doesn't list return codes. Does anyone know where to find a list? Anyone with experience with this? We have other scripts that grep for certain return strings in the log, and they send an email when in error. However, they often miss unanticipated codes.
I am then putting the reason into the log and the email.
The ftp command does not return anything other than zero on most implementations that I've come across.
It's much better to process the three digit codes in the log - and if you're sending a binary file, you can check that bytes sent was correct.
The three digit codes are called 'series codes' and a list can be found here
I wrote a script to transfer only one file at a time and in that script use grep to check for the 226 Transfer complete message. If it finds it, grep returns 0.
ftp -niv < "$2"_ftp.tmp | grep "^226 "
Install the ncftp package. It comes with ncftpget and ncftpput which will each attempt to upload/download a single file, and return with a descriptive error code if there is a problem. See the “Diagnostics” section of the man page.
I think it is easier to run the ftp and check the exit code of ftp if something gone wrong.
I did this like the example below:
# ...
ftp -i -n $HOST 2>&1 1> $FTPLOG << EOF
quote USER $USER
quote PASS $PASSWD
cd $RFOLDER
binary
put $FOLDER/$FILE.sql.Z $FILE.sql.Z
bye
EOF
# Check the ftp util exit code (0 is ok, every else means an error occurred!)
EXITFTP=$?
if test $EXITFTP -ne 0; then echo "$D ERROR FTP" >> $LOG; exit 3; fi
if (grep "^Not connected." $FTPLOG); then echo "$D ERROR FTP CONNECT" >> $LOG; fi
if (grep "No such file" $FTPLOG); then echo "$D ERROR FTP NO SUCH FILE" >> $LOG; fi
if (grep "access denied" $FTPLOG ); then echo "$D ERROR FTP ACCESS DENIED" >> $LOG; fi
if (grep "^Please login" $FTPLOG ); then echo "$D ERROR FTP LOGIN" >> $LOG; fi
Edit: To catch errors I grep the output of the ftp command. But it's truly it's not the best solution.
I don't know how familier you are with a Scriptlanguage like Perl, Python or Ruby. They all have a FTP module which you can be used. This enables you to check for errors after each command. Here is a example in Perl:
#!/usr/bin/perl -w
use Net::FTP;
$ftp = Net::FTP->new("example.net") or die "Cannot connect to example.net: $#";
$ftp->login("username", "password") or die "Cannot login ", $ftp->message;
$ftp->cwd("/pub") or die "Cannot change working directory ", $ftp->message;
$ftp->binary;
$ftp->put("foo.bar") or die "Failed to upload ", $ftp->message;
$ftp->quit;
For this logic to work user need to redirect STDERR as well from ftp command as below
ftp -i -n $HOST >$FTPLOG 2>&1 << EOF
Below command will always assign 0 (success) as because ftp command wont return success or failure. So user should not depend on it
EXITFTP=$?
lame answer I know, but how about getting the ftp sources and see for yourself
I like the solution from Anurag, for the bytes transfered problem I have extended the command with grep -v "bytes"
ie
grep "^530" ftp_out2.txt | grep -v "byte"
-instead of 530 you can use all the error codes as Anurag did.
You said you wanted to FTP the file there, but you didn't say whether or not regular BSD FTP client was the only way you wanted to get it there. BSD FTP doesn't give you a return code for error conditions necessitating all that parsing, but there are a whole series of other Unix programs that can be used to transfer files by FTP if you or your administrator will install them. I will give you some examples of ways to transfer a file by FTP while still catching all error conditions with little amounts of code.
FTPUSER is your ftp user login name
FTPPASS is your ftp password
FILE is the local file you want to upload without any path info (eg file1.txt, not /whatever/file1.txt or whatever/file1.txt
FTPHOST is the remote machine you want to FTP to
REMOTEDIR is an ABSOLUTE PATH to the location on the remote machine you want to upload to
Here are the examples:
curl --user $FTPUSER:$FTPPASS -T $FILE ftp://$FTPHOST/%2f$REMOTEDIR
ftp-upload --host $FTPHOST --user $FTPUSER --password $FTPPASS --as $REMOTEDIR/$FILE $FILE
tnftp -u ftp://$FTPUSER:$FTPPASS#$FTPHOST/%2f$REMOTEDIR/$FILE $FILE
wput $FILE ftp://$FTPUSER:$FTPPASS#$FTPHOST/%2f$REMOTEDIR/$FILE
All of these programs will return a nonzero exit code if anything at all goes wrong, along with text that indicates what failed. You can test for this and then do whatever you want with the output, log it, email it, etc as you wished.
Please note the following however:
"%2f" is used in URLs to indicate that the following path is an absolute path on the remote machine. However, if your FTP server chroots you, you won't be able to bypass this.
for the commands above that use an actual URL (ftp://etc) to the server with the user and password embedded in it, the username and password MUST be URL-encoded if it contains special characters.
In some cases you can be flexible with the remote directory being absolute and local file being just the plain filename once you are familiar with the syntax of each program. You might just have to add a local directory environment variable or just hardcode everything.
IF you really, absolutely MUST use regular FTP client, one way you can test for failure is by, inside your script, including first a command that PUTs the file, followed by another that does a GET of the same file returning it under a different name. After FTP exits, simply test for the existence of the downloaded file in your shell script, or even checksum it against the original to make sure it transferred correctly. Yeah that stinks, but in my opinion it is better to have code that is easy to read than do tons of parsing for every possible error condition. BSD FTP is just not all that great.
Here is what I finally went with. Thanks for all the help. All the answers help lead me in the right direction.
It may be a little overkill, checking both the result and the log, but it should cover all of the bases.
echo "open ftp_ip
pwd
binary
lcd /out
cd /in
mput datafile.csv
quit"|ftp -iv > ftpreturn.log
ftpresult=$?
bytesindatafile=`wc -c datafile.csv | cut -d " " -f 1`
bytestransferred=`grep -e '^[0-9]* bytes sent' ftpreturn.log | cut -d " " -f 1`
ftptransfercomplete=`grep -e '226 ' ftpreturn.log | cut -d " " -f 1`
echo "-- FTP result code: $ftpresult" >> ftpreturn.log
echo "-- bytes in datafile: $bytesindatafile bytes" >> ftpreturn.log
echo "-- bytes transferred: $bytestransferred bytes sent" >> ftpreturn.log
if [ "$ftpresult" != "0" ] || [ "$bytestransferred" != "$bytesindatafile" ] || ["$ftptransfercomplete" != "226" ]
then
echo "-- *abend* FTP Error occurred" >> ftpreturn.log
mailx -s 'FTP error' `cat email.lst` < ftpreturn.log
else
echo "-- file sent via ftp successfully" >> ftpreturn.log
fi
Why not just store all output from the command to a log file, then check the return code from the command and, if it's not 0, send the log file in the email?
Sending a message from the Unix command line using mail TO_ADDR results in an email from $USER#$HOSTNAME. Is there a way to change the "From:" address inserted by mail?
For the record, I'm using GNU Mailutils 1.1/1.2 on Ubuntu (but I've seen the same behavior with Fedora and RHEL).
[EDIT]
$ mail -s Testing chris#example.org
Cc:
From: foo#bar.org
Testing
.
yields
Subject: Testing
To: <chris#example.org>
X-Mailer: mail (GNU Mailutils 1.1)
Message-Id: <E1KdTJj-00025z-RK#localhost>
From: <chris#localhost>
Date: Wed, 10 Sep 2008 13:17:23 -0400
From: foo#bar.org
Testing
The "From: foo#bar.org" line is part of the message body, not part of the header.
In my version of mail ( Debian linux 4.0 ) the following options work for controlling the source / reply addresses
the -a switch, for additional headers to apply, supplying a From: header on the command line that will be appended to the outgoing mail header
the $REPLYTO environment variable specifies a Reply-To: header
so the following sequence
export REPLYTO=cms-replies#example.com
mail -aFrom:cms-sends#example.com -s 'Testing'
The result, in my mail clients, is a mail from cms-sends#example.com, which any replies to will default to cms-replies#example.com
NB: Mac OS users: you don't have -a , but you do have $REPLYTO
NB(2): CentOS users, many commenters have added that you need to use -r not -a
NB(3): This answer is at least ten years old(1), please bear that in mind when you're coming in from Google.
On Centos 5.3 I'm able to do:
mail -s "Subject" user#address.com -- -f from#address.com < body
The double dash stops mail from parsing the -f argument and passes it along to sendmail itself.
GNU mailutils's 'mail' command doesn't let you do this (easily at least). But If you install 'heirloom-mailx', its mail command (mailx) has the '-r' option to override the default '$USER#$HOSTNAME' from field.
echo "Hello there" | mail -s "testing" -r sender#company.com recipient#company.com
Works for 'mailx' but not 'mail'.
$ ls -l /usr/bin/mail
lrwxrwxrwx 1 root root 22 2010-12-23 08:33 /usr/bin/mail -> /etc/alternatives/mail
$ ls -l /etc/alternatives/mail
lrwxrwxrwx 1 root root 23 2010-12-23 08:33 /etc/alternatives/mail -> /usr/bin/heirloom-mailx
mail -s "$(echo -e "This is the subject\nFrom: Paula <johny#paula.com>\n
Reply-to: 1232564#yourserver.com\nContent-Type: text/html\n")"
milas.josh#gmail.com < htmlFileMessage.txt
the above is my solution....any extra headers can be added just after the from and before the reply to...just make sure you know your headers syntax before adding them....this worked perfectly for me.
Plus it's good to use -F option to specify Name of sender.
Something like this:
mail -s "$SUBJECT" $MAILTO -- -F $MAILFROM -f ${MAILFROM}#somedomain.com
Or just look at available options:
http://www.courier-mta.org/sendmail.html
It's also possible to set both the From name and from address using something like:
echo test | mail -s "test" example#example.com -- -F'Some Name<example2#example.com>' -t
For some reason passing -F'Some Name' and -fexample2#example.com doesn't work, but passing in the -t to sendmail works and is "easy".
I derived this from all the above answers. Nothing worked for me when I tried each one of them. I did lot of trail and error by combining all the above answers and concluded on this. I am not sure if this works for you but it worked for me on Ununtu 12.04 and RHEL 5.4.
echo "This is the body of the mail" | mail -s 'This is the subject' '<receiver-id1#email.com>,<receiver-id2#email.com>' -- -F '<SenderName>' -f '<from-id#email.com>'
One can send the mail to any number of people by adding any number of receiver id's and the mail is sent by SenderName from from-id#email.com
Hope this helps.
Here are some options:
If you have privelige enough, configure sendmail to do rewrites with the generics table
Write the entire header yourself (or mail it to yourself, save the entire message with all headers, and re-edit, and send it with rmail from the command line
Send directly with sendmail, use the "-f" command line flag and don't include your "From:" line in your message
These aren't all exactly the same, but I'll leave it to you look into it further.
On my portable, I have sendmail authenticating as a client to an outgoing mail server and I use generics to make returning mail come to another account. It works like a charm.
I aggregate incoming mail with fetchmail.
I don't know if it's the same with other OS, but in OpenBSD, the mail command has this syntax:
mail to-addr ... -sendmail-options ...
sendmail has -f option where you indicate the email address for the FROM: field. The following command works for me.
mail recepient#example.com -f from#example.com
On CentOS this worked for me:
echo "email body" | mail -s "Subject here" -r from_email_address email_address_to
Thanks BEAU
mail -s "Subject" user#address.com -- -f from#address.com
I just found this and it works for me. The man pages for mail 8.1 on CentOS 5 doesn't mention this. For -f option, the man page says:
-f Read messages from the file named by the file operand instead of the system mailbox. (See also folder.) If no file operand is specified, read messages from mbox instead of the system mailbox.
So anyway this is great to find, thanks.
On Debian 7 I was still unable to correctly set the sender address using answers from this question, (would always be the hostname of the server) but resolved it this way.
Install heirloom-mailx
apt-get install heirloom-mailx
ensure it's the default.
update-alternatives --config mailx
Compose a message.
mail -s "Testing from & replyto" -r "sender <sender#example.com>" -S replyto="sender#example.com" recipient#example.net < <(echo "Test message")
echo "body" | mail -S from=address#foo.com "Hello"
-S lets you specify lots of string options, by far the easiest way to modify headers and such.
echo "test" | mailx -r fake#example.com -s 'test' email#example.com
It works in OpenBSD.
this worked for me
echo "hi root"|mail -rsawrub#testingdomain.org -s'testinggg' root
On CentOS 5.5, the easiest way I've found to set the default from domain is to modify the hosts file. If your hosts file contains your WAN/public IP address, simply modify the first hostname listed for it. For example, your hosts file may look like:
...
11.22.33.44 localhost default-domain whatever-else.com
...
To make it send from whatever-else.com, simply modify it so that whatever-else.com is listed first, for example:
...
11.22.33.44 whatever-else.com localhost default-domain
...
I can't speak for any other distro (or even version of CentOS) but in my particular case, the above works perfectly.
What allowed me to have a custom reply-to address on an Ubuntu 16.04 with UTF-8 encoding and a file attachment:
Install the mail client:
sudo apt-get install heirloom-mailx
Edit the SMTP configuration:
sudo vim /etc/ssmtp/ssmtp.conf
mailhub=smtp.gmail.com:587
FromLineOverride=YES
AuthUser=???#gmail.com
AuthPass=???
UseSTARTTLS=YES
Send the mail:
sender='send#domain.com'
recipient='recipient#domain.com'
zipfile="results/file.zip"
today=`date +\%d-\%m-\%Y`
mailSubject='My subject on the '$today
read -r -d '' mailBody << EOM
Find attached the zip file.
Regards,
EOM
mail -s "$mailSubject" -r "Name <$sender>" -S replyto="$sender" -a $zipfile $recipient < <(echo $mailBody)
None of the above solutions are working for me...
#!/bin/bash
# Message
echo "My message" > message.txt
# Mail
subject="Test"
mail_header="From: John Smith <john.smith#example.com>"
recipients="recipient#example.com"
#######################################################################
cat message.txt | mail -s "$subject" -a "$mail_header" -t "$recipients"
I recent versions of GNU mailutils mail it is simply mail -r foo#bar.com.
Looking at the raw sent mail, it seems to set both Return-Path: <foo#bar.com> and From: foo#bar.com.
The answers provided before didn't work for me on CentOS5. I installed mutt. It has a lot of options. With mutt you do this this way:
export EMAIL=myfrom#example.com
export REPLYTO=myreplyto#example.com
mutt -s Testing chris#example.org