i'm running syslog-ng inside docker, i'm collecting logs from local files, process them and then write to another logfile or send them to slack.
I noticed that whenever i need to updated syslog-ng config and restart container, syslog-ng re-reads all messages from source logfiles which causes duplications in destination files, slack channel.
Is there option to tell syslog that after reboot only new messages should be processed or maybe process only 1hour old logfiles?
i tried to google/check documentation but without luck, i'm probably not asking the question correctly because i would assume this option exist or not?
thanks
syslog-ng, by default, persists positions for sources where the concept of "bookmarking" or "position-tracking" is applicable.
This is true for regular file sources as well.
All you have to do is keep the syslog-ng persist file intact (syslog-ng.persist under the /var folder).
It sounds like you might be losing the persist file that was mentioned by MrAnno. You could try putting this and the log file on a persisted folder so that syslog-ng can reference where it was last and process from there.
Related
I need to backup RTMP streams that I send to my server (nginx with nginx-rtmp-module): in case one of them fails, I need another to be automatically substituted when I grab it from the server.
Is it possible?
I have figured out a bit tricky way to do this and put the solution on Github
It is a bunch of Shell scripts that allows you to do the following (assuming your server has DNS yourserver.ex:
Send your main stream to rtmp://yourserver.ex/main/somekey, backup stream to rtmp://yourserver.ex/backup/somekey and watch the result on rtmp://yourserver.ex/out/somekey. (More instructions on Github)
There could be a slight delay in switching streams, however, it works better than nothing.
I'm testing six distinct .jar-files that all need to handle the possibility of no online access.
Unfortunately, I am on a network disc, so disabling the network connection or pulling the ethernet cable does not work unless I move all the files to /tmp or /scratch and change my $HOME environment variable, all of which I'd rather not have to do as it ends up being a lot of work.
Is there a way to invoke java -jar and disable the process from accessing the internet? I have not found any such flag in the man-pages. Is there perhaps a UNIX-way of doing this, as in:
disallowinternetaccess java -jar Foo.jar
Tell your Java program to access the network through a proxy. For all internet access this would be a SOCKS5 proxy.
java -DsocksProxyHost=socks.example.com MyMain
I believe that if no proxy is running you should get an appropriate exception in your program. If you need full control of what is happening, you can look into - and possibly modify - http://jsocks.sourceforge.net/
See http://docs.oracle.com/javase/7/docs/technotes/guides/net/proxies.html for details.
Note: You can do this without any native Unix stuff, so this question fits perfectly fine on SO.
You need just turn on SecurityManager: -Djava.security.manager=default
see details - https://stackoverflow.com/a/4645781/814304
With this solution you can even handle which resource you want to show and which to hide.
I have a huge file on a server, e.g. a movie. Someone starts to down load that file. The download is not immediate, because the network has a certain maximum transmission rate. While the server is in the process of sending the file, I enter the command to delete the file.
What is the expected behavior?
Is the transmission cancelled?
Is the transmission completed first?
And if it is completed first, what if another request to download that file comes in before the delete command is carried out? Is that request queued behind the delete command or is it carried out parallel to other commands so that it is begun before the delete comes into effect, effectively keeping on blocking it.
On my desktop computer I cannot delete a file that is in use. Do web servers differ?
If the platform is Windows you can't delete the file.
if the platform is Unix- or Linux-based you can delete the file: however it remains in existence while it is open, which includes while it is being transmitted.
I'm not aware of any operating system where you are notified that a file you have open has been deleted, which is the only mechanism that could possibly cause transmission to be cancelled.
I work with another company that is moving from regular FTP to SFTP for their connections. They have informed me that my existing username and password are the same, but that I now need to connect via SFTP on port 22. I have a couple of PHP scripts that do some basic things, like connect to their site, get a directory listing, and upload and download a file each day. Those all work fine on FTP, so I need to just swap out the protocol to SFTP.
After doing some research, the consensus seems to be that phpseclib is the easiest and most robust way to perform SFTP using PHP. My server is running Linux and Apache. I downloaded the library and tried to run just the basic example given by phpseclib, but I get an error:
Notice: SSH_AUTH_SOCK not found in System/SSH/Agent.php on line 244
When I look at Agent.php, I see that the script is looking for SSH_AUTH_SOCK to be defined in either $_SERVER or $_ENV. I must be missing something obvious, but I have no idea how to get past this error.
You are trying to connect to SSH authentication agent (ssh-agent).
When the agent is run, it exports the SSH_AUTH_SOCK environment variable.
What is obviously not happening in your case.
But you didn't tell us, why are you trying to use the agent, if at all. And what did you do to set it up, if anything. So it's difficult to give you a more concrete advice.
See also How is SSH_AUTH_SOCK setup and used by ssh-agent?
Run the bellow command:
eval ssh-agent -s
It works fine for me.
I'm using WinSCP .NET assembly. When I call the Session.PutFiles method, it sends the following series of commands:
TYPE A
PASV
MLSD
TYPE A
PASV
STOR myfile
Is there a way to tell it NOT to send the MLSD? (MLSD requests the contents of the remote directory be sent back.) At the very least, I don't need this information so it's just wasting bandwidth. I don't even know how I would access it -- maybe WinSCP is doing something with it internally? What worries me more, though, is that I was given very specific specs about the series of FTP commands that I was supposed to send, which includes several non-standard commands, apparently the site at the other end has a customized FTP server. So I don't want an extra command to screw things up.
In the latest version, with default transfer settings, WinSCP does not use the MLSD command.
It's used only with OverwriteMode.Resume or OverwriteMode.Append to retrieve attributes of the remote file.
Also, WinSCP issues the MLSD command once for every destination directory (not for each file).