What I did is really simple to write incoming mail to file:
mailbox_command = path/to/mycript.py
luser_relay = unknown add, unknown = "|/path/to/myscript.py" to aliases file.
myscript.py reads from stdin line by line and writes the lines to file.
These lines redirects any incoming mail, no matter whether user exists
or not, to my script. My script reads data from stdin and writes it to file. If my script receives mail with one recipient it creates one file. But if it receives with 2 recipients, it creates 2 files. What I want is that it still should create 1 file. I want to make number of files created by script not related to number of recipients but related to number of messages received.
Thanks,
You can store in a file the email count, or just save each file, using epoch as filename
I would do it like this in ruby.
# read from stdin, and save a timestamp filename
IO.write Time.now.to_i.to_s,STDIN.read
Related
I am getting stuck after creating a ".csv data config element" by inserting a file and a TCP Sampler with the proper Server ID and Port Number.
I have also created a thread group for it.
Given your CSV files looks like:
foo
bar
baz
and you want to send the next line on each iteration of each user you can do the following setup:
Add CSV Data Set Config as a child of your TCP Sampler and configure it like:
In the TCP Sampler use the JMeter Variable you specified in the "Variable Names" where needed:
That's it, when you run your test the JMeter Variable placeholder will be substituted with the respective value from the CSV file:
I am streaming a Avro encoded file over the network from a S3 compliant object store and trying to read it and put it in some data-structure.
Issue: The issue I am facing sometimes ( one or two times in one / two days in test node when running continuously) is that half way through the file it hits this exception Invalid sync! in the nextRawBlock() method in DataFileStream class.
I would like to detect the root-cause of this and fix. I have been trying to
reproduce this in a test app but unable to do so successfully. I am looking for ideas on
what might potentially cause this ?
any better ways of reproducing this.
More details
a) The Avro file is not downloaded to disk , I get a handle to the file stream using S3ObjectInputStream and feed it to DataFileStream constructor
and then read from the stream directly.
b) The app tries to read records from the Avro encoded file in batches
of 500 records at a time.
c) The file contains a header section containing a Long count and a KV Map of String to Integer. After that it contains a array of records where each record contains a String and a long array. The schema uses Avro's union construct for enabling this.
d) Number of records in the file on average is around 5M
e) This entire download happens in separate thread and not in any user request.
f) The file is uploaded to the store by a separate process.
Other observation:
a) Upon failure the app closes the stream and tries to again download and read the stream. What I observe is this takes the node to a high oldgen state slowing down user requests.
I would like to implement the following functionality:
downloading all the files from a specified remote directory to a local directory.
after downloading all the files I need a list file which contains all the downloaded files.
(I only want this list file when all the files were downloaded successfully.)
Point 1:
Let's say we have around 10 files in the remote directory.
I can use an int-sftp:inbound-channel-adapter component to download all the files but 10 poll cycles are needed to download all of them since the inbound component is only able to download 1 file per poll request.
Spring Integration creates 10 File messages one by one.
Questions:
How can I identify the last file (message) received from the FTP server?
I don't want let users access to list file till all the files from the FTP is successfully received.
How can I achive this?
I can write file names into a list file using the int-file:outbound-channel-adapter but users can read temorary information from that file before the download process is finished.
How can I trigger the event that all files which are on the FTP are downloaded?
Thanks for your advices
Ferenc
First of all this isn't correct:
the inbound component is only able to download 1 file per poll request
You can configure it to to download infinitely during the single poll - max-messages-per-poll=-1. Anyway it is a default option on <poller>.
Anyway if it is your case to dowload one file per poll, you can go ahead with that requirements.
Since any Messaging system tries to achieve stateless paradigm, it is normal that one message doesn't know anything about another. And with that they all don't impact each other. The async scenario is the best for Messaging. With that we can process the second message quicker, than the first one.
Your requirement is enough interest and I won't dare to call it strange. Because any business may have place.
Since you are going to process several download files as one group, there will be need to have some marker on the remote server. Or it can be some timeframe, which we can extract from file timestamp. Or there will be need to store on the remote server some marker file to point that a set of files are finished and you can process them from your application using their local version. Would be great, if that marker file can contain a list of file names of that group.
Otherwise we don't have any hook to group messages for those files.
From other side you can consider to use <int-sftp:outbound-gateway> with MGET command: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/sftp.html#sftp-outbound-gateway
I have mp3 players set up on my site to play mp3s. At the moment, users can easily look through the source, run a search for "mp3" and download all of the music on my site. I know it's virtually impossible to completely prevent a determined user from downloading the music but I want to make it harder for the average user. Is there any way I can obfuscate the links to the mp3s?
Relevant site: http://boyceavenue.com/music
You did not specify the language you are using. To expand upon what Marc B wrote, I would recommend using the PHP http_send_file command along with the checksum of the file.
To send the file, use the following:
$filename = "/absolute/or/relative/path/to/file.ext";
$mime_type = "audio/mpeg"; // See note below
http_send_content_disposition($filename, true);
http_send_content_type($mime_type);
http_throttle(0.1, 2048);
http_send_file($filename);
If you are serving up multiple types of files using PHP 5.3.0 or later, you could determine the mimetype this way:
$filename = "/absolute/or/relative/path/to/file.ext";
$finfo = finfo_open(FILEINFO_MIME_TYPE);
$mime_type = finfo_file($finfo, $filename);
finfo_close($finfo);
Calculating the checksum is simple enough. Just use md5_file.
So, what can you do with the checksum?
Well, you could create an array of checksums and filenames that cross-reference each other. That is, you include the checksum in the links to the file, but have a little routine that looks up the checksum and delivers the mp3 file. You also could do this in a database. You also could do like some apps that store files in a directory structure based on their checksums (music/3/3a/song.mp3 with a checksum of 3a62f6 or whatever).
If you don't care about the filenames being mangled, you could save the files with a checksum for the filename. That could be done at upload time (if your files are being uploaded) or through a batch script (using the CLI).
Another thing you should do is to put a default document (index.php or whatever) that tells people to look elsewhere. Also disable browsing the directory. If only a very small number of people will need access, you could also throw a password on the directory, thus requiring a login to access the files.
I have a BizTalk 2009 send port that uses the %datetime_bts2000% macro in the file name. When I look at the tracked message event, I don't get the name of the actual file that was sent. I thought I could get it from the context property:
OutboundTransportLocation
SFTP://xxx#xxxx.xxx.com:22/Inbound/Encrypted/xxx.xxx.xxx.201101280410324
Promoted
http://schemas.microsoft.com/BizTalk/2003/system-properties
It turns out that the date is equivalent to my deployment date... not the actual date time stamp of the sent file.
Where do I find the actual filename generated by the Macro?
I happen to agree with #Bryan. I have in fact created such a pipeline to preserve OR set the attachment file name explicitly in a pipeline that I use for email.