I have to imagine an interface that can decompress one tar.gz (ZIJA) file into 3 tar.gz files (ZIJB, ZIJC and ZIJD).
These three files has to be treated independantly during the orchestration as it is an asynchronous one (only one ZIJA.tar.gz and his siblings can be treated at a time).
If I don't make a corelation between my receives shape and my send shapes my ZIJA.tar.gz is decompressed into 3 files (ZIJB, ZIJC and ZIJD) correctly but the orchestration is launched 3 times (Once for the sibling file). And I want only 1 use to manage the 3 files
The problem is when I receive the file through my custom pipeline to decompress the ZIJA file, I have only 1 file that is treated by my orchestration (the first one, ZIJB). The other are not treated at all.
Edit : Thank you guys for your answer but let me explain with more details.
The objective of this interface is to decompress twice in a row tar.gz files organised this way : ZIJA.Tar.gz containing 1 to 3 files (ZIJB.targz and/Or ZIJC.targz and/Or ZIJD.targz) which has to be decompress in respective and different folders.
Moreover, I have to keep the information of the first targz content in order to recompress the original file with exactly the same organisation.
Between decompressing and recompressing I call SAP to make a treatement and I have to wait the answer of sap to start the recompression step.
To conclude, I can't start an other first decompression step (of ZIJA.targz) until all the steps above are finished.
One ZIJA.targz at a time. (One instance, only)
Sum up : ZIJA.targz => decompression => 1 to 3 files => decompression of the files toward the right separate folders => Call Sap => answer of SAP with the new files => Recompression of the 1 to 3 files => recompression of the ZIJA.targz with the exact same content of the first one I have decompressed.
Related
"File Source (Async)" filter supports only one file per it's life.
Is the a way to play two files in a sequence without rebuilding a graph?
File Source (Async) only supplies random access byte stream to the filter graph, there are other components vital for playback: demultiplexers, decoders. No, it is not possible to enqueue another file through File Source (Async) filter.
Playing multiple files seamlessly otherwise is possible but requires to split graph into parts and connect them together in terms of sending data from one graph (reading from file, the one you rebuild with file change) to the other (with renderers, the one being never rebuilt and providing seamless playback user experience).
Read up other questions on bridging graphs:
GMFBridge usage in DirectShow
When changing a file name, Recording Start is overdue for 3 seconds.
I am trying to make .txt file look like a .jpg file so it can be sent across wi-fi using an Eye-Fi SD card. The card only sends .jpg files for several reasons. One reason is that the transmission path of the picture from an the SD card to the computer looks like:
Camera writes pictures to EYE-FI SD -> EYE-FI connects to local router -> local router uploads to EYE-FI servers -> EYE-FI servers upload to your computer.
[Explanation]
There could be some filter on the server end, so I found some software that allows the user to bypass the eye-fi servers so now I know I am only dealing with the SD card. It's also nice to know that no one else is looking at my files. After some experimentation, I figured out that I can put .jpg files on the card and have them transmitted once a picture is taken. I also found how that the pictures must be named in short format; a name not longer than 8 characters(excluding file extensions), this probably has to do with the fact the card is formatted in fat32 (the card can be reformatted and still works). I tried uploading a .txt file to the card and gave it a similar format, and renamed it as a .jpg file. It did transfer which indicates to me there is probably something other than a file extension which denotes how the file is formatted.
[Questions]
1) Is there someway I can spoof .txt files to make them look like .jpg files?
2) Is there some kind of program I can use (for linux) to play around with values on the card so I can figure out what triggers an upload? Any ideas one what could trigger the upload?
1) Yes, there are hex value in the file that indicate it is a .jpg. If you open up a .jpg file with a hex editor, you will notice that there are header lines that have a bunch of information about how the image was compress, sometimes what made the image, some firmware information etc. In the editor, you can find the string "FF D8" this indicates the beginning of image file. This is followed shortly by "FF C0". The next 6 bytes contain information about the size of the image, and (I am guessing) is used by whatever software displays the image. The end of a jpg file is denoted by the 2 byes "FF D9". Fun fact, I played around with the jpg file I was using and it seems that you can put text after the "FF D9" and still have the jpg operate. I thought this was neat. Source
None of this was needed to get the eye-fi to upload the file though. As I said in my question, the card needs the name of the file to be in short format (which means the title cannot have more than 8 characters) and needs to have an acceptable file extension, in my case I used ".jpg". I wrote a text file, and just saved it as a "text.jpg". I found that there is a minimum size required in order to transfer the file, which is strange.
My hex editor of choice for this was bless, it is good for opening files, but I have yet to figure out if it can open volumes. It doesn't seem like it can.
I am using connect direct with scp and trying to send some pdf files from unix to mainframes.
On unix end, I have archive containing pdfs which I am simply renaming to ABC.XYZ.LMN.PQR (mainframe file name) and then sending to mainframe.
The archive contains variable length pdf files.
However, the requirement is:
For any variable length file mainframe needs to know the maximum possible length, of any record in the file. For e.g. say the LRECL is 1950.
How to include LRECL as well when the pdf files inside the archive file to be sent is of variable length ?
An alternative would be to transfer the files to a Unix System Services file (z/OS Unix) instead of a "traditional" z/OS dataset. Then the folks on the mainframe side could use their utilities to copy the file to a "traditional" mainframe dataset if that's what they need.
For Variable Blocked datasets Only! If your maximum record size is 1950 you will want to specify RECFM=VB,LRECL=1954 adding 4 bytes more than your maximum record. This 4 byte allowance is for the Record Descriptor Word (RDW). If you need to specify BLKSIZE then the minimum is the size of LRECL plus another 4 bytes.
So in your example, your JCL will have DCB parameters that looks like: RECFM=VB,LRECL=1954,BLKSIZE=1958
Ideally, for optimal storage, BLKSIZE should be set to the largest size that does not exceed the device specific recommendation. i.e. TAPE devices typically try to use BLKSIZE=32670 (32 * 1024K - 8 for RDW & BDW). Disk drives may vary but in our shop BLKSIZE=23476 is considered optimal.
Incorrect blocking factors can waste tremendous amounts of space. When in doubt, let your system defaults apply or consult with your local system guru's for their device specific recommendations.
I am writing an application which uses the DCM MKDIR, We save our images in little bigger name, but when I am trying to use the DCMMKDIR application, which is asking me to input file name max 8 characters.
Presently, I am planning to rename my Images starting from 1 to N. But remapping these images to known names(on the disk) will be bit difficult(I feel).
are there are any other methods/process to achieve the same.
The restriction of the filename to eight characters is derived from the DICOM Standard to ensure compatibility with applications that support e.g. only ISO 9660 as a file system for CDs.
About the naming you can have a look at the specifications the german CD Testat (http://www.dicom-cd.de/docs/DRG-RequirementsSpecification-2006.pdf). As a vendor you can be certified to conform to certain standards for interoperability of patient CDs which is currently the most common usage of DICOMDIRs.
The DICOMDIR file generated by DCMMKDIR is mainly a kind of index to tell an application, what DICOM files in a certain directory exist and this kind of file structure is usually more common for transfer media.
It is very common to use subdirectories to circumvent the restriction on max 8 length filenames. For example, a file could be 20110331/183421/000001 which would identify it properly with date, time and index without exceeding the somewhat arcane filename limit.
I want to get a better understanding of how disk reads work for a simple ls command and for a cat * command on a particular folder.
As I understand it, disk reads are the "slowest" operation for a server/any machine, and a webapp I have in mind will be making ls and cat * calls on a certain folder very frequently.
What are "ball park" estimates of the disk reads involved for an "ls" and for a "cat *" for the following number of entries?
Disk reads for ls Disk reads for cat *
200
2,000
20,000
200,000
Each file entry is just a single line of text
Tricky to answer - which is probably why it spent so long getting any answer at all.
In part, the answer will depend on the file system - different file systems will give different answers. However, doing 'ls' requires reading the pages that hold the directory entries, plus reading the pages that hold the inodes identified in the directory. How many pages that is - and therefore how many disk reads - depends on the page size and on the directory size. If you think in terms of 6-8 bytes of overhead per file name, you won't be too far off. If the names are about 12 characters each, then you have about 20 bytes per file, and if your pages are 4096 bytes (4KB), then you have about 200 files per directory page.
If you just list names and not other attributes with 'ls', you are done. If you list attributes (size, etc), then the inodes have to be read too. I'm not sure how big a modern inode is. Once upon a couple of decades ago on a primitive file system, it was 64-bytes each; it might have grown since then. There will be a number of inodes per page, but you can't be sure that the inodes you need are contiguous (adjacent to each other on disk). In the worst case, you might need to read another page for each separate file, but that is pretty unlikely in practice. Fortunately, the kernel is pretty good about caching disk pages, so it is unlikely to have to reread a page. It is impossible for us to make a good guess on the density of the relevant inode entries; it might be, perhaps, 4 inodes per page, but any estimate from 1 to 64 might be plausible. Hence, you might have to read 50 pages for a directory containing 200 files.
When it comes to running 'cat' on the files, the system has to locate the inode for each file, just as with 'ls'; it then has to read the data for the file. Unless the data is stored in the inode itself (I think that is/was possible in some file systems with biggish inodes and small enough file bodies), then you have to read one page per file - unless partial pages for small files are bunched together on one page (again, I seem to remember hearing that could happen in some file systems).
So, for a 200 file directory:
Plain ls: 1 page
ls -l: 51 pages
cat *: 251 pages
I'm not sure I'd trust the numbers very far - but you can see the sort of data that is necessary to improve the estimates.