I can stream viewable data in the Windows Console, I also stream the data into a text file using
--raw>output.txt as the command line
This text file will continue to grow in size as long as data is streaming. Is it possible to end streaming into the initial text file and start streaming into a second text file? Perhaps after 10 minutes?
you need to use pipe operator to redirect output to a tool that will perform required cutting. cutting can be implemented easily by yourself or probably you'll find something ready for use
Related
I have a lot of pcap files containing traces of different JPEG images being transmitted. Each of those images is divided into a couple hundred of packets of base64 encoded data. I have written a program to extract those images.
The problem is that here and there one or a few of these packets are missing.
I would like to recover as much of those images as possible. Currently the extracting software stops whenever a packet is missing and I end up with partial images. When opened in an image viewer, they look normal on the top, but then as soon as a packet is missing the rest of the image is grey due to the missing data.
So I would like to write a program that modifies a pcap file and inserts a dummy packet into the trace whenever a piece of an image is missing. This way, only the portion of the image that is really missing would be greyed out.
What would be an easy way to achieve this? I would appreciate any hints for useful tools/formats etc.
Thanks!
I can easily identify the locations in Wireshark where it would look as follows when following a TCP stream:
...
OJuOhptqVqcJAMzxzXGm7OlhdtuxWB4YtvGVYvd6V4gvUIQNkT7/AErZWaXLy4SpZ2IIA+1V
uHWJbtkPLB3XpXt7bVqMHsi3Cyn06TAI69P61nujdGNdEvCrMW[1368 bytes missing in capture file].eioyN6ANH4fMB3M7YUApIaUTP0r0Hgv8FlIUncjaenvXA/DEBzNSEn4fKUDP
zFegMOZKWkaSIjnt7VTMvgUviY8pvKrpkhBcTISeT0+1cLflTiidyeK7r4g2wfyret7nRpcC
...
But is there a way to automate it? Maybe some other file format that is easier to parse and modify?
You could render what you have into txt, edit it, and then use text2pcap to convert back to pcap.
I have referred this documentation.
They have mentioned when using client libraries for speech to text, "the long audio stream (up to 10 minutes)".
Whether speech to text accepts audio file greater than 10 minutes?
What will happen if we pass audio file > 10 minutes?
And in my use case, I need to pass audio file greater than 30 minutes. So what we have to do for these situations?
You can split your longer audio streams programmatically using ffmpeg and pass those chunks to this client library. You can check this to programmatically divide long audio streams into time-specified chunks: https://superuser.com/questions/525210/splitting-an-audio-file-into-chunks-of-a-specified-length.
You can then combine your text from these chunks to get the entire text back. Not the cleanest of the ways - but something that will scale.
I have a very large simulink input file (*.csv) that is too big to the handled in a single node...
I am wondering if it is possible to not read the whole file once at the beginning of the simulation, but instead stream the data in real time as needed by the simulation.
My first thought was to implement a custom script in JAVA or C# (sender) that reads line by line the input csv file and STREAM the data to simulink via TCP . Simulink would receive the data using a TCP block receiver.
My questions are two:
Is my approach feasible?
Given the problem stated, what would
be your solution?
I suspect it would be easier to the run simulation using sequential chunks of the data, saving the model state at the end of each chunk, and starting the simulation from the state saved at the end of the previous chunk. The doc describing how to do this is Save and Restore Simulation State as SimState.
You might try writing an S-Function in C that opens your file and streams your data line by line. The easiest way to do this would be using the S-Function Builder block. You would nonetheless need to parse your file in C. (which, in the case of a CSV file, shouldn't be hard)
I am trying to make .txt file look like a .jpg file so it can be sent across wi-fi using an Eye-Fi SD card. The card only sends .jpg files for several reasons. One reason is that the transmission path of the picture from an the SD card to the computer looks like:
Camera writes pictures to EYE-FI SD -> EYE-FI connects to local router -> local router uploads to EYE-FI servers -> EYE-FI servers upload to your computer.
[Explanation]
There could be some filter on the server end, so I found some software that allows the user to bypass the eye-fi servers so now I know I am only dealing with the SD card. It's also nice to know that no one else is looking at my files. After some experimentation, I figured out that I can put .jpg files on the card and have them transmitted once a picture is taken. I also found how that the pictures must be named in short format; a name not longer than 8 characters(excluding file extensions), this probably has to do with the fact the card is formatted in fat32 (the card can be reformatted and still works). I tried uploading a .txt file to the card and gave it a similar format, and renamed it as a .jpg file. It did transfer which indicates to me there is probably something other than a file extension which denotes how the file is formatted.
[Questions]
1) Is there someway I can spoof .txt files to make them look like .jpg files?
2) Is there some kind of program I can use (for linux) to play around with values on the card so I can figure out what triggers an upload? Any ideas one what could trigger the upload?
1) Yes, there are hex value in the file that indicate it is a .jpg. If you open up a .jpg file with a hex editor, you will notice that there are header lines that have a bunch of information about how the image was compress, sometimes what made the image, some firmware information etc. In the editor, you can find the string "FF D8" this indicates the beginning of image file. This is followed shortly by "FF C0". The next 6 bytes contain information about the size of the image, and (I am guessing) is used by whatever software displays the image. The end of a jpg file is denoted by the 2 byes "FF D9". Fun fact, I played around with the jpg file I was using and it seems that you can put text after the "FF D9" and still have the jpg operate. I thought this was neat. Source
None of this was needed to get the eye-fi to upload the file though. As I said in my question, the card needs the name of the file to be in short format (which means the title cannot have more than 8 characters) and needs to have an acceptable file extension, in my case I used ".jpg". I wrote a text file, and just saved it as a "text.jpg". I found that there is a minimum size required in order to transfer the file, which is strange.
My hex editor of choice for this was bless, it is good for opening files, but I have yet to figure out if it can open volumes. It doesn't seem like it can.
I made a console application, using directshow, that record from a live source (now a webcam, then a tv capture card), add current date and time in overlay and then save audio and video as .asf.
Now I want that the output file is going to change every 60 minutes without stopping the graph. I must not loose any seconds of the live stream.
The graph is something like this one:
http://imageshack.us/photo/my-images/543/graphp.jpg/
I took a look at the GMFBridge but I have some compiling problem with their examples.
I am wondering if there is a way to split what exist from the overlay filter and audio source, connect them to another asf writer (paused) and then switch them every 60 minutes.
The paused asf filter's file name must change (pp.asf, pp2.asf, pp4.asf ...). Something like this:
http://imageshack.us/photo/my-images/546/graph1f.jpg/
with pp1 paused. I found some people in internet that say that the asf writer deletes the current file if the graph does not go in stop mode.
Well, I have the product (http://www.videophill.com) that does exactly what you described (its used for broadcast compliance recording purposes) - and I found that only way to do that is this:
create a dshow graph that will be used only to capture the audio and video
then, at the end of the graph, insert samplegrabber filters, both for audio and video
then, use IWMWritter to create and save wmv file, using samples fetched from samplegrabber filters
when time comes, close one IWMWritter and create another one.
That way, you won't lose single frame when switching the output files.
Of course, there is also question of queue-ing and storing the samples (when switching the writters) and properly re-aligning the audio/video timestamps, but from my research, that's the only 'normal' way to do it, and I used in practice.
The solution is in writing a custom DShow filter with two input pins in your case. One for audio stream and the other for video stream. Inside that filter (doesn't have to be inside from the architecture point of view, because you can also use callbacks for example and do the job somewhere else) you should create asf files. While switching files, A/V data would be stored in cache (e.g. big enough circular buffer). You can also watch and modify A/V sync in that filter. For writing ASF files I would recommend Windows Media Format SDK.You can also add output pins if you like to pass A/V data further if necessary for preview, parallel streaming etc...
GMFBridge is a viable, but complicated solution, a more direct approach I have implemented in the past is querying your ASF Writer for the IWMWriterAdvanced2 interface and setting a custom sink. Within that interface you have methods to remove and add sinks to your ASF writer. The sink automatically connected will write to the file that you speficifed. One way to write whereever you want to is
1.) remove all default sinks:
pWriterAdv->RemoveSink(NULL);
2.) register a custom sink:
pWriterAdv->AddSink((IWMWriterSink*)&streamSink);
The custom sink can be a class that implements IWMWriterSink, which requires implementing callback methods that are called i.e. when the ASF header is written (OnHeader(/* [in] */ INSSBuffer *pHeader);) and when a data packet is written (OnDataUnit(/* [in] */ INSSBuffer *pDataUnit);) - in your implementation you can then write them wherever you want, for example offer additional methods on this class where you can specify the file name you want to write to.
Note that this solution does not quite get you were you want to if you need to write out the header information in each of the 60 minute files - after the initial header you will only get ASF packet data. A workaround for that could be to re-write the intial header before any packet data of each file, however this will produce an unindexed (non-seekable) ASF file.