Monitor own network traffic in java - networking

I have a Java program which connects to the internet and send files (emails with attachments, SSL, javamail).
It sends only one email at a time.
Is there a way that my program could track the network traffic it itself is generating?
That way I could track progress of emails being sent...
It would also be nice if it was cross-platform solution...

Here's another approach that only works for sending messages...
The data for a message to be sent is produced by the Message.writeTo method and filtered through various streams that send it directly out the socket. You could subclass MimeMessage, override the writeTo method, wrap the OutputStream with your own OutputStream that counts the data flowing through it (similar to my other suggestion), and reports that to your program. In code...
public class MyMessage extends MimeMessage {
...
public void writeTo(OutputStream os, String[] ignoreList) throws IOException, MessagingException {
super.writeTo(new MyCountingStream(os), ignoreList);
}
}
If you want percent completion you could first use Message.writeTo to write the message to a stream that does nothing but count the amount of data being written, while throwing away the data. Then you know how big the message really is, so when the message is being sent you can tell what percent of the message that is.
Hope that helps...

Another user's approach is here:
Using JProgressBar with Java Mail ( knowing the progress after transport.send() )
At a lower level, if you want to monitor how many bytes are being sent, you should be able to write your own SocketFactory that produces Sockets that produce wrapped InputStreams and OutputStreams that monitor the amount of data passing through them. It's a bit of work, and perhaps lower level than you really want, but it's another approach.

I've been meaning to do this myself for some time, but I'm still waiting for that round tuit... :-)
Anyway, here's just a bit more detail. There might be gotchas I'm not aware of once you get into it...
You need to create your own SocketFactory class. There's a trivial example in the JavaMail SSLNOTES.txt file that delegates to another factory to do the work. Instead of factory.createSocket(...), you need to use "new MySocket(factory.createSocket(...))", where MySocket is a class you write that overrides all the methods to delegate to the Socket that's passed in the constructor. Except the getInputStream and getOutputStream methods, which have to use a similar approach to wrap the returned streams with stream classes you create yourself. Those stream classes then have to override all the read and write methods to keep track of how much data if being transferred, and make that information available however you want to your code that wants to monitor progress. Before you do an operation that you want to monitor, you reset the count. Then as the operation progresses, the count will be updated. What it won't give you is a "percent completion" measure, since you have no idea how much low level data needs to be sent to complete the operation.

Related

How to await connections to socket and convert them into a FusedStream in Rust

I have a loop {} around a futures::select!. I'd like to
await new connections on a Unix socket.
await a new message on one of the Unix streams created in 1. (preferably with some kind of uuid(sent by the connecting client in 1. directly after connecting) attached to it)
I still haven't understood the concept of a Stream correctly, especially futures-rs's "FusedFuture". I tried to create the streams I need using tokio's async-stream, but that didn't work as it does not implement futures-rs's FusedFuture (which I require, as I check other streams in the same select! that benefit/require FusedFutures is_terminated() method)
I think something like tokio_stream::StreamMap is interesting (the key as uuid and the value as a Stream that contains a tupel with the message received on the socket and reference to the UnixStream in order to write back.
I'm really sorry that I cannot provide any examples or something similar to make the issue less unclear. I've tried a lot but I was unable to come any closer to my goal. I hope the way a wrote this is somewhat understandable 😟
Whether to use a std::os::unix::net::UnixListener or tokio::net::UnixListener is up in the air. I've tried both but was unable to achieve anything with either of them.

Efficiently connecting an asynchronous IMFSourceReader to a synchronous IMFTransform

Given an asynchronous IMFSourceReader connected to a synchronous only IMFTransform.
Then for the IMFSourceReaderCallback::OnReadSample() callback is it a good idea not to call IMFTransform::ProcessInput directly within OnReadSample, but instead push the produced sample onto another queue for another thread to call the transforms ProcessInput on?
Or would I just be replicating identical work source readers typically do internally? Or put another way does work within OnReadSample run the risk of blocking any further decoding work within the source reader that could have otherwise happened more asynchronously?
So I am suggesting something like:
WorkQueue transformInputs;
...
// Called back async
HRESULT OnReadSampleCallback(... IMFSample* sample)
{
// Push sample and return immediately
Push(transformInputs, sample);
}
// Different worker thread awoken for transformInputs queue samples
void OnTransformInputWork()
{
// Transform object is not async capable
transform->TransformInput(0, Pop(transformInputs), 0);
...
}
This is touched on, but not elaborated on here 'Implementing the Callback Interface':
https://learn.microsoft.com/en-us/windows/win32/medfound/using-the-source-reader-in-asynchronous-mode
Or is it completely dependent on whatever the source reader sets up internally and not easily determined?
It is not a good idea to perform a long blocking operation in IMFSourceReaderCallback::OnReadSample. Nothing is going to be fatal or serious but this is not the intended usage.
Taking into consideration your previous question about audio format conversion though, audio sample data conversion is fast enough to happen on such callback.
Also, it is not clear or documented (depends on actual implementation), ProcessInput is often instant and only references input data. ProcessOutput would be computationally expensive in this case. If you don't do ProcessOutput right there in the same callback you might run into situation where MFT is no longer accepting input, and so you'd have to implement a queue anyway.
With all this in mind you would just do the processing in the callback neglecting performance impact assuming your processing is not too heavy, or otherwise you would just start doing the queue otherwise.

Logging queued connections

I am using a complex state engine system build with Qt 5.4 (using custom state engine classes).
Part of that code is logging of events, transitions, etc. It is very important for me to log all events the engine/state objects are receiving so I can completely track what is happening in the state engines.
For most event types logging is easy. However I failed to log queued connections (i.e. meta call events). QMetaCallEvent is private so there is not much I can do. However it is hard to believe that such an integral part of Qt can not be inspected properly.
Is there some way I missed that allows to log queued connections (including signal name, slot name, sender name, receiver name and arguments if possible)?
Install an event filter and intercept events with ev->type() == QEvent::MetaCall. All members visible in the debugger.
Need access to private headers? Use QT += core-private in your .pro file.
(tone mode="original poster")It's hard to believe that nobody reads documentation(/tone)
There is no official API that allows to do what I intend.
Inspecting QMetaCall events (using private framework headers) is a bad idea. First they are private (and may break your code any time) second the QMetaCall event sender() pointer may be invalid if sender was deleted immediately and I could not find a clean way to inspect events in such cases.
The way I am using now is totally different. Instead of inspecting the arriving event objects I am using a modified variant of QSignalSpy that allows to do more than the original class and helps logging the signal emissions using secondary connections.
In my situation this seems feasible even if it is pretty complicated and not a universal solution. At least no private headers are involved.

How to handle passing different types of serialized messages on a network

I'm currently sitting with the problem of passing messages that might contain different data over a network. I have created a prototype of my game, and now I'm busy implementing networking for my game.
I want to send different types of messages, as I think it would be silly to constantly send all the information every network-tick and I would rather send different messages that contain different data. What would be the best way to distinguish what message is received on the receiving side?
Currently I have a system where I prepend a string which distinguishes a certain type of message. My message is then sent through my own message parser class where it determines the type, and deserializes it to the correct type.
What I would like to know is if there is a better way of doing this? It seems like it should be a fairly common problem and so there must be a more trivial solution, unless I'm already doing it the trivial way.
Thanks!
I have read again carefully your question, and now I do not understand what is your problem, you say Currently I have a system where I prepend a string which distinguishes a certain type of message. My message is then sent through my own message parser class where it determines the type, and deserializes it to the correct type.
Looks OK, you may reduce the size of your message with my answer below horizontal line but the principle stays identical.
This the right way for asynchronous communication, but if you do synchrone you know that when you send A message you will receive B answer, so you do not have to prepend with a string which distinguishes the message, but you have to take care not sending another message before having the answer from the previous ...
So if you know how is formatted the answer you do not need any identification bytes, for example you know that the first four bytes is an integer, then a float on eight bytes, etc ...
Use boost::serialization, typically you save your structures, even with pointers, within a dumb bytes buffer, send that buffer over your network, and the other side de-serialize.
This example shows how Boost.Serialization can be used with asio to encode and decode structures for transmission over a socket.
Even if it is using boost::asio you could extract only the serialization part easily.

Flex Remote Object multiple parallel calls

I'm on Flash Builder 4.5 and I'm using remote object with amfphp and when I call two method (method1 and method2) at the same time the response of method2 always arrives after method1's response even though method2 is much more faster to return the result.
Here's the scenario:
I set a remote object which refers to a remote php class "Newsletter" which contains the sendNewsletter and getProgress methods.
Here's the code:
-sendNewsletter() reads the email archive and send the newsletter. After each email has sent it writes a log into the database.
-getProgress() reads the log wrote by sendNewsletter, counts how many email have been sent, compares it with the total number of the email that have to be sent and return the progress percentage
From the flex interface the users select a Newsletter to be sent and click on a "send" button which calls a function that calls the sendNewsletter() and then instantiate a loop of calls to getProgress (as you can see when getProgress returns something it calls the setProgress which updates a progress bar and calls getProgress again until the progress percentage reach 100%.
So right after I call sendNewsletter() I call getProgress() on the same remoteClass().
sendNewsletter() can take several minutes to complete (in my tests for sending 4 email it takes about 4 seconds so I think that sending thousands of email will take much more!!) and the trouble I'm encountering here is that getProgress() result arrives only after sendNewsletter() concludes its execution while what I would like to achieve is:
-call sendNewsletter()
-while sendNewsletter() does its stuff() call getProgress several time in order to get the progress percentage:
What I've got now:
call to sendNewsletter()----------------------->response
call to getProgress()------------------------------------->response after sendNewsletter()
What I want to achieve:
sendNewsletter()------------------------------------------------------------------>response()
getProgress()--->response, getProgress() again--->response-->getProgress()-->respone-->etc...
I read many post on how to work around this problem but no solution worked for me.
I tried to to "emulate" to different channel by creating two remote object with endpoint set once to gateway.php?parallel=0 and once gateway.php?parallel=1, but flash builder still send everything in one big request and get the response in one big http packet (I need tow different packet since sendNewsletter takse ages to complete compared to getProgress)
I also tried to delay the call of getProgress() after sendNewsletter() with a Timer of 500ms and flash builder makes two different calls (I can see them in firebug) but the call of getProgress gives response after sendNewsletter() anyway.
I alse tried to call sendNewsletter this way
this.myNewsletter.getOperation("sendNewsletter").send(idNewsletter)
this.myNewsletter.getOperation("sendNewsletter").cancel()
in order to let flash builder forget about the response but no way!!!
So far the only way to work around I found is creating a common httpservice which refers to a php which instantiate the Newsletter class and calls the getProgress method.
By using two different channel I can call the getProgress httpservice while sendNewsletter is being execute. It works but I don't like it and I don't want to create an httpservice for each method I need to call in background, so I want to achieve this with remote object only.
Anyone has addressed the same problem?
You Flash builder guru, I know you're around, please help me!!!!!
Thanks in advance!!!
Bye,
Luke
P.S.
sorry if this post is a little bit long but the situation it's quite complicated.
I don't know exactly, what you want to..
But when working with Remote Object, there is a best practices to use Responder to handle the responses that were arrived parallel from the single Remote Object.
So try to add responder to your service calls like
remoteObject.methodCall().addResponder(new YourResponder(resultEvent, faultEvent));
So when specific response will be come, it will be handled by your different custom responders.
And by that you will be able to handle your response separately.

Resources