Pull last batch of messages from a nats jetstream - nat

I want to write a java client consumer, such that it need to pull a set of messages from a jetStream. Here I want to either pull all of the messages or pull the last messages(batch size can be specified).
Is there anyway to implement without using PullSubscribeOptions.durable() ?
Because I found out that if I don't use the durable, everytime the server sends the same batch.
public static void main(String[] args)
throws IOException, InterruptedException, JetStreamApiException, IllegalStateException, TimeoutException {
String userName = "local";
String password = "password";
String host = "0.0.0.0";
String port = "37733";
String streamName = "test_stream";
String subjectName = "test_subject_1";
int BATCH_SIZE = 100;
Connection nc = Nats.connect("nats://" + userName + ":" + password + "#" + host + ":" + port);
JetStream js = nc.jetStream();
PullSubscribeOptions pullOptions = PullSubscribeOptions.builder()
.durable("configurator_service_9")
.stream(streamName)
.build();
outerloop:
while(true){
JetStreamSubscription sub = js.subscribe(subjectName, pullOptions);
List<Message> messages = sub.fetch(1000, Duration.ofMillis(1000));
for (Message m : messages) {
System.out.println(m);
m.ack();
if(m==null) {
System.out.println(m);
nc.flush(Duration.ofSeconds(1));
nc.close();
break;
}
}
System.out.println("Message count is " + messages.size());
}
By the way as I see we can pull the entire steam on nats-cli. I need to do this again and again.

Related

Udp connection clients don't receive messages outside of local network even though ports are forwarded

Ok so I built a semi-simple multiplayer game that connects through UDP. However I can't get it to fully connect outside my local network. Now! I have seen this question asked and answered many times so here is where it differs from similar questions:
I have already port forwarded the correct ports.
That's right! I am a Minecraft veteran so the one thing I do know how to do is forward ports. ;)
Now, this works fine on my local network. It works if I run two instances on the same machine of course. But it also works if I use a different computer connecting to the LAN IP or if I use a different computer connecting to my routers public address. The connection only seems to fail if the messages are originating from outside the network they are being hosted on.
The host receives incoming messages from clients, but clients do not receive messages from host. And it doesn't mater who is the host and who is the client. So I had my friend try connecting to my game from his house and had a break point on the function that receives the messages. Sure enough I see the expected message come into the host, and then the host sends out the acknowledgement message, but his client never received it. Similarly, if my friend hosts and I try to connect, his host sees the message and sends the acknowledgment but my client never receives it. So at this point we both have the port forwarded but only the host ever receives anything.
So.... Anyone have any idea what I might be doing wrong? Or what Cthulhu of the internet I need to kill?
My Code Below:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
public enum MessageType
{
None = 0,
Connect = 1,
Command = 2,
EntityUpdate = 3,
Response = 4,
}
[System.Serializable]
public class Message
{
public MessageType _MessageType = MessageType.None;
public object Data = null;
public Guid Origin = Guid.Empty;
public Guid MessageID;
public Message(object data, MessageType type, Guid origin)
{
if (type == MessageType.None || data == null)
{
throw new Exception("Invalid Message!");
}
Data = data;
_MessageType = type;
Origin = origin;
MessageID = Guid.NewGuid();
}
}
public static class NetLayer
{
private const int FRAMES_BETWEEN_SENDS = 2;
private const int NUM_RETRIES = 10;
private const int FAMES_BEFORE_RETRY_ENTITY = 120;
private const int FAMES_BEFORE_RETRY_CLIENT = 30;
private const int BUFF_SIZE = 8 * 2048;
private const int DEFAULT_PORT = 27015;
private const string DEFAULT_ADDRESS = "localhost";
private const int MAX_BYTES = 1100;
private const int MAX_BYTES_PER_PACKET = 1100 - (322 + 14); //281 is number of existing bytes in empty serilized packet //14 is... I dunno. the overhead for having the byte aray populated I assume
private static Socket mSocket = null;
private static List<EndPoint> mClients = null;
private static Dictionary<EndPoint, PlayerData> mUnconfirmedClients = null;
private static string mServerAdress = DEFAULT_ADDRESS;
private static int mPort = DEFAULT_PORT;
public static bool IsServer { get { return mClients != null; } }
public static bool IsConnected = false;
public static bool Initiated { get { return mSocket != null; } }
private static Dictionary<Guid, Packet> mUnconfirmedPackets = null;
private static Queue<Packet> ResendQueue = new Queue<Packet>();
private static EndPoint mCurrentPacketEndpoint;
private static Guid mCurrentMessageID;
private static byte[] mDataToSend;
private static int mCurrentDataIndex = 0;
private static int mCurrentPacketIndex = 0;
private static int mTotalPacketsToSend = 0;
private static Queue<Packet> ResponseQueue = new Queue<Packet>();
private static Dictionary<Guid, Dictionary<int, Packet>> IncomingPacketGroups = new Dictionary<Guid, Dictionary<int, Packet>>();
private static Queue<QueueuedMessage> MessageQueue = new Queue<QueueuedMessage>();
public static int QueueCount { get { return MessageQueue.Count; } }
private static int mCurrentFramesToRetryClient;
private static int mCurrentFramesToRetryEntity;
private static int mLastSendFrame = 0;
[System.Serializable]
public class Packet
{
public byte[] Data;
public Guid MessageID;
public Guid PacketID;
public int PacketNumber;
public int NumPacketsInGroup;
public int NumBytesInGroup;
public bool RequireResponse;
[System.NonSerialized]
public int retries; //We don't need to send this data, just record it locally
[System.NonSerialized]
public EndPoint Destination; //We don't need to send this data, just record it locally
public Packet(byte[] data, Guid messageID, int packetNumber, int numPacketsinGroup, int numBytesInGroup, EndPoint destination, bool requireResponse)
{
Data = data;
if(data.Length > MAX_BYTES_PER_PACKET)
{
Debug.LogError("Creating a packet with a data size of " + data.Length + " which is greater then max data size of " + MAX_BYTES_PER_PACKET);
}
MessageID = messageID;
PacketID = Guid.NewGuid();
PacketNumber = packetNumber;
NumPacketsInGroup = numPacketsinGroup;
NumBytesInGroup = numBytesInGroup;
retries = 0;
Destination = destination;
RequireResponse = requireResponse;
}
}
private class QueueuedMessage
{
public Message m;
public EndPoint e;
public QueueuedMessage(Message message, EndPoint endpoint)
{
m = message;
e = endpoint;
}
}
[System.Serializable]
private class PacketResponse
{
public Guid MessageIDToConfirm;
public PacketResponse(Guid newGuid)
{
MessageIDToConfirm = newGuid;
}
}
public static void InitAsServer(string address, int port)
{
Reset();
mClients = new List<EndPoint>();
mUnconfirmedClients = new Dictionary<EndPoint, PlayerData>();
mSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
mSocket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.ReuseAddress, true);
mSocket.Blocking = false;
mSocket.SendBufferSize = BUFF_SIZE;
mSocket.ReceiveBufferSize = BUFF_SIZE;
mSocket.Bind(new IPEndPoint(IPAddress.Parse(address), port));
mPort = port;
IsConnected = true;
mUnconfirmedPackets = new Dictionary<Guid, Packet>();
Packet newPacket = new Packet(new byte[MAX_BYTES_PER_PACKET], Guid.NewGuid(), 1, 15, MAX_BYTES_PER_PACKET, null, true);
byte[] serilized = Util.Serialize(newPacket);
Debug.Log("Total Packet Size = " + serilized.Length);
}
public static void InitAsClient(string address, int port)
{
Reset();
mSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
mSocket.Blocking = false;
mSocket.SendBufferSize = BUFF_SIZE;
mSocket.ReceiveBufferSize = BUFF_SIZE;
mSocket.Bind(new IPEndPoint(IPAddress.Parse(GetLocalIPAddress()), port));
mServerAdress = address;
mPort = port;
mUnconfirmedPackets = new Dictionary<Guid, Packet>();
}
public static void NetTick()
{
if (Initiated)
{
//Keep retrying connection messages untill we get a response from the player, in case they miss our first message
if(IsServer)
{
mCurrentFramesToRetryClient++;
if (mCurrentFramesToRetryClient >= FAMES_BEFORE_RETRY_CLIENT)
{
mCurrentFramesToRetryClient = 0;
foreach (EndPoint ep in mUnconfirmedClients.Keys)
{
Debug.Log("Pinging client again: " + ep);
SendConnectionConfirmationToClient(ep, mUnconfirmedClients[ep]); //TODO Add timeout. Should stop pinging after 10 attempts
}
}
}
//Retry messages untill we get a responce or give up
mCurrentFramesToRetryEntity++;
if (mCurrentFramesToRetryEntity >= FAMES_BEFORE_RETRY_ENTITY)
{
List<Guid> deadMessages = new List<Guid>();
List<Guid> resendMessages = new List<Guid>();
mCurrentFramesToRetryEntity = 0;
foreach (Guid messageID in mUnconfirmedPackets.Keys)
{
Packet packet = mUnconfirmedPackets[messageID];
packet.retries++;
if (packet.retries > NUM_RETRIES)
{
Debug.LogError("Failed to send Packet " + messageID);
deadMessages.Add(messageID);
}
else
{
resendMessages.Add(messageID);
}
}
//Resend everything we have selected to resend
for (int i = 0; i < resendMessages.Count; i++)
{
Packet packet = mUnconfirmedPackets[resendMessages[i]];
ResendQueue.Enqueue(packet);
}
//Give up on messages that have been marked dead
for (int i = 0; i < deadMessages.Count; i++)
{
mUnconfirmedPackets.Remove(deadMessages[i]);
}
}
if (Time.frameCount - mLastSendFrame >= FRAMES_BETWEEN_SENDS)
{
if (ResponseQueue.Count > 0) //Send responses first
{
Packet p = ResponseQueue.Dequeue();
SendImmediate(p, p.Destination, false);
}
else if (ResendQueue.Count > 0) //Then resends
{
Packet p = ResendQueue.Dequeue();
SendImmediate(p, p.Destination, false);
}
else //Then normal messages
{
StreamSend();
}
mLastSendFrame = Time.frameCount;
}
}
}
public static string GetLocalIPAddress()
{
var host = Dns.GetHostEntry(Dns.GetHostName());
foreach (var ip in host.AddressList)
{
if (ip.AddressFamily == AddressFamily.InterNetwork)
{
return ip.ToString();
}
}
return null;
}
public static void Reset()
{
if(mSocket != null)
{
mSocket.Close();
}
mSocket = null;
mClients = null;
mServerAdress = DEFAULT_ADDRESS;
mPort = DEFAULT_PORT;
IsConnected = false;
}
public static void Send(Message newMessage, EndPoint endpoint = null)
{
if (IsServer)
{
if (endpoint != null)
{
SendToQueue(new QueueuedMessage(newMessage, endpoint));
}
else
{
for (int i = 0; i < mClients.Count; i++)
{
SendToQueue(new QueueuedMessage(newMessage, mClients[i]));
}
}
}
else
{
SendToQueue(new QueueuedMessage(newMessage, new IPEndPoint(IPAddress.Parse(mServerAdress), mPort)));
}
}
private static void SendToQueue(QueueuedMessage newPacket)
{
MessageQueue.Enqueue(newPacket);
}
private static void StreamSend()
{
if(mDataToSend != null)
{
//Get number of bytes to send in this packet
int bytesToSend = Mathf.Min(mDataToSend.Length - mCurrentDataIndex, MAX_BYTES_PER_PACKET);
//Populate packet byte array
byte[] data = new byte[bytesToSend];
for(int i = 0; i < bytesToSend; i++)
{
data[i] = mDataToSend[mCurrentDataIndex + i];
}
//Increment index by the nubmer of bytes sent
mCurrentDataIndex += bytesToSend;
//Create and send packet
Packet newPacket = new Packet(data, mCurrentMessageID, mCurrentPacketIndex, mTotalPacketsToSend, mDataToSend.Length, mCurrentPacketEndpoint, true);
SendImmediate(newPacket, mCurrentPacketEndpoint, true);
//Increment packet index
mCurrentPacketIndex++;
if(mCurrentDataIndex >= mDataToSend.Length)
{
//We have finished sending. Clear data array
mDataToSend = null;
}
}
else if(MessageQueue.Count > 0)
{
//Prepare the next group for packing
QueueuedMessage queueuedMessage = MessageQueue.Dequeue();
mCurrentMessageID = queueuedMessage.m.MessageID;
mCurrentPacketEndpoint = queueuedMessage.e;
mDataToSend = Util.Serialize(queueuedMessage.m);
mCurrentDataIndex = 0;
mCurrentPacketIndex = 1;
mTotalPacketsToSend = Mathf.CeilToInt((float)mDataToSend.Length / (float)MAX_BYTES_PER_PACKET);
}
}
public static void SendImmediate(Packet packet, EndPoint endpoint, bool verify)
{
byte[] data = Util.Serialize(packet);
if(data.Length > MAX_BYTES)
{
Debug.LogError("Attempting to send packet with " + data.Length + " bytes. This will most likely fail! Please ensure all packets are limited to " + MAX_BYTES + " bytes or less!");
}
mSocket.SendTo(data, endpoint);
//Don't assume the packet is late untill we have actually sent it.
//The old way of verifying. Instead things sending imediate that don't want to be veiried should set verify to false if (newMessage._MessageType != MessageType.Response && newMessage._MessageType != MessageType.Connect && !mUnconfirmedPackets.ContainsKey(newMessage.MessageID))
if (verify)
{
mUnconfirmedPackets.Add(packet.PacketID, packet);
}
}
public static void SendConnectionMessage(PlayerData playerData)
{
Message message = new Message(playerData, MessageType.Connect, playerData.ID);
Send(message);
}
public static void SendCommand(Command command, EndPoint ep = null)
{
Message message = new Message(command, MessageType.Command, GameManager.LocalPlayer.PlayerID);
Send(message, ep);
}
public static void SendEntityUpdate(EntityData entityData, EndPoint ep = null)
{
Message message = new Message(entityData, MessageType.EntityUpdate, GameManager.LocalPlayer.PlayerID);
Send(message, ep);
}
public static void Receive()
{
if (Initiated && mSocket.Available > 0)
{
byte[] bytes = new byte[BUFF_SIZE];
EndPoint epFrom = new IPEndPoint(IPAddress.Any, 0); //TODO Do something here to stop reciving messages from dead clients...
mSocket.ReceiveFrom(bytes, ref epFrom);
Reassemble(bytes, epFrom);
}
}
public static void Reassemble(byte[] bytes, EndPoint epFrom)
{
Packet receivedPacket = Util.Deserilize(bytes) as Packet;
//Create new packet group if one does not already exist
if(!IncomingPacketGroups.ContainsKey(receivedPacket.MessageID))
{
IncomingPacketGroups.Add(receivedPacket.MessageID, new Dictionary<int, Packet>());
}
Dictionary<int, Packet> currentPacketGroup = IncomingPacketGroups[receivedPacket.MessageID];
currentPacketGroup[receivedPacket.PacketNumber] = receivedPacket;
if(currentPacketGroup.Count == receivedPacket.NumPacketsInGroup)
{
byte[] completedData = new byte[receivedPacket.NumBytesInGroup];
int index = 0;
for(int i = 0; i < receivedPacket.NumPacketsInGroup; i++)
{
Packet currentPacket = currentPacketGroup[i + 1];
for(int j = 0; j < currentPacket.Data.Length; j++)
{
completedData[index] = currentPacket.Data[j];
index++;
}
}
//Clean up the packet group now that it is compleated
IncomingPacketGroups.Remove(receivedPacket.MessageID);
//Call this once we have re-assembled a whole message
Message receivedMessage = Util.Deserilize(completedData) as Message;
ReceiveCompleateMessage(receivedMessage, epFrom);
}
//Make sure to call back to sender to let them know we got this packet
if(receivedPacket.RequireResponse)
{
SendMessageResponse(receivedPacket.PacketID, epFrom);
}
}
private static void ReceiveCompleateMessage(Message receivedMessage, EndPoint epFrom)
{
if (receivedMessage.Origin == GameManager.LocalPlayer.PlayerID)
{
//Reject messages we sent ourselves. Prevent return from sender bug.
return;
}
//TODO Consider adding a check that adds the endpoing into the connections even if we don't have a connecton message because players can sometimes re-connect.
//TODO or maybe reject messages from unknown clients becasue we shouldn't trust them
switch (receivedMessage._MessageType)
{
case MessageType.None:
Debug.LogError("Recived message type of None!");
break;
case MessageType.EntityUpdate:
GameManager.ProcessEntityUpdate(receivedMessage);
break;
case MessageType.Command:
GameManager.ProcessCommand(receivedMessage);
break;
case MessageType.Response:
ProcessResponse(receivedMessage);
break;
case MessageType.Connect:
ProcessConnection(receivedMessage, epFrom);
break;
}
//If this is the server reciving a messgae, and that message was not sent by us
if (IsServer && receivedMessage._MessageType != MessageType.Connect && receivedMessage._MessageType != MessageType.Response && receivedMessage.Origin != GameManager.LocalPlayer.PlayerID)
{
//Mirror the message to all connected clients
Send(receivedMessage);
}
}
public static void ProcessConnection(Message receivedMessage, EndPoint epFrom)
{
//This is the only time where we should process the data directly in net layer since we need the endpoint data
if (IsServer)
{
if (!mClients.Contains(epFrom))
{
if (epFrom != null)
{
mClients.Add(epFrom);
Debug.Log("Client attempting to join: " + epFrom.ToString()); //TODO becasue we are binding socket this won't work locally anymore but see if it at least works globally before trying to fix
//Get new player from recived data
PlayerData newPlayer = receivedMessage.Data as PlayerData;
newPlayer.Team = GameManager.GetNewTeam();
newPlayer.HostHasResponded = true;
if (!mUnconfirmedClients.ContainsKey(epFrom))
{
mUnconfirmedClients.Add(epFrom, newPlayer);
}
SendConnectionConfirmationToClient(epFrom, newPlayer);
}
else
{
Debug.LogError("Got connection message from null!");
}
}
else
{
if (mUnconfirmedClients.ContainsKey(epFrom))
{
PlayerData newPlayerData = receivedMessage.Data as PlayerData;
if (newPlayerData.HostHasResponded)
{
mUnconfirmedClients.Remove(epFrom);
if (epFrom != null)
{
Debug.Log("Client Confirmed, sending update: " + epFrom.ToString());
//Sent all known entitys to the new client so it's up to date
foreach (Guid key in GameManager.Entities.Keys)
{
SendEntityUpdate(GameManager.Entities[key].GetEntityData(), epFrom);
}
}
}
else
{
Debug.LogWarning("We sent client an confirmation but it is still sending us initial connection messages: " + epFrom.ToString());
}
}
else
{
Debug.LogWarning("Client Already connected but still sent connnection message: " + epFrom.ToString());
}
}
}
else
{
ReciveConnectionConfirmationFromHost(receivedMessage, epFrom);
}
}
public static void ProcessResponse(Message message)
{
PacketResponse response = (PacketResponse)message.Data;
mUnconfirmedPackets.Remove(response.MessageIDToConfirm);
}
public static void SendConnectionConfirmationToClient(EndPoint epFrom, PlayerData newPlayer)
{
//endpoint can be null if this is the server connecting to itself
if (epFrom != null)
{
//Tell client we have accepted its connection, and send back the player data with the updated team info
Message message = new Message(newPlayer, MessageType.Connect, GameManager.LocalPlayer.PlayerID);
//Send(message, epFrom);
byte[] data = Util.Serialize(message);
Packet newPacket = new Packet(data, message.MessageID, 1, 1, data.Length, epFrom, false);
ResponseQueue.Enqueue(newPacket);
}
}
public static void ReciveConnectionConfirmationFromHost(Message receivedMessage, EndPoint epFrom)
{
//Clear the loading win we had active
GameObject.Destroy(GameManager.LoadingWin);
PlayerData newPlayer = receivedMessage.Data as PlayerData;
GameManager.LocalPlayer.SetPlayerData(newPlayer);
IsConnected = true;
PingBack(epFrom);
}
public static void PingBack(EndPoint ep)
{
PlayerData localPlayer = GameManager.LocalPlayer.GetPlayerData();
Message message = new Message(localPlayer, MessageType.Connect, localPlayer.ID);
byte[] data = Util.Serialize(message);
Packet newPacket = new Packet(data, message.MessageID, 1, 1, data.Length, ep, false);
ResponseQueue.Enqueue(newPacket);
}
private static void SendMessageResponse(Guid packetIDToConfirm, EndPoint epFrom)
{
PacketResponse messageIDtoConfirm = new PacketResponse(packetIDToConfirm);
Message message = new Message(messageIDtoConfirm, MessageType.Response, GameManager.LocalPlayer.PlayerID);
byte[] data = Util.Serialize(message);
Packet newPacket = new Packet(data, message.MessageID, 1, 1, data.Length, epFrom, false);
ResponseQueue.Enqueue(newPacket);
}
}
[Edit]
So I continued to do research and refine my code. (code above has been updated to current version). One thing I was definitely doing wrong before was blowing up the MTU because I didn't realize the external limit was much smaller than the internal limit. Still getting the same problem though. Server can see the client message, uses the incoming endpoint to send a connection message back, and client never gets it. Currently the server will attempt to resend the connection confirmation indefinitely until the client responds back which never happens because none of the server responses ever reach the client.
My understanding of a UDP punch through is that both machines must first connect to a known external server. That external server then gives each incoming connection the others IP and port, allowing them to talk to each other directly. However, since I am manually feeding the client the IP and port of the game server, I shouldn't need an external server to do that handshake.
Client doesn't need to connect to an external server first to get the game server IP and port because I manual type in the IP and port of the game server and the game server network has the port already forwarded, so the servers address IS already known. I know for a fact that the client message is reaching the game server because server logs the incoming connection.
The game server gets the IP and port of the client from the incoming message, which is exactly what would be handed to it by the external server by the handshake. I have double checked the IP the server receives from the connection and it matches the external IP address of the client. I also verified that both machines firewalls are allowing the program through.
I would deeply appreciate some more help on figuring out why server is failing to deliver a message back to the client. Explanations or links to useful documents or research please. Telling me to do more research or look online for the answers isn't helpful. I am already doing that. The point of a questions and answers is to get help from people who already know the answers.
The problem is that there exists NAT between LAN and WLAN. You cannot just work through it so simple.

How to intercept the headers from in call to one service and insert it to another request in gRPC-java?

I have two servers - HelloServer and WorldServer.
Both implement the same proto file:
// The greeting service definition.
service GreeterService {
// Sends a greeting
rpc GreetWithHelloOrWorld (GreeterRequest) returns (GreeterReply) {}
rpc GreetWithHelloWorld (GreeterRequest) returns (GreeterReply) {}
}
message GreeterRequest {
string id = 1;
}
// The response message containing the greetings
message GreeterReply {
string message = 1;
string id = 2;
}
I want to add traceIds to the requests. As far as I understand, this is achieved through adding the traceId in the Metadata object.
Here is the test I am using to check that the traceIds are passed along. The request is made to the HelloServer which in turns calls the WorldServer and then finally returns the response.
#Test
public void greetHelloWorld() {
String traceId = UUID.randomUUID().toString();
Metadata metadata = new Metadata();
metadata.put(MetadataKeys.TRACE_ID_METADATA_KEY, traceId);
Greeter.GreeterRequest greeterRequest = Greeter.GreeterRequest.newBuilder().setId(traceId).build();
ManagedChannel channel = ManagedChannelBuilder
.forAddress("localhost", 8080)
.usePlaintext(true)
.build();
AtomicReference<Metadata> trailersCapture = new AtomicReference<>();
AtomicReference<Metadata> headersCapture = new AtomicReference<>();
ClientInterceptor clientInterceptor = MetadataUtils.newAttachHeadersInterceptor(metadata);
ClientInterceptor metadataCapturingClientInterceptor = MetadataUtils.newCaptureMetadataInterceptor(headersCapture, trailersCapture);
GreeterServiceBlockingStub blockingStub = GreeterServiceGrpc.newBlockingStub(ClientInterceptors.intercept(channel, clientInterceptor, metadataCapturingClientInterceptor));
GreeterServiceStub asyncStub = GreeterServiceGrpc.newStub(channel);
try {
Greeter.GreeterReply greeterReply = blockingStub.greetWithHelloWorld(greeterRequest);
String idInResponse = greeterReply.getId();
String idInHeaders = headersCapture.get().get(MetadataKeys.TRACE_ID_METADATA_KEY);
logger.info("Response from HelloService and WorldService -- , id = {}, headers = {}", greeterReply.getMessage(), idInResponse, idInHeaders);
assertEquals("Ids in response and header did not match", idInResponse, idInHeaders);
} catch (StatusRuntimeException e) {
logger.warn("Exception when calling HelloService and WorldService\n" + e);
fail();
} finally {
channel.shutdown();
}
}
Implementation of ServerInterceptor:
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {
String traceId = headers.get(MetadataKeys.TRACE_ID_METADATA_KEY);
logger.info("objId=" + this.toString().substring(this.toString().lastIndexOf('#')) + " Trace id -- 1=" + headers.get(MetadataKeys.TRACE_ID_METADATA_KEY));
return next.startCall(new ForwardingServerCall.SimpleForwardingServerCall<ReqT, RespT>(call) {
#Override
public void sendHeaders(Metadata headers) {
headers.put(MetadataKeys.TRACE_ID_METADATA_KEY, traceId);
logger.info("objId=" + this.toString().substring(this.toString().lastIndexOf('#')) + " Trace id -- 2 " + headers.get(MetadataKeys.TRACE_ID_METADATA_KEY));
super.sendHeaders(headers);
}
#Override
public void sendMessage(RespT message) {
logger.info("objId=" + this.toString().substring(this.toString().lastIndexOf('#')) + " message=" + message.toString());
super.sendMessage(message);
}
}, headers);
Here is the implementation of the greetWithHelloWorld() method:
public void greetWithHelloWorld(com.comcast.manitoba.world.hello.Greeter.GreeterRequest request,
io.grpc.stub.StreamObserver<com.comcast.manitoba.world.hello.Greeter.GreeterReply> responseObserver) {
Greeter.GreeterRequest greeterRequest = Greeter.GreeterRequest.newBuilder().setId(request.getId()).build();
Metadata metadata = new Metadata();
metadata.put(MetadataKeys.TRACE_ID_METADATA_KEY, "");
ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 8081)
.usePlaintext(true).build();
AtomicReference<Metadata> trailersCapture = new AtomicReference<>();
AtomicReference<Metadata> headersCapture = new AtomicReference<>();
ClientInterceptor clientInterceptor = MetadataUtils.newAttachHeadersInterceptor(metadata);
ClientInterceptor metadataCapturingClientInterceptor = MetadataUtils.newCaptureMetadataInterceptor(headersCapture, trailersCapture);
GreeterServiceGrpc.GreeterServiceBlockingStub blockingStub = GreeterServiceGrpc.newBlockingStub(ClientInterceptors.intercept(channel, clientInterceptor, metadataCapturingClientInterceptor));
String messageFromWorldService = "";
String replyIdFromWorldService = "";
try {
Greeter.GreeterReply greeterReply = blockingStub.greetWithHelloOrWorld(greeterRequest);
messageFromWorldService = greeterReply.getMessage();
replyIdFromWorldService = greeterReply.getId();
logger.info("Response from WorldService -- {}, id = {}", messageFromWorldService, replyIdFromWorldService);
} catch (StatusRuntimeException e) {
logger.warn("Exception when calling HelloService\n" + e);
}
Greeter.GreeterReply greeterReply = Greeter.GreeterReply.newBuilder().setMessage("Hello" + messageFromWorldService).setId(replyIdFromWorldService).build();
responseObserver.onNext(greeterReply);
responseObserver.onCompleted();
}
The problem is in the greetWithHelloWorld() method, I don't have access to Metadata, so I cannot extract the traceId from the header and attach it to the request to the World server. However, if I put a breakpoint in that method, I can see that request object does have the traceId in it which is private to it and unaccessible.
Any ideas how can I achieve this? Also, is this the best way to do pass traceIds around? I found some references to using Context. What is the difference between Context and Metadata?
The expected approach is to use a ClientInterceptor and ServerInterceptor. The client interceptor would copy from Context into Metadata. The server interceptor would copy from Metadata to Context. Use Contexts.interceptCall in the server interceptor to apply the Context all callbacks.
Metadata is for wire-level propagation. Context is for in-process propagation. Generally the application should not need to interact directly with Metadata (in Java).

How to send a push notification to more than one device (iOS)?

I'm trying to optimize the push notifications on my server. For now I send those one by one (with an old library) and it takes a while (4 hours).
I refactored my service to send a notification with a lot of device tokens (For now I tried with batches of 500 tokens). For that I'm using the Redth/PushSharp library. I followed the sample code then I adapted it to send the notifications to severals device tokens.
PushService service = new PushService();
//Wire up the events
service.Events.OnDeviceSubscriptionExpired += new PushSharp.Common.ChannelEvents.DeviceSubscriptionExpired(Events_OnDeviceSubscriptionExpired);
service.Events.OnDeviceSubscriptionIdChanged += new PushSharp.Common.ChannelEvents.DeviceSubscriptionIdChanged(Events_OnDeviceSubscriptionIdChanged);
service.Events.OnChannelException += new PushSharp.Common.ChannelEvents.ChannelExceptionDelegate(Events_OnChannelException);
service.Events.OnNotificationSendFailure += new PushSharp.Common.ChannelEvents.NotificationSendFailureDelegate(Events_OnNotificationSendFailure);
service.Events.OnNotificationSent += new PushSharp.Common.ChannelEvents.NotificationSentDelegate(Events_OnNotificationSent);
service.Events.OnChannelCreated += new PushSharp.Common.ChannelEvents.ChannelCreatedDelegate(Events_OnChannelCreated);
service.Events.OnChannelDestroyed += new PushSharp.Common.ChannelEvents.ChannelDestroyedDelegate(Events_OnChannelDestroyed);
//Configure and start ApplePushNotificationService
string p12Filename = ...
string p12FilePassword = ...
var appleCert = File.ReadAllBytes(p12Filename);
service.StartApplePushService(new ApplePushChannelSettings(true, appleCert, p12FilePassword));
var appleNotification = NotificationFactory.Apple();
foreach (var itemToProcess in itemsToProcess)
{
itemToProcess.NotificationDateTime = DateTime.Now;
mobile.SubmitChanges();
string deviceToken = GetCleanDeviceToken(itemToProcess.MobileDevice.PushNotificationIdentifier);
appleNotification.ForDeviceToken(deviceToken);
}
service.QueueNotification(appleNotification
.WithAlert(itemsToProcess[0].MobileDeviceNotificationText.Text)
.WithSound("default")
.WithBadge(0)
.WithCustomItem("View", itemsToProcess[0].Value.ToString()));
//Stop and wait for the queues to drains
service.StopAllServices(true);
Then I tried to send 3 notifications to 2 devices. Only the first device got them (and the problem is not device-related because I tried with both of them separately).
Right after that an OperationCanceledException is thrown in the PushChannelBase class. So I don't know what's wrong. Any idea?
You should queue a separate notification for each item to process.
It is not possible set multiple device tokens on a single notification. The OperationCanceledException will occur, because you do.
Example: Console C# Application
This assumes
you have valid production and development certificates
you have stored multiple device tokens within your database
you have a notification that comes from your database
You are using PushSharp Library
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using PushSharp;
using PushSharp.Core;
using PushSharp.Apple;
using System.IO;
namespace MyNotification
{
class Program
{
//args may take "true" or "false" to indicate the app is running for
//development or production (Default = false which means Development)
static void Main(string[] args)
{
bool isProduction = false;
if (args != null && args.Length == 1)
{
Console.Write(args[0] + Environment.NewLine);
bool.TryParse(args[0], out isProduction);
}
try
{
//Gets a notification that needs sending from database
AppNotification notification = AppNotification.GetNotification();
if (notification != null && notification.ID > 0)
{
//Gets all devices to send the above notification to
List<IosDevice> devices = IosDevice.GetDevices(!isProduction);
if (devices != null && devices.Count > 0)
{
PushBroker push = new PushBroker();//a single instance per app
//Wire up the events for all the services that the broker registers
push.OnNotificationSent += NotificationSent;
push.OnChannelException += ChannelException;
push.OnServiceException += ServiceException;
push.OnNotificationFailed += NotificationFailed;
push.OnDeviceSubscriptionExpired += DeviceSubscriptionExpired;
push.OnChannelCreated += ChannelCreated;
push.OnChannelDestroyed += ChannelDestroyed;
//make sure your certifcates path are all good
string apnsCertFile = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "../../Certificate/Certificates_Apple_Push_Production.p12");
if (!isProduction)
apnsCertFile = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "../../Certificate/Certificates_Apple_Push_Development.p12");
var appleCert = File.ReadAllBytes(apnsCertFile);
push.RegisterAppleService(new ApplePushChannelSettings(isProduction, appleCert, "135TrID35")); //Extension method
foreach (IosDevice device in devices)
{
//if it is required to send additional information as well as the alert message, uncomment objects[] and WithCustomItem
//object[] obj = { "North", "5" };
push.QueueNotification(new AppleNotification()
.ForDeviceToken(device.DeviceToken)
.WithAlert(DateTime.Now.ToString())//(notification.AlertMessage)
//.WithCustomItem("Link", obj)
.WithBadge(device.BadgeCount + 1)
.WithSound(notification.SoundFile));//sound.caf
}
push.StopAllServices(waitForQueuesToFinish: true);
}
}
Console.WriteLine("Queue Finished, press return to exit...");
Console.ReadLine();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.ReadLine();
}
}
static void NotificationSent(object sender, INotification notification)
{
Console.WriteLine("Sent: " + sender + " -> " + notification);
}
static void NotificationFailed(object sender, INotification notification, Exception notificationFailureException)
{
Console.WriteLine("Failure: " + sender + " -> " + notificationFailureException.Message + " -> " + notification);
}
static void ChannelException(object sender, IPushChannel channel, Exception exception)
{
Console.WriteLine("Channel Exception: " + sender + " -> " + exception);
}
static void ServiceException(object sender, Exception exception)
{
Console.WriteLine("Service Exception: " + sender + " -> " + exception);
}
static void DeviceSubscriptionExpired(object sender, string expiredDeviceSubscriptionId, DateTime timestamp, INotification notification)
{
Console.WriteLine("Device Subscription Expired: " + sender + " -> " + expiredDeviceSubscriptionId);
}
static void ChannelDestroyed(object sender)
{
Console.WriteLine("Channel Destroyed for: " + sender);
}
static void ChannelCreated(object sender, IPushChannel pushChannel)
{
Console.WriteLine("Channel Created for: " + sender);
}
}
}

Opening a http connection on a specific port on Blackberry

I know that you can open a connection to a URL when programming in Blackberry but is it possible to open a connection on a specific port ? For example I want to send some data to the echo port of the server to check if it is alive and measure the ping time. Any ideas ?
Try something like this;
// Create ConnectionFactory
ConnectionFactory factory = new ConnectionFactory();
// use the factory to get a connection descriptor
ConnectionDescriptor conDescriptor = factory.getConnection("socket://www.abc.com:portnumber");
You can specify the port number when specifying the url to open the connection.
try this code :-
String host = "Your address" ;
new Thread()
{
run()
{
try {
SocketConnection connection = (SocketConnection)Connector.open("socket://" + host + ":80");
OutputStream out = connection.openOutputStream();
InputStream in = connection.openInputStream();
// Standard HTTP GET request all in text
// Only the required Host header, no body
String request = "GET / HTTP/1.1\r\n" +
"Host:" + host + "\r\n" +
"\r\n" +
"\r\n";
out.write(request.getBytes());
out.flush();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int firstByte = in.read();
if (firstByte >= 0) {
baos.write((byte)firstByte);
int bytesAvailable = in.available();
while(bytesAvailable > 0) {
byte[] buffer = new byte[bytesAvailable];
in.read(buffer);
baos.write(buffer);
bytesAvailable = in.available();
}
}
baos.close();
connection.close();
final_OP(new String(baos.toByteArray()) );
} catch (IOException ex) {
final_OP(ex.getMessage());
}
}
}.start();
public void final_OP(final String message) {
UiApplication.getUiApplication().invokeLater(new Runnable() {
public void run() {
Dialog.alert("Output" + message);
}
});
}

Java mail getInputStream left recipient

i'm writing a mail send method with javamail.
I can not understand why I get and error as: Recipient not set.
This is my code:
public static void sendMail(String to, String subj, String body, String attachmentName, byte[] attachment, String mime) throws Exception {
Properties p = System.getProperties();
Session session = Session.getInstance(p);
MimeMessage dummyMessage = new MimeMessage(session);
dummyMessage.setFrom(new InternetAddress(LovProvider.getOpzioni().get("mail.address")));
dummyMessage.setSubject(subj);
String[] tos = to.split(";");
Address[] tosAddr = new InternetAddress[tos.length];
for (int i = 0; i < tos.length; i++) {
tosAddr[i] = new InternetAddress(tos[i]);
}
dummyMessage.setRecipients(Message.RecipientType.TO, tosAddr);
Multipart mp = new MimeMultipart();
MimeBodyPart bp = new MimeBodyPart();
bp.setText(body);
mp.addBodyPart(bp);
if (attachmentName != null && attachment != null) {
DataSource dataSource = new ByteArrayDataSource(attachment, mime);
MimeBodyPart attachBodyPart = new MimeBodyPart();
attachBodyPart.setDataHandler(new DataHandler(dataSource));
attachBodyPart.setFileName(attachmentName);
mp.addBodyPart(attachBodyPart);
}
dummyMessage.setContent(mp);
//***** DEBUGGING here I find the recipient
sendMail(dummyMessage.getInputStream());
}
public static void sendMail(InputStream emlFile) throws Exception {
Properties props = System.getProperties();
props.put("mail.host", LovProvider.getOpzioni().get("mail.out.host"));
props.put("mail.transport.protocol", LovProvider.getOpzioni().get("mail.out.protocol"));
props.put("mail." + LovProvider.getOpzioni().get("mail.out.protocol") + ".port", LovProvider.getOpzioni().get("mail.out.port"));
Session mailSession = Session.getDefaultInstance(props, PasswordAuthentication.getAuth(LovProvider.getOpzioni().get("mail.out.user"), LovProvider.getOpzioni().get("mail.out.password")));
MimeMessage message = new MimeMessage(mailSession, emlFile);
//***** DEBUGGING here I CAN NOT find the recipient
Transport.send(message);
}
As I wrote in comments in debug mode i can see the recipient correctly set in the first part, whant i convert it to InputStream to the second method I can not find recipient anymore.
I can't debugging your code, but maybe this examples can help you:
Examples about sending/receiving mail via/from gmail
http://famulatus.com/component/search/?searchword=gmail&searchphrase=all&Itemid=9999

Resources