I've been trying to get NServiceBus.Router working to allow endpoints using the AmazonSQS transport and the AzureServiceBus transport to communicate with each other. So far, I am able to get a command sent from the ASB endpoint through the router and handled by the SQS endpoint. However, when I publish an event from the SQS endpoint, it is not handled by the ASB endpoint even though I have registered the SQS endpoint as a publisher. I have no idea what I'm doing wrong, but looking at every example I can find from from the docs, it seems like it should work.
I have already tried adding another forwarding route in the reverse of what is below (SQS to ASB), but that did not solve the issue.
The endpoints and router are each running in .net 5 worker services.
I've made a sample project that reproduces the issue here, but here are some quick at-a-glance snippets that show the relevant setup:
Router Setup
var routerConfig = new RouterConfiguration("ASBToSQS.Router");
var azureInterface = routerConfig.AddInterface<AzureServiceBusTransport>("ASB", t =>
{
t.ConnectionString(Environment.GetEnvironmentVariable("ASB_CONNECTION_STRING"));
t.Transactions(TransportTransactionMode.ReceiveOnly);
t.SubscriptionRuleNamingConvention((entityType) =>
{
var entityPathOrName = entityType.Name;
if (entityPathOrName.Length >= 50)
{
return entityPathOrName.Split('.').Last();
}
return entityPathOrName;
});
});
var sqsInterface = routerConfig.AddInterface<SqsTransport>("SQS", t =>
{
t.UnrestrictedDurationDelayedDelivery();
t.Transactions(TransportTransactionMode.ReceiveOnly);
var settings = t.GetSettings();
// Avoids a missing setting error
//https://github.com/SzymonPobiega/NServiceBus.Raw/blob/master/src/AcceptanceTests.SQS/Helper.cs#L18
bool isMessageType(Type t) => true;
var ctor = typeof(MessageMetadataRegistry).GetConstructor(
BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance, null,
new[] {typeof(Func<Type, bool>)}, null);
#pragma warning disable CS0618 // Type or member is obsolete
settings.Set<MessageMetadataRegistry>(ctor.Invoke(new object[] {(Func<Type, bool>) isMessageType}));
#pragma warning restore CS0618 // Type or member is obsolete
});
var staticRouting = routerConfig.UseStaticRoutingProtocol();
staticRouting.AddForwardRoute("ASB", "SQS");
routerConfig.AutoCreateQueues();
ASB Endpoint Setup
var endpointConfiguration = new EndpointConfiguration("ASBToSQSRouter.ASBEndpoint");
var transport = endpointConfiguration.UseTransport<AzureServiceBusTransport>();
transport.SubscriptionRuleNamingConvention((entityType) =>
{
var entityPathOrName = entityType.Name;
if (entityPathOrName.Length >= 50)
{
return entityPathOrName.Split('.').Last();
}
return entityPathOrName;
});
transport.Transactions(TransportTransactionMode.ReceiveOnly);
transport.ConnectionString(Environment.GetEnvironmentVariable("ASB_CONNECTION_STRING"));
var bridge = transport.Routing().ConnectToRouter("ASBToSQS.Router");
bridge.RouteToEndpoint(typeof(ASBToSQSCommand), "ASBToSQSRouter.SQSEndpoint");
bridge.RegisterPublisher(typeof(ASBToSQSEvent), "ASBToSQSRouter.SQSEndpoint");
endpointConfiguration.EnableInstallers();
SQS Endpoint Setup (nothing special because it doesn't need to know about the router)
var endpointConfiguration = new EndpointConfiguration("ASBToSQSRouter.SQSEndpoint");
var transport = endpointConfiguration.UseTransport<SqsTransport>();
transport.UnrestrictedDurationDelayedDelivery();
transport.Transactions(TransportTransactionMode.ReceiveOnly);
endpointConfiguration.EnableInstallers();
Any help would be greatly appreciated!
Unfortunately one of the recent SQS transport releases contains a change that makes the subscription work only by default in the context of a full NServiceBus endpoint. This feature is subscription batching.
In order for the Router to work correctly (Router does not run a full endpoint, just NServiceBus transport), you need to add this magic line to the SQS interface configuration:
settings.Set("NServiceBus.AmazonSQS.DisableSubscribeBatchingOnStart", true);
This is an undocumented flag that disables the subscription batching and allows router to complete the subscribe operations normally.
I am sorry for the inconvenience.
Related
I spend a lot of time trying to understand where is the mistake.
I try to swap USDC to any token on uniswap router v3 with ethers.js on frontend.
I use Polygon network in app and in metamask too.
I have a confirmation in my metamask to approve USDC, but after it I have no confirmation about swapping. I see an error in console:
inpage.js:1 MetaMask - RPC Error: Internal JSON-RPC error. code: -32603 data: code: -32000 message: "execution reverted"
This is my code:
const provider = new ethers.providers.Web3Provider(window.ethereum);
const send_account = await provider.getSigner().getAddress(); //my address
//Swap from USDC to MTS or any other token
addressOfUSDC='0x2791bca1f2de4661ed88a30c99a7a9449aa84174' // Address of USDC in Polygon
const ERC20Contract = new ethers.Contract(addressOfUSDC, ERC20ABI, provider);
swapRouterAddress="0xe592427a0aece92de3edee1f18e0157c05861564"; //Uniswap v3 router
indadr="0x440416D85E5D51424da371297E3d7Ef28312BF62"; //Any token address to which I want to swap
approvalAmount=12; //12 USDC
await ERC20Contract
.connect(provider.getSigner())
.approve(swapRouterAddress, approvalAmount*1000000)
//After this step we see confirmation to spend 12$ in metamask
bn=ethers.BigNumber.from(approvalAmount*1000000); //BigNumber from approvalAmount
const params = {
tokenIn: addressOfUSDC,
tokenOut: indadr,
fee: ethers.BigNumber.from(3000),
recipient: send_account,
deadline: ethers.BigNumber.from(Math.floor(Date.now() / 1000) + (60 * 10)),
amountIn: bn,
amountOutMinimum: ethers.BigNumber.from(0),
sqrtPriceLimitX96: ethers.BigNumber.from(0),
}
const UniswapContract= new ethers.Contract(swapRouterAddress, ERC20ABI2, provider);
dd=await UniswapContract
.connect(provider.getSigner())
.exactInputSingle(params)
console.log(dd);
And after this step I have no confirmation in metamask and error in console: MetaMask - RPC Error: Internal JSON-RPC error. execution reverted
Maybe the problem is that I use two different router ABIs?
Please help me to understand the problem! Thanks in advanse!
I tried to change ABI, router address. Also I tried to use Uniswap V2 router with function swapExactTokensForTokens.
I try to swap USDC to any token on uniswap router v3 with ethers.js on
frontend.
you cannot swap USDC with any kind token. you need to have pool pair contract. for example ETH/USD, you have to find the pool address for WETH/USDC and create a pool contract. WETH is ERC20 version of ETH
you need to get the pool address and create a pool contract to get the immutables.
const poolContract = new ethers.Contract(
poolAddress,
IUniswapV3PoolABI,
provider
);
this is what you set as the first params
tokenIn: immutables.token1,
tokenOut: immutables.token0,
Then you have to have a swap router contract
const swapRouterContract = new ethers.Contract(
swapRouterAddress,
SwapRouterABI,
provider
);
the way how you created the swap router contract is wrong
const UniswapContract= new ethers.Contract(swapRouterAddress, ERC20ABI2, provider);
you are passing ERC20ABI2 here. this should be SwapRouterABI. ERC20ABI2 is used to construct the USDC token contract and other token contract that you are going to swap. Because eventually, you are swapping two ERC20 contracts
If you need to swap ERC20 tokens, first you have to allow the swap amount, so you have to call approve method of token contracts that you construct. you need to allow Uniswap to withdraw money from token contracts.
Finally you call the contract function
dd=await UniswapContract
// maybe you should have const signer= await provider.getSigner()
.connect(provider.getSigner())
.exactInputSingle(params)
:: Using A Custom DistributedCache (Redis Implementation) ::
On all Redirect(url) calls, the session changes. I know the official MS Docs says that IAuthenticationHandler "Created per request to handle authentication for a particular scheme". Meaning even redirects cause a new session
public async Task<AuthenticateResult> AuthenticateAsync() { }
The issue I am having is that during the new session ALL CONTEXT is lost. Everything I saved in the Context (including the ClaimsPrincipal of the user) gets lost. I cannot even fetch transients or singletons attached to my service through:
var SessionProvider = Context.RequestServices.GetService(typeof(ISessionProvider)) as SessionProvider;
Because there is no context. Here is the implementation I followed::
ASP.NET Core 2.0 authentication middleware
Please guide me on any assistance on how to persist Session during AuthenticationAysnc() calls.
Edit :: Here is my service code
service.AddAuthentication(options => {
options.DefaultAuthenticateScheme = "CoreAuthScheme";
options.DefaultChallengeScheme = "CoreAuthScheme";
options.DefaultScheme = "CoreAuthScheme";
}).AddCustomAuth(o => {})
And With Web App
//Register Session Security (Goes before Routing)
app.UseSession();
app.UseAuthentication();
app.UseAuthorization();
Thank you
Solution ::
Proper Implementation of these two library srcs are needed if you are using a custom IDistributedCache. Obviously The IDistributedCache here is Redis Implementation, but the same goes for SQL, and other DB caching (get the proper src implementation of IDistributedCache)
Session Management
https://github.com/dotnet/aspnetcore/tree/c85baf8db0c72ae8e68643029d514b2e737c9fae/src/Middleware/Session/src
IDistributedCache (Redis)
https://github.com/dotnet/aspnetcore/tree/c85baf8db0c72ae8e68643029d514b2e737c9fae/src/Caching/StackExchangeRedis/src
The reason why I was losing session is because the default implementation of DistributedSession uses Offsets and special encodings to process values returned from the IDistributedCache Redis I implemented. See below:
DistributedSession.Deserialize(...)
var expectedEntries = DeserializeNumFrom3Bytes(content);
_sessionIdBytes = ReadBytes(content, IdByteCount);
for (var i = 0; i < expectedEntries; i++)
{
var keyLength = DeserializeNumFrom2Bytes(content);
var key = new EncodedKey(ReadBytes(content, keyLength));
var dataLength = DeserializeNumFrom4Bytes(content);
_store.SetValue(key, ReadBytes(content, dataLength));
}
Once you have those two libraries properly sourced and implemented. Session Management Works :). AuthenticateAsync calls the Distributed Session which calls DistributedCache and data is properly serialized and deserialized
----------------------------------------------------------- alimaslax
I have set up an SSE connection using Koa like so:
const Koa = require('koa');
const Router = require('koa-router');
const app = new Koa();
const router = new Router();
// Sets up the HTTP header and sends a message via SSE
function writeSSE(ctx, message) {
ctx.res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
'Access-Control-Allow-Origin': '*',
});
ctx.res.write(`id: 01\n`);
ctx.res.write(`data: ${message}\n\n`);
}
// Router Middleware
router.get('/stream', (ctx, next) => {
writeSSE(ctx, 'Stream reached.');
});
app.use(router.routes()).use(router.allowedMethods());
app.listen(8080);
Where my React components starts the connection like so:
new EventSource("http://localhost:8080/stream")
The component then receives the answer sent by the writeSSE method on the backend.
But for some reason the /stream endpoint is reached every 3 seconds or so, as if the connection was being reestablished.
And my error listener on the front-end catches a CONNECTING event every time.
this.state.source.onerror = (e) => {
if (e.target.readyState == EventSource.CONNECTING) {
console.log("Connecting...");
}
};
And on the back-end, ctx.response equals { status: 404, message: 'Not Found', header: {} }.
Would anyone know the cause of this issue? Is it linked to the way I use Koa?
this is a bit too late, but I will write my experience with sse using Koa.
First of all using ctx.res directly is not much appreciated by Koa, if you still want to use it make sure to put ctx.respond = false to bypass koa response mecanism.
In my experience a stream is the best way to use SSE with Koa you can do something like :
const stream = require('stream');
const koa = require('koa');
const app = new koa();
app.use(async ctx => {
ctx.set({
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});
ctx.status = 200;
const stream = new stream.PassThrough()
ctx.body = stream; // here koa will pipe the ctx.res to stream and end the ctx.res when ever the stream ends.
let counter = 5;
const t = setInterval(() => {
stream.write(`data: hi from koa sse ${counter}`);
counter--;
if (counter === 0) {
stream.end();
clearInterval(t);
}
}, 1000);
});
Hope this help anyone will play with SSE on koa.
PS: I wrote this on hurry if there is anything wrong with code tell me and I will correct it.
I'm in the process of implementing a Koa-based server for SSE. I've been running into the same problem, and here are my thoughts / working solution:
As far as I can tell, the reason why onmessage and onerror keep getting called is because the EventSource object on the client side is emitting an error event. This is causing the connection to be disconnected, which causes the client to send another request to initialize the stream to the server. From here, the process repeats itself indefinitely.
Based on my own testing, EventSource is emitting an error due to the data that is being sent back from the server. Per the docs, a 200 response that has as Content-Type other than 'text/event-stream' will cause a failure.
In your example, you have declared your response as 'text/event-stream' and are passing a string into the ctx.res.write method. While this looks correct, and in fact works when using comparable code and Express, it seems that it doesn't work in Koa. However, if you change the 'data' you are writing to your response to a stream, such as this example here, you'll find that the connection establishes correctly.
Maybe try the following:
//require Passthrough
const PassThrough = require('stream').PassThrough;
//then, in your writeSSE function, try this:
let stream = new PassThrough();
stream.write(`data: ${message}\n\n`);
ctx.res.write(stream);
I'm not 100% sure why this change works. My best guess is that there is something about Koa's ctx object that prevents a plain string or template literal from being viewed as valid text/event-stream data, but this is entirely supposition (this begs the question as to why it works in Express, but hopefully someone more knowledgeable can answer this for both of us). From what I've seen of other snippets published online, the stream approach is the one to take in Koa.
I'm not sure what your results will be, as it looks like you may be using a different version of Koa than I am, but I'd give it a shot. I was able to get my connection established correctly making this small change.
I am in the process to try to separate out the mobile from the desktop part of my application and thought I try DDP.connect as a means for the mobile application to share data with the desktop application.
My first hurdle is concerning Meteor internal collections and publications.
How am I supposed to authenticate users? I know I can call the login method to authenticate a user, but that still doesn't give me all the other nice reactive features I am used to with Meteor.users
Is this supposed to work, and if so what is the pattern.
Thanks
This is what integrated completely with a remote server (except code refresh, which forgets user session)
if (Meteor.isClient) {
Meteor.connection = DDP.connect('http://remote.site.com');
Accounts.connection = Meteor.connection;
Meteor.users = new Meteor.Collection('users');
SomeCollection = new Meteor.Collection('remote_collection');
Meteor.connection.subscribe('users');
Meteor.connection.subscribe('remote_collection');
// rest if the code just as always
}
This way you can use login directly (via accounts-base, accounts-passed, etc) and don't need to call a login method. Just add accounts-ui and include {{>loginButtons}} and it works
I had a similar problem. I wanted to have two different front-ends (although both are for desktop) to the same back-end, so they could use same database, publications, and methods. After looking through Meteor's source code (version 1.1.0.3) I've managed to do this as follows.
1) Start back-end server project.
$ meteor --port 3100
2) In front-end project(s), put following in server/server.config.js.
var backendUrl = process.env.BACKEND_URL;
if (backendUrl) {
__meteor_runtime_config__.BACKEND_URL = backendUrl;
__meteor_runtime_config__.ACCOUNTS_CONNECTION_URL = backendUrl;
console.log('config', __meteor_runtime_config__);
}
3) In front-end project(s), put following in client/lib/client.connection.js. APS is just a namespace for my application. Be sure to have this loaded before you use subscriptions or methods (that's why it's in lib folder).
if (typeof APS == 'undefined') APS = {};
var backendUrl = __meteor_runtime_config__.BACKEND_URL;
if (backendUrl) {
APS.backendConnection = DDP.connect(backendUrl);
Meteor.connection = APS.backendConnection;
_.each(['subscribe', 'methods', 'call', 'apply', 'status', 'reconnect', 'disconnect'], function(name) {
Meteor[name] = _.bind(Meteor.connection[name], Meteor.connection);
});
console.log('connected to backend', APS.backendConnection);
}
4) Start front-end server with BACKEND_URL environment variable pointing to your back-end server.
$ BACKEND_URL=http://192.168.33.10:3100 meteor
That's all. Refresh on client works OK. And we don't have to fiddle with Accounts.*.
UPDATE: Just found a problem with my solution. When calling server methods, this.userId is always null. This is because Meteor.connection and Accounts.connection were two separate connections, despite to the same BACKEND_URL. Upon authentication, user ID gets associated only with the latter. Fixed client.connection.js is as follows.
if (typeof APS == 'undefined') APS = {};
var backendUrl = __meteor_runtime_config__.BACKEND_URL;
if (backendUrl) {
APS.originalConnection = Meteor.connection;
// Accounts is already connected to our BACKEND_URL
APS.backendConnection = Accounts.connection;
// Reusing same (authenticated) connection for method calls and subscriptions
Meteor.connection = APS.backendConnection;
_.each(['subscribe', 'methods', 'call', 'apply', 'status', 'reconnect', 'disconnect'], function(name) {
Meteor[name] = _.bind(Meteor.connection[name], Meteor.connection);
});
console.log('Connected to backend', APS.backendConnection);
}
You can authenticate using code like this:
var connection = DDP.connect("<url>")
To authenticate
connection.call("login", {"password":"qwerty","user":{"username":"user_1"}});
to get the user, add this on the other server)
Meteor.methods({
whoami: function() { return Meteor.user() }
});
Then you can run further commands as if you were authenticated, like this to get who's logged in
console.log(connection.call("whoami");
User account creation/Authentication:
In client.js, create a DDP connection and set it to Accounts.connection
Accounts.connection = Meteor.remoteConnection;
Create an Accounts.users collection in the client and subscribe its contents from the external server as below.
Accounts.users = new Meteor.Collection('users', {connection: Meteor.remoteConnection});
Meteor.remoteConnection.subscribe('users');
Now call the login method required as below and set the token returned in the localStorage. This works for all the internal clicks and routing.
Meteor.loginWithPassword(login_email, login_password, function(err) {
submit_button.button("reset");
if (err)
{
console.log(err);
pageSession.set("errorMessage", err.message);
return false;
}else{
console.log("logged in as "+Meteor.userId());
var token = Accounts._storedLoginToken();
localStorage.setItem('_storedLoginToken', token);
}
});
The problem with the above code is that, the token is reset after every manual client refresh. The result object contains the below signed in information. We have to take the token and login with token for every external client refresh.
id:"5RigABaSzbARHv9ZD"
token:"MItg8P59gsl_T5OXtaWRSjUnETqzns0hGEV26xWYxj7"
tokenExpires:Thu Jul 20 2017 12:46:31 GMT+0530 (India Standard Time)
In client.js, start-up call the loginwithtoken function with the returned token as below, whenever there is no user available.
var user = Meteor.user();
var token = localStorage.getItem('_storedLoginToken');
if(user==null){
console.log("Token"+token +user);
if(token)
Meteor.loginWithToken(token, function(err){
// this is going to throw error if we logged out
if(!err) {
console.log('logged in !!!! ',token);
}
});
}
Meteor throws an error while logging in with the token,
Error logging in with token: Error: You've been logged out by the server. Please log in again. [403]
To overcome this issue, we have to write a tracker function to track the logged in session and login again if required. This is basically a hack suggested in meteor forums.
Tracker.autorun(function () { var user = Meteor.user(); var token
= localStorage.getItem('_storedLoginToken'); if(user==null){ console.log("Token"+token +user); if(token)
Meteor.loginWithToken(token, function(err){
// this is going to throw error if we logged out
if(!err) {
console.log('logged in !!!! ',token); }
}); } });
Reset the localStorage if user navigates to the login path. In Layout.js,
if(path=='/login')
localStorage.setItem('_storedLoginToken',null);
I would like to get the user IP address in my meteor application, on the server side, so that I can log the IP address with a bunch of things (for example: non-registered users subscribing to a mailing list, or just doing anything important).
I know that the IP address 'seen' by the server can be different than the real source address when there are reverse proxies involved. In such situations, X-Forwarded-For header should be parsed to get the real public IP address of the user. Note that parsing X-Forwarded-For should not be automatic (see http://www.openinfo.co.uk/apache/index.html for a discussion of potential security issues).
External reference: This question came up on the meteor-talk mailing list in august 2012 (no solution offered).
1 - Without a http request, in the functions you should be able to get the clientIP with:
clientIP = this.connection.clientAddress;
//EX: you declare a submitForm function with Meteor.methods and
//you call it from the client with Meteor.call().
//In submitForm function you will have access to the client address as above
2 - With a http request and using iron-router and its Router.map function:
In the action function of the targeted route use:
clientIp = this.request.connection.remoteAddress;
3 - using Meteor.onConnection function:
Meteor.onConnection(function(conn) {
console.log(conn.clientAddress);
});
Similar to the TimDog answer but works with newer versions of Meteor:
var Fiber = Npm.require('fibers');
__meteor_bootstrap__.app
.use(function(req, res, next) {
Fiber(function () {
console.info(req.connection.remoteAddress);
next();
}).run();
});
This needs to be in your top-level server code (not in Meteor.startup)
This answer https://stackoverflow.com/a/22657421/2845061 already does a good job on showing how to get the client IP address.
I just want to note that if your app is served behind proxy servers (usually happens), you will need to set the HTTP_FORWARDED_COUNT environment variable to the number of proxies you are using.
Ref: https://docs.meteor.com/api/connections.html#Meteor-onConnection
You could do this in your server code:
Meteor.userIPMap = [];
__meteor_bootstrap__.app.on("request", function(req, res) {
var uid = Meteor.userId();
if (!uid) uid = "anonymous";
if (!_.any(Meteor.userIPMap, function(m) { m.userid === uid; })) {
Meteor.userIPMap.push({userid: uid, ip: req.connection.remoteAddress });
}
});
You'll then have a Meteor.userIPMap with a map of userids to ip addresses (to accommodate the x-forwarded-for header, use this function inside the above).
Three notes: (1) this will fire whenever there is a request in your app, so I'm not sure what kind of performance hit this will cause; (2) the __meteor_bootstrap__ object is going away soon I think with a forthcoming revamped package system; and (3) the anonymous user needs better handling here..you'll need a way to attach an anonymous user to an IP by a unique, persistent constraint in their request object.
You have to hook into the server sessions and grab the ip of the current user:
Meteor.userIP = function(uid) {
var k, ret, s, ss, _ref, _ref1, _ref2, _ref3;
ret = {};
if (uid != null) {
_ref = Meteor.default_server.sessions;
for (k in _ref) {
ss = _ref[k];
if (ss.userId === uid) {
s = ss;
}
}
if (s) {
ret.forwardedFor = ( _ref1 = s.socket) != null ?
( _ref2 = _ref1.headers) != null ?
_ref2['x-forwarded-for'] : void 0 : void 0;
ret.remoteAddress = ( _ref3 = s.socket) != null ?
_ref3.remoteAddress : void 0;
}
}
return ret.forwardedFor ? ret.forwardedFor : ret.remoteAddress;
};
Of course you will need the current user to be logged in. If you need it for anonymous users as well follow this post I wrote.
P.S. I know it's an old thread but it lacked a full answer or had code that no longer works.
Here's a way that has worked for me to get a client's IP address from anywhere on the server, without using additional packages. Working in Meteor 0.7 and should work in earlier versions as well.
On the client, get the socket URL (unique) and send it to the server. You can view the socket URL in the web console (under Network in Chrome and Safari).
socket_url = Meteor.default_connection._stream.socket._transport.url
Meteor.call('clientIP', socket_url)
Then, on the server, use the client's socket URL to find their IP in Meteor.server.sessions.
sr = socket_url.split('/')
socket_path = "/"+sr[sr.length-4]+"/"+sr[sr.length-3]+"/"+sr[sr.length-2]+"/"+sr[sr.length-1]
_.each(_.values(Meteor.server.sessions), (session) ->
if session.socket.url == socket_path
user_ip = session.socket.remoteAddress
)
user_ip now contains the connected client's IP address.