I have started using the new AWS Version 3 sdk for some of my services. Unfortunately it is not always clear how to use some features in the modular version 3 code that are available in the Version 2 sdk.
To set timeouts for the non-modular sdk, you can do the following:
AWS.config.update({
httpOptions: {
connectTimeout: 10000,
timeout: 10000
}
});
However, when I want to use the Version 3 sdk and use the Dynamo client, I am explicitly trying not to use the global AWS object. As far as I can tell the configuration input to DynamoDBClient does not accept httpOptions, which is where a timeout would normally get set.
import { DynamoDBClient } from "#aws-sdk/client-dynamodb";
const REGION = process.env.AWS_REGION;
const v3DynamoClient: DynamoDBClient = new DynamoDBClient({ region: REGION });
How do I set a timeout for the DynamoDBClient in the AWS V3 sdk?
This link about upgrading seems to have an example, in the section about httpOptions. There is an obvious typo in the example there. Looking at the new AWS v3 code directly, it is apparent that the agent NEED NOT BE SPECIFIED (there are defaults if the agent is not passed) - so this example is sufficient.
// Use default Https agent, but override the socket timeout
const requestHandler = new NodeHttpHandler({
connectionTimeout: 30000,
socketTimeout: 30000,
});
const options = {
region: AWS_REGION,
maxAttempts: 2,
requestHandler, // Use handler with alternate settings for timeouts
};
export const dynamodbClient = new DynamoDBClient(options);
Here's an example of setting TLS v1.2 options that should help:
const https = require("https");
const {NodeHttpHandler} = require("#aws-sdk/node-http-handler");
const {DynamoDBClient} = require("#aws-sdk/client-dynamodb");
const client = new DynamoDBClient({
region: "us-west-2",
requestHandler: new NodeHttpHandler({
httpsAgent: new https.Agent({secureProtocol: 'TLSv1_2_method'})
})
});
You should be able to set connectionTimeout or socketTimeout in the options to NodeHttpHandler.
Also, worth reading the SDK v3 Developer Guide.
Related
Trying to figure out how to modify request headers for the graph client toolkit and have it apply when using the GET component. I don't necessarily want to override the entire graph client.
For reference, here's how you do it:
// Already likely on page, adding as reference
TeamsProvider.microsoftTeamsLib = microsoftTeams;
const provider = new TeamsProvider(config);
const options = {
authProvider: provider,
fetchOptions: { headers: {'ConsistencyLevel':'eventual'}}
};
const client = Client.initWithMiddleware(options);
provider.graph = new Graph(client)
// Now set provider with new graph
Providers.globalProvider = provider
I have gotten this to work without having to re-initialize the graph client by doing this:
Providers.globalProvider.graph.client.config.fetchOptions = { headers: {'ConsistencyLevel':'eventual'}}
You can also remove the request headers by setting fetchOptions to {}. This can be toggled at any time between requests, and can be set even after the globalProvider has been initialized without having to re-create the middleware chain.
I posted a comment in the issues section of the MGT repo here.
I've been trying to get NServiceBus.Router working to allow endpoints using the AmazonSQS transport and the AzureServiceBus transport to communicate with each other. So far, I am able to get a command sent from the ASB endpoint through the router and handled by the SQS endpoint. However, when I publish an event from the SQS endpoint, it is not handled by the ASB endpoint even though I have registered the SQS endpoint as a publisher. I have no idea what I'm doing wrong, but looking at every example I can find from from the docs, it seems like it should work.
I have already tried adding another forwarding route in the reverse of what is below (SQS to ASB), but that did not solve the issue.
The endpoints and router are each running in .net 5 worker services.
I've made a sample project that reproduces the issue here, but here are some quick at-a-glance snippets that show the relevant setup:
Router Setup
var routerConfig = new RouterConfiguration("ASBToSQS.Router");
var azureInterface = routerConfig.AddInterface<AzureServiceBusTransport>("ASB", t =>
{
t.ConnectionString(Environment.GetEnvironmentVariable("ASB_CONNECTION_STRING"));
t.Transactions(TransportTransactionMode.ReceiveOnly);
t.SubscriptionRuleNamingConvention((entityType) =>
{
var entityPathOrName = entityType.Name;
if (entityPathOrName.Length >= 50)
{
return entityPathOrName.Split('.').Last();
}
return entityPathOrName;
});
});
var sqsInterface = routerConfig.AddInterface<SqsTransport>("SQS", t =>
{
t.UnrestrictedDurationDelayedDelivery();
t.Transactions(TransportTransactionMode.ReceiveOnly);
var settings = t.GetSettings();
// Avoids a missing setting error
//https://github.com/SzymonPobiega/NServiceBus.Raw/blob/master/src/AcceptanceTests.SQS/Helper.cs#L18
bool isMessageType(Type t) => true;
var ctor = typeof(MessageMetadataRegistry).GetConstructor(
BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance, null,
new[] {typeof(Func<Type, bool>)}, null);
#pragma warning disable CS0618 // Type or member is obsolete
settings.Set<MessageMetadataRegistry>(ctor.Invoke(new object[] {(Func<Type, bool>) isMessageType}));
#pragma warning restore CS0618 // Type or member is obsolete
});
var staticRouting = routerConfig.UseStaticRoutingProtocol();
staticRouting.AddForwardRoute("ASB", "SQS");
routerConfig.AutoCreateQueues();
ASB Endpoint Setup
var endpointConfiguration = new EndpointConfiguration("ASBToSQSRouter.ASBEndpoint");
var transport = endpointConfiguration.UseTransport<AzureServiceBusTransport>();
transport.SubscriptionRuleNamingConvention((entityType) =>
{
var entityPathOrName = entityType.Name;
if (entityPathOrName.Length >= 50)
{
return entityPathOrName.Split('.').Last();
}
return entityPathOrName;
});
transport.Transactions(TransportTransactionMode.ReceiveOnly);
transport.ConnectionString(Environment.GetEnvironmentVariable("ASB_CONNECTION_STRING"));
var bridge = transport.Routing().ConnectToRouter("ASBToSQS.Router");
bridge.RouteToEndpoint(typeof(ASBToSQSCommand), "ASBToSQSRouter.SQSEndpoint");
bridge.RegisterPublisher(typeof(ASBToSQSEvent), "ASBToSQSRouter.SQSEndpoint");
endpointConfiguration.EnableInstallers();
SQS Endpoint Setup (nothing special because it doesn't need to know about the router)
var endpointConfiguration = new EndpointConfiguration("ASBToSQSRouter.SQSEndpoint");
var transport = endpointConfiguration.UseTransport<SqsTransport>();
transport.UnrestrictedDurationDelayedDelivery();
transport.Transactions(TransportTransactionMode.ReceiveOnly);
endpointConfiguration.EnableInstallers();
Any help would be greatly appreciated!
Unfortunately one of the recent SQS transport releases contains a change that makes the subscription work only by default in the context of a full NServiceBus endpoint. This feature is subscription batching.
In order for the Router to work correctly (Router does not run a full endpoint, just NServiceBus transport), you need to add this magic line to the SQS interface configuration:
settings.Set("NServiceBus.AmazonSQS.DisableSubscribeBatchingOnStart", true);
This is an undocumented flag that disables the subscription batching and allows router to complete the subscribe operations normally.
I am sorry for the inconvenience.
I'm trying use Telegraf library with Firebase Functions but it's not working as I expected.
I follow these this article and instructions as appear in webhooks (as appears for express example) and webhookcallback as appear in telegraf docs.
const Telegraf = require('telegraf')
// The Cloud Functions for Firebase SDK to create Cloud Functions and setup triggers.
const functions = require('firebase-functions')
// The Firebase Admin SDK to access the Firebase Realtime or Firestore Database.
const admin = require('firebase-admin')
// set telegraf and responses.
const BOT_TOKEN = 'my-telegram-bot-token'
const bot = new Telegraf(BOT_TOKEN)
bot.start((ctx) => ctx.reply("Start instructions"))
bot.help((ctx) => ctx.reply("This is help"))
bot.hears('hi', (ctx) => ctx.reply('Hola'))
bot.on('text', (ctx) => ctx.reply('Response to any text'))
bot.catch((err, ctx) => {
console.log(`Ooops, ecountered an error for ${ctx.updateType}`, err)
})
// initialize bot
bot.launch() // <-- (2)
//appends middleware
exports.ideas2coolBot = functions.https.onRequest(bot.webhookCallback(`/my-path`));
In firebase server I need add bot.launch() (2) to get worked, but it works just for max timeout set in Firebase Function. I need to recall Telegram "setWebhook" API to get work again and it works for the same time. It's like it's generate one function instance and shut down when time is over.
I noted the telegraf.launch() have options to start in poll or webhook mode but its not pretty clear for me how to use this options.
How should I use telegram.launch() to get worked in webhook mode in Firebase?
Edit:
When I used getWebhookInfo I get this result:
{
"ok": true,
"result": {
"url": "https://0dbee201.ngrok.io/test-app-project/us-central1/testAppFunction/bot",
"has_custom_certificate": false,
"pending_update_count": 7,
"last_error_date": 1573053003,
"last_error_message": "Read timeout expired",
"max_connections": 40
}
}
and console shows incoming conection but do nothing...
i functions: Beginning execution of "ideas2coolBot"
i functions: Finished "ideas2coolBot" in ~1s
Edit2:
I've been trying adding Express too...
app.use(bot.webhookCallback('/bot'))
app.get('/', (req, res) => {
res.send('Hello World from Firebase!')
})
exports.ideas2coolBot = functions.https.onRequest(app);
it's works '/' path but got nothing with '/bot'. POST to '/bot' not response.
By the way, I tried a express standalone version and works prefect, but using it with firebase doesn't respond ("Read timeout expired").
delete
bot.launch()
try add this
exports.YOURFUNCTIONNAME = functions.https.onRequest(
(req, res) => bot.handleUpdate(req.body, res)
)
then set ur webhook manually
https://api.telegram.org/bot{BOTTOKEN}/setWebhook?url={FIREBASE FUNCTION URL}'
i followed the sample of authorized-https-endpoint and only added console.log to print the req.cookies, the problem is the cookies are always empty {} I set the cookies using client JS calls and they do save but from some reason, I can't get them on the server side.
here is the full code of index.js, it's exactly the same as the sample:
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const express = require('express');
const cookieParser = require('cookie-parser')();
const cors = require('cors')({origin: true});
const app = express();
const validateFirebaseIdToken = (req, res, next) => {
console.log(req.cookies); //// <----- issue this is empty {} why??
next();
};
app.use(cors);
app.use(cookieParser);
app.use(validateFirebaseIdToken);
app.get('/hello', (req, res) => {
res.send(`Hello!!`);
});
exports.app = functions.https.onRequest(app);
store cookie:
curl http://FUNCTION_URL/hello --cookie "__session=bar" // req.cookies =
{__session: bar}
doesn't store:
curl http://FUNCTION_URL/hello --cookie "foo=bar" // req.cookies =
{}
If you are using Firebase Hosting + Cloud Functions, __session is the only cookie you can store, by design. This is necessary for us to be able to efficiently cache content on the CDN -- we strip all cookies from the request other than __session. This should be documented but doesn't appear to be (oops!). We'll update documentation to reflect this limitation.
Also, you need to set Cache-Control Header as private
res.setHeader('Cache-Control', 'private');
Wow this cost me 2 days of debugging. It is documented (under Hosting > Serve dynamic content and host microservices > Manage cache behavior, but not in a place that I found to be useful -- it is at the very bottom "Using Cookies"). The sample code on Manage Session Cookies they provide uses the cookie name session instead of __session which, in my case, is what caused this problem for me.
Not sure if this is specific to Express.js served via cloud functions only, but that was my use case. The most frustrating part was that when testing locally using firebase serve caching doesn't factor in so it worked just fine.
Instead of trying req.cookies, use req.headers.cookie. You will have to handle the cookie string manually, but at least you don't need to implement express cookie parser, if that's a problem to you.
Is the above answer and naming convention still valid? I can't seem to pass any cookie, to include a session cookie named "__session", to a cloud function.
I setup a simple test function, with the proper firebase rewrite rules:
export const test = functions.https.onRequest((request, response) => {
if (request.cookies) {
response.status(200).send(`cookies: ${request.cookies}`);
} else {
response.status(200).send('no cookies');
}
});
The function gets called every time I access https://www.xxxcustomdomainxxx.com/test, but request.cookies is always undefined and thus 'no cookies' is returned.
For example, the following always returns 'no cookies':
curl https://www.xxxcustomdomainxxx.com/test --cookie "__session=testing"
I get the same behavior using the browser, even after verifying a session cookie named __session was properly set via my authentication endpoint. Further, the link cited above (https://firebase.google.com/docs/hosting/functions#using_cookies) no longer specifies anything about cookies or naming conventions.
This is for all Node.js versions 6+
Say I have currently have a TCP server with multiple clients:
const server = net.createServer(s => {
});
server.listen(6000);
and I connect to it with clients:
const s1 = net.createConnection({port:6000});
const s2 = net.createConnection({port:6000});
const s3 = net.createConnection({port:6000});
TCP can sometimes be a bit slow on a local machine. I hear that there might be a way to substitute a host/port combination with Unix Domain Sockets, but maintain the TCP server style interface. Is this possible and how?
The Node.js docs mention you can create a server that listens on a path:
https://nodejs.org/api/net.html#net_server_listen_path_backlog_callback
but it doesn't specify what kind of file that needs to be and how to create that file.
It turns out on MacOS/Linux this is easy. You don't need to create the file. You need to make sure the file does not exist, and then point the Node.js core libraries to an empty path.
For the server:
const udsPath = path.resolve('some-path.sock');
const wss = net.createServer(s => {
});
wss.listen(udsPath, () => {
});
For the clients:
const udsPath = path.resolve('some-path.sock'); // same file path as above
const ws = net.createConnection(udsPath, () => {
});