As per the official documentation of AsyncAPI Specification to define event-driven APIs, there would be the need to define the basic components like info, server, channel, message, schema section using the specifications.
This is good enough for most use cases where messages are sent to the broker and consumed by one subscriber.
In case of AWS SNS, the message sent to the SNS topic will be consumed by many consumers as per the filtering criteria given during the subscription of the SNS topic.
How to define all those filtering criteria using the AsyncAPI Specification? Do we need to specify all the different possible examples of message payload to map each consumers specific payloads? Or is there any other better way to define this routing by SNS topic?
There is a pull request for these bindings. I suggest that you add your requirements and questions there - https://github.com/asyncapi/bindings/pull/84
Related
I am building a microservice application, and I try to follow the best practices. I use event sourcing and event driven state transfer in many places, but I realized that sometimes I just need to call another service in an asynchronous way to kindly ask it to do something e.g. send out a registration email (as the email service is a technical component and not a domain). I noticed that many times, services just call the other's service API endpoint, but that wouldn't be asynchronous. As I don't expect any returned value when calling from another service, the command would only produce events, RPC is not necessary.
In the end, my plan is to implement commands/actions that can be triggered by clients from REST API (then the commands may also produce responses) or by events or other services from RabbitMQ or similar. This leads me to how should I define the data structure of the command/action, is there any specification for that? Or existing solutions for Python? Or should I do something differently?
We have a simple application, who upon every update of an entity sends out a notification to SNS(it could very well have been any other queuing system). Clients are listening to these notifications and they do a get of updated entity based on these notifications.
The problem we are facing is, when clients do a get, sometimes data is not replicated and we return 404 or sometimes stale data(even worse).
How can we mitigate this while sending notifications?
Here are Few strategies to mitigate this with pros and cons
Instead of sending notification from application send notification using database streams
For example dynamodb streams ans aws lambda. This pattern can be useful in the case of multiregion deployment as well. where all the subscriber, publisher will subscribe to their regional database streams. And also atomicity of sending message and writing to database is preserved. And we wont loose events in the case of regional failure.
Send delayed messages to your broker
Some borkers like activemq and sqs support this functionality, but SNS does not. A workaround for that could be writing to sqs queue which then writes to sns. This might be a good option when your database does not support streams.
Send special error code for retry-able gets
Since we know that eventual consistency is there we can return special error code to clients, so that they can retry based on this error code. The retry strategy should be exponential backoff. but this may mean giving away your problems to clients. Also we should have some sort of versioning in place.
Fetch from another region
If entity is not found in the same region application can go to another region or master database to fetch it. NOTE Don't do this. as it is an anti pattern. I am mentioning it here just for the sake of completion.
Send the full entity in message
If entities to be fetched by rest service is small and there are no security constrain around who can access what, we can send the full entity in message. This is ensure that client don't have to do explicit fetch of it every time a new message is arrived.
I am sending fcm notifications.
Now I want to register user to multiple topics in one request, how can I achieve that?
I have read the documentation but I have not found any documentation for that?
Also what is the limit of number of topics for a single device after which it starts displaying TOO_MANY_TOPICS errors?
Usually there should be no limits for subscriptions and topics (see https://firebase.google.com/docs/cloud-messaging/android/topic-messaging). I personally go with a foreach loop if there are 20-30 topics to send for one user.
There is no API to subscribe to multiple topics in one go. You'll have to call the API separately for each topic. As the documentation says, this call needs to be made from a server or otherwise trusted environment.
I'd take this approach up to a few dozen topics. If you need more than that, consider creating some broader topics. For example: many apps have a topic called all that they also send all messages to. That way people that want all messages, can subscribe to the all topic, instead of to each individual topic.
I am looking in to how to use API gateway in my project. My current API gateway pattern idea is to create a layer of composite APIs calling multiple APIs, and expose the composite APIs through the API gateway.
Researching elsewhere, I've found that it is not recommended to create composite APIs if using an API gateway, but rather that composition should happen in the API gateway itself. But if I need to do the composition in API gateway, doesn't that mean that I need to expose some domain logic there, because would I need to construct a meaningful request contract and possibly a sequence of calling APIs? Personally, I don't feel comfortable doing that.
Is there other way to do this without exposing some logic?
From what I research, it is not recommended to create composite API if using API gateway but composition should happen in the API gateway itself.
could you refer where did you find the statement?
IMHO the service composition is not task of the api gateway. It's task of n integration layer (you don't need to have a separate esb product or service, under integration lyer you can understand any services or service layer implementing the capabilities, such as composition, transformation, etc)
You did not specify any api product or service, it depends on the product if it is capable to execute any logic or not (some do, some some don't). Even if the api gateway product is capable of executing any logic, indeed I don't recommend it either (you will have to fight with maintenance, error and state handling,..)
But if I need to do the composition in API gateway, doesn't it means that I need to expose some domain logic there
(personally, I don't feel comfortable doing that) because I need to construct a meaningful request contract and possibly sequence of calling APIs?
what do you mean by composition in API gateway? I assume under the term executing some logic. Then the logic is hidden from the api clients.
Or could it be layer of composite API sounds better?
Indeed.
Generally try to treat an api gateway as a smart proxy (doing authorization, throttling, client facing api store,..) but the exposed backend services should be already exposed as they are
I'm trying to find out if there are any limitations for the topic names for FCM. Managed to find info about the number of topics (no limitations), but nothing like length of topic name or allowed characters.
Yes, not all characters are allowed, having space between words is not allowed in topic names.
From the docs:
sending messages to a Firebase Cloud Messaging topic is very similar to sending messages to an individual device or to a user group. The app server sets the topic key in the message body with a value like yourTopic. Developers can choose any topic name that matches the regular expression: "[a-zA-Z0-9-_.~%]+"
for more info check this: https://firebase.google.com/docs/cloud-messaging/android/topic-messaging (build request section)
Topic messaging supports unlimited subscriptions for each topic. However, FCM enforces limits in these areas:
One app instance can be subscribed to no more than 2000 topics.
If you are using batch import to subscribe app instances, each request is limited to 1000 app instances.
The frequency of new subscriptions is rate-limited per project. If you send too many subscription requests in a short period of time, FCM servers will respond with a 429 RESOURCE_EXHAUSTED ("quota exceeded") response. Retry with exponential backoff.
Yes there are limitations on FCM topic names.
FCM topic must match the following regular expression: [a-zA-Z0-9-_.~%]{1,900}.