We have a requirement wherein we will have to periodically monitor stream of data and include them in particular buckets. What will be the best language or tool to implement such requirement ?
you can use Kafka
"Kafka™ is used for building real-time data pipelines and streaming apps"
https://kafka.apache.org/
You could use scala if you have very complicated pattern, but if it's simple, it's a perfect match to use a message broker like RabbitMQ.
Based on my humble experience I would reccomend nodejs, it works nicely with sockets and streams, you can implement thing fast, and performance are good too (async I/O). If you are not familiar with it think of it as a js engine with socket/HTTP support and async I/O. Performnces are really good. If you need really fast or peculiar things you can even extend with C++, but I think it's not necessary. Give a look https://nodejs.org/en/
Related
I would like to know some popular frameworks that are available for implementing CQRS, ES, Saga in the application.
As a part of my research, I have to compare these frameworks and evaluate them based on various -ilities.
I have to compare these [event-sourcing] frameworks and evaluate them based on various -ilities.
The premise of the question is that you need a framework to implement event sourcing but, in fact, you do not.
Greg Young, one of the most influential proponents of event sourcing, frequently expresses his misgivings about frameworks. See, for instance, his QCon London 2013 keynote, esp. mark 9'.
Event sourcing is conceptually simple and doesn't need the kind of magic that frameworks typically bring with them. For instance, rebuilding the state from a stream of events simply consists in a left fold over the stream in question. Moreover, you don't necessarily need a specialised database; I know people who have successfully implemented event sourcing by simply appending events to a file.
If your research aims at comparing event-sourcing frameworks, I would argue that you should consider the case where no framework is used at all.
Axon is a popular framework/server for building CQRS/ES applications.
EventStoreDB is a popular EventStore database for the EventSourcing part.
A simple starting point if you want to write your own framework/library is to check out some of the code I co-authored at https://www.cqrs.nu/
If you are looking for a managed solution, you can also check out what we at Serialized provide.
In addition to Axon, on the JVM there's also the Akka ecosystem (the cluster sharding, persistence, sharded daemon process, and projection modules are the most relevant to CQRS/ES/DDD). One benefit of Akka Persistence is the ability to choose from a variety of datastores to use as an event store (JDBC SQL databases and Cassandra are the most common, but there are many more supported). My experience with it has been that it is capable of exceptionally high availability and since it allows a stateful event-sourced application to be deployed as if it's stateless (e.g. in Kubernetes without needing an operator) there's a lot of deployment flexibility. Note that because it's built on the actor model, a lot of JVM observability tooling doesn't work particularly well with it (often assuming a stronger mapping of threads to tasks), so certain commercially-licensed observability tooling is recommended.
Additionally, Kalix also provides a polyglot (all you need is to express domain logic in a language which supports grpc) event-sourcing implementation.
Disclaimer: since answering this question (almost a year after answering this question), I became employed by Lightbend, the maintainers of Akka and provider of Kalix.
I am working on a project and implementing the search functionality.
I have a text box and there will be an auto suggestion implemented.
I have two ways to go.
Make a single call to the DB and filter the list of the auto-suggest or.
Make multiple calls in the DB and update the auto-suggest list using ajax
what is the best solution performance-wise and why?
It depends on how "heavy" both approaches are from the database perspective and how fast should auto-suggest response be. Well-behaved application built on connection pool pattern should not take too many resources for 2nd approach, however, this way network traffic and latency come into play. On the other hand, 1st approach might take more resources.
So I would recommend testing it out in real conditions using a load testing tool like Apache JMeter, producing the same load against 2 implementations, and measuring which one works faster and consumes fewer resources. See The Real Secret to Building a Database Test Plan With JMeter to get familiarized with the databases load testing concept.
I'm writing a custom email sending service for a client. The client also wants message templating as well, but they didn't specify whether or not they wanted it in the messaging service or not. So, I'm thinking about best practice, here. Should a messaging service be responsible for templating as well? Or should the templating happen before the call to the messaging service? What have you done? What works better and makes the most sense?
This is easy to answer with a question: Are you going to use the messaging service for sending all kinds of messages (with or without templates) or just templated ones? (a.k.a reusability of messaging service's functionality).
You mentioned two solutions in your question. Lets call them solution A and solution B.
Since clients constantly change their mind, you might have to later change whichever solution you adopted. Your implementation must be easy to change later on, so you can chose which one to implement like this:
think that you have implemented solution A and you have to change it into B. How hard will it be and what will it involve? Let's call this result 1;
think that you have implemented solution B and you have to change it into A. How hard will it be and what will it involve? Let's call this result 2;
compare result 1 and 2 with pros and cons.
Choose the one with the most pros.
You could also opt for a solution C. Make the messaging service send all kind of messages (generic) and include loosely coupled plugable templating (more specific). Package them together and you get a specific tool that you can later split with ease or add more templating implementations to it if needed.
Just my 2 cents!
I'm working on an app which will, like most apps, have a whole boat load of buisness logic, almost all of which will need to be executed both on the server and the Flash-based client… And I'm trying to figure out the best (read: least complex) way to implement the rules engine.
These are the parameters of the problem:
The rules engine must both run in a web browser (ie, in Flash Player) and on the server. Duplicating the logic (eg, by writing a "server" version and a "client" version) would be an unacceptable risk.
The input/output data is fairly complex, so serialization is a nontrivial problem. We are currently using AMF for all of our serialization needs, and using another protocol would add significant complexity… So it should probably be avoided.
It is infeasible to implement a "rules description language". Experimentation has shown that rules are sufficiently complex that any such language would need to be Turing complete… Which would also add a significant amount of complexity.
The rules engine will not need to make some, but not very many, service calls.
Currently, the best contenders are:
Writing the code in ActionScript, then running it on the server. In theory it's possible to start up an AVM instance, get it long-polling a gateway, then pass data back and forth that way… But that seems less than ideal. Is there a "good" way of doing this?
Writing the code in Haxe. I don't know anything about Haxe's AMF support, so that could be a deal-breaker.
Something involving Tamarin. Seems like a viable option, but I haven't done enough research to tell either way.
So, what do you think? Are any of these options clearly better than others? Is there something I haven't though of that's worth considering?
Finally, thanks for reading this wall of text :)
How much data are you talking about? You can use Air if you want to run it on the server and access a queue or something.
I am planning to create a site that provides several games for self written game-agents.
There will be a Tic-Tac-Toe for example (and some more complex games of course). Programmers can register their agents and from time to time there will be a tournament. The gameserver will contact the registered agents, present the board and ask for a move.
My basic problem is: What would be a good way for the gameserver to communicate with the agents?
No communication: A special script language for the agents that can be submitted and interpreted on the server itself. This avoids the problem altogether, but it would be more fun if everyone could use his favorite language.
REST: This is a bit sparse in one direction.
SOAP?
REST with POST?
What connection-method would you like, if you would participate in such a game?
I'd use a form of REST supported by sever state, SOAP is too inhuman. If you use REST people can even make bots with cUrl, so you'll have the widest audience
I had great success using Twisted and developing my own application protocol built off of that.
I would use a private protocol, that leaves you the option to implement server and clients in different languages. You can add your favorite encryption mode as well.
The scripting language would definitely be the easiest thing to write on your end. Personally, as long as you specify the protocol exactly on the site, and give an example of formatting the messages, any person scripting should be able to use whatever format you choose.
For Go, there is already the existing Go Text Protocol. I would use similar protocols for other games.
Plain old TCP with a set of custom commands for your game?