I have been building .Net Web API's for years... normally I have one API that has 10 or so different controllers who handle everything from signing users up, handling business logic, payment, etc. Those all talk to class libraries to talk to the database and such. Nothing fancy, but it has been effective.
Fast forward to today... I am building a version 2 for an app that gets a good amount of traffic. I know my app is gonna get hit hard so I am looking for something with a foundation of efficiency and scale.
This has led me to embrace the coolness of Service Fabric and ASP.Net Core Web APIs. I have been reading lots of tutorials, articles, and SO questions and from what I understand, the beauty of Service Fabric is that it will spawn up multiple nodes in a single VM when things get busy.
So, if I maintain my normal pattern and make a single Web API with 10+ controllers, can Service Fabric do what it needs to do? Or am I supposed to create multiple little API's that are more focused so that the Service Fabric can add/remove them as things get busy?
That sounds like the right thing to do, and I have set up my code to do just that by putting my Models and Data classes in their own class libraries so they can be reused by the different API's, but I just wanted to double check before I do something potentially stupid.
If I split up, say each controller into its own Service Fabric service, will the Azure server be more efficient and scale better?
Nodes
In Service Fabric clusters (on Azure / stand alone) a Node equals a VM. If you increase the amount of machines, more Nodes appear in the cluster. (This is not the case for your local dev cluster.) Scaling in Azure Clusters is simple: just change the VMSS instance count.
Only if you configure Stateless Services with instance count -1, Service Fabric will spawn new instances of it. This is caused by the addition of nodes, not by load itself.
You can configure autoscaling for VMSS'es.
Web API
Service Fabric just tries to balance the load of all running SF Services across the available resources. That could be one instance of one service type on every node, or multiple instances of many types. So one service can just use all the resources of the node it's running on, like with IIS. (This is why Container support is coming by the way.)
Web API design isn't directly influenced by Service Fabric. The same rules apply as when running on IIS or elsewhere. It's really your choice.
Microservices
Your normal pattern will work. But making smaller services from it could help reduce the impact of changes. (At the cost of increased complexity.) Consider creating services that offer common functionality following the Microservices paradigm.
In Microservices, your code changes are scoped to smaller modules, less testing is needed, performance is less degraded during updates. This way, in theory, you can release new features in less time.
It depends.
If you have a natural division in your controllers regarding the resources they use then you may get some benefit if you split your services along that division line. Say service A uses lots of CPU and service B uses mostly HTTP then giving SF the ability to split CPU loads on their own may mean fewer affected HTTP calls.
You can optimize how SF distributes load by reporting load from inside your app but do so in the simplest way possible and don't add numerous dimensions, maybe one per service at most.
If all your controllers use the same type of resources roughly the same then there's no real benefit to splitting them away in separate services, just complications in code management, deployments and potentially inter-service communications.
Related
We are designing a new system, which will be based on microservices.
I would like to consult, whether it is considered an anti-pattern to produce and consume from the same queue.
The service should be a REST-based microservice.
The backend microservice should handle large-scale operations on IoT devices, which may not always be connected to the internet, the API must be asynchronous and provide robust features such as the number of X retries before final failure, etc.
The API should immediately return a UUID with 201 responses from the POST request and expose 'Get Current Status' (pending, active, done.. etc.) of the operation status of the IoT device, by accepting the UUID in the get request.
We've thought about two solutions (described at a high level) for managing this task:
Implement both API GW Microservice and the logic handler Microservice.
The API GW will expose the REST API methods and will publish messages via RabbitMQ that will be consumed by multiple instances of the logic handler microservice.
The main drawback is that we will need to manage multiple deployments, and keep consistency between the APIs exposed in service A, to the consumer in service B.
Implement one microservice, that exposes the relevant APIs, and in order to handle the scale and the asynchronous operations, will publish the request to own queue over RabbitMQ, being consumed by the same microservice at a background worker.
The second solution looks easier to develop, maintain, and update because all the logic including the REST API's handling will take care of the same microservice.
To some members of the team, this solution looks a bit dirty, and we can't decide whether it is an anti-pattern to consume messages of your own queue.
Any advice will be helpful.
I would like to consult, whether it is considered an anti-pattern to produce and consume from the same queue.
That definitely sounds suspicious, not that I have a lot of experience with queues. But, you also talk about microservices and "producing" & consuming from those - that sounds fine, there's no reason why a microservice (and by extension, it's API) can't do both. But then I'm a bit confused because in reading the rest of the question I don't really see how that issue is a factor.
Having some kind of separation between the API and the microservice sounds wise, because you can change the microservices implementation without affecting callers, assuming it's a non-breaking change. It means you have the ability to solve API / consumer problems, and backend problems, separately.
Version control is just a part of good engineering practice, and probably not an ideal reason to bend your architecture. You should be able to run more than one version in parallel - a lot of API providers operate a N+2 model, where they support the current version, plus the last two (major) releases. That way you give your consumers a reasonable runway for upgrading.
As long as you keep the two concerns modular so you'd be able to separate them when it would make sense it doesn't matter.
I'd think in the longer term you'd probably want to treat them as two aspects of the same service as they'd probably have different update cycle (e.g. the gateway part may need things like auth, maybe additional api in gRPC, etc.),different security reqs (one can accessible to the outside where the other consumes internal resource) different scalability concerns (you'd probably need more resources for the processing) etc.
We have a process that involves loading a large block of data, applying some transformations to it, and then outputting what has changed. We currently run a web app where multiple instances of these large blocks of data are processed in the same CLR instance, and this leads to garbage collection thrashing and OOM errors.
We have proven that hosting some tracked state in a longer running process works perfectly to solve our main problem. The issue we now face is, as a stateful system, we need to host it and manage coordination with other parts of the system (also change tracking instances).
I'm evaluating Actors in Service Fabric and Akka at the moment, there are a number of other options, but before I proceed, I would like peoples thoughts on this approach with the following considerations:
We have a natural partition point in our system (Authority) which means we can divide our top level data set easily. Each partition will be represented by a top level instance that needs to organise a few sub-actors in its own local cluster, but we would expect a single host machine to be able to run multiple clusters.
Each Authority Cluster of actors would ideally be hosted together on a single machine to benefit from local communication and some use of shared local resources to get around limits on message size.
The actors themselves should be separate processes on the same box (Akka seems to run local Actors in the same CLR instance, which would crash everything on OOM - is this true?), this will enable me to spin up a process, run the transformation through it, emit the results and tear it down without impacting the other instances memory / GC. I appreciate hardware resource contention would still be a problem, but I expect this to be more memory than CPU intensive, so expect a RAM heavy box.
Because the data model is quite large, and the messages can contain either model fragments or changes to model fragments, it's difficult to work with immutability. We do not want to clone every message payload into internal state and apply it to the model, so ideally any actor solution used would enable us to work with the original message payload. This may cause problems with restoring an actor state as it wants to save and replay these on wakeup, but as we have state tracking internally, we can just store the resulting output of this on sleep.
We need a coordinator that can spin up instances of an Authority Cluster. There needs to be some elasticity in terms of the number of VM's/Machines and the number of Authority Clusters hosted on them, and something needs to handle creation and destruction of these.
We have a lot of .NET code, all our models, transformations and validation is defined in it, and will need to be heavily re-used. Whatever solution will need to support .Net
My questions then are:
While this feels like a good fit for Actors, I have reservations and wonder if there is something more appropriate? Everything I have tried has come back to a hosted processes of some kind.
If actors are the right way to go, which tech stack would put me closest to what I am trying to achieve with the above concerns taken into account?
IMO (coming at this from a JVM Akka perspective, thus why I changed the akka tag to akka.net; I don't have a great knowledge about the CLR side of things), there seems to be a mismatch between
We do not want to clone every message payload into internal state and apply it to the model, so ideally any actor solution used would enable us to work with the original message payload.
and
The actors themselves should be separate processes on the same box (Akka seems to run local Actors in the same CLR instance, which would crash everything on OOM - is this true?)
Assuming that you're talking about the same OS process, those are almost certainly mutually incompatible: exchanging messages strongly suggests serialization and is thus isomorphic to a copy operation. It's possible that something using shared memory between OS processes could work, but you may well have to make a choice about which is more important.
Likewise, the parent/child relationship in the "traditional" (Erlang/Akka) style actor model trivially gives you the local cluster of actors (which, since they're running in the same OS process allows the Akka optimization of not copying messages until you cross an OS process boundary), while "virtual actor" implementations as found in Service Fabric or Orleans (or, I'd argue Cloudstate or Lagom) basically assume distribution.
Semantically, the virtual actor models implicitly assume that actors are eternal (though their eternal essence may not always be incarnate). For your use-case, this doesn't necessarily seem to be the case.
I think a cluster of Akka.Net instances with sharded Authority actors spawning shorter-lived child actors best fits, assuming that you're getting OOM issues from trying to process multiple large blocks of data simultaneously. You would have to implement the instance scale-up/down logic yourself.
I have not worked with Akka.net so I can't speak to that at all, but I'd be happy to speak to what you're talking about in a Service Fabric context.
Service Fabric has no issue with the concept of running multiple clusters. In its terminology, the whole of your system would be called an Application and would have a version when deployed to the SF cluster. If you wanted to create multiple instances of it, all you'd need to do is select what you wanted to call the deployed app instance and it'll stand up provisioning for you.
SF has a notion of placement constraints, metric balancing and custom rules that you can utilize if you think you can better balance the various resources than its automatic balancing (or you need to for network DMZ purposes). While I've never personally grouped things down to a single machine, I frequently limit access of services to single VM scale sets (we host in Azure).
To the last point though, you'll still have message size limits, but you can also override them to some degree. In your project containing service interfaces, just set the following attribute above your namespace:
[assembly:FabricTransportRemotingSettings(MaxMessageSize=<(long)new size in bytes>)] and you're good to go.
Services can be configured to run using a Shared or Exclusive process model.
Regarding your state requirement, it's not necessarily clear to me what you're trying to do, but I think you're saying that that it's not critical that your actors store any state since they can work from some centrally-provided model.
You might look then at volatile state persistence then as it'll mean that state is saved for the actors in memory, but should you lose the replicas, nothing is written to disk so it's all lost. Or if you don't care and are ok just sending the model to the actors for any work, you can configure them to be stateless.
On the other hand, if you're still looking to retain state in the actors and simply are concerned about immutability, rest assured that actor state isn't immutable and can be updated trivially. There are simply order of operation concerns you need to keep in mind (e.g. if you retrieve the state, make a change, save it, 1) you must commit the transaction for it to take and 2) if you modify the state but don't save it, it'll obviously not persist - pull a fresh copy in a new transaction for any modifications). There's a whole pile of guidelines here.
Assuming your coordinator is intended to save some sort of state, might I recommend a singleton stateful service. Presumably it's not receiving an inordinate amount of use so a single instance is sufficient and it can easily save state (without the annoyance of identifying which state is on which partition). As for spinning up services, I covered this in the first bullet, but use the ApplicationManager on the built-in FabricClient to set up new applications and the ServiceManager to create instances of necessary services within each.
Service Fabric supports .NET Core 3.1 through .NET 5 as of the latest 8.0 release though note a minor serialization issues with an easy workaround with .NET 5.
If you have an Azure support subscription, I'd encourage you to write to the team under Development questions and share your concerns. Alternatively, on the third Thursday of each month at 10 AM PST, they also have a community call on Teams that you're welcome to join and you can find past calls here.
Again, I can't speak to whether this is a better fit than Akka.NET, but our stack is built atop Service Fabric. While it has some shortcomings (what framework doesn't?) it's an excellent platform for distributed software development.
I have been researching workflow foundation for a week or so now, but have been aware of it and the concepts and use cases for it for many years, just never had the chance to dedicate any time to going deeper.
We now have some projects where we would benifit from a centralized business logic exposed as services as these projects require many different interfaces on different platforms I can see the "Business Logic Silos" occuring.
I have had a play around with some proof of concepts to discover what is possible and how it can be achieved and I must say, its a bit of a fundamental phase shift for a regular C# developer.
There are 3 things that I want to achieve:
Runtime instanciated state machines
Customizable by the user (perform different tasks in different orders and have unique functions called between states).
WCF exposed
So I have gone down the route of testing state machine workflows, xamlx wcf services, appfabric hosted services with persistance and monitoring, loading xamlx services from the databse at runtime, etc, but all of these examples seem not to play nicely together. For example, a hosted state machine service, when in appfabric, has issues with the sequence of service method calls such as:
"Operation 'MethodName' on service instance with identifier 'efa6654f-9132-40d8-b8d1-5e611dd645b1' cannot be performed at this time. Please ensure that the operations are performed in the correct order and that the binding in use provides ordered delivery guarantees".
Also, if you call instancial workflow services at runtime from an sql store, they cannot be tracked in appfabric.
I would like to Thank Ron Jacobs for all of his very helpful Hands On Labs and blog posts.
Are there any examples out there that anyone knows of that will tie together all of these concepts?
Am I trying to do something that is not possible or am I attempting this in the right way?
Thanks for all your help and any comments that you can make to assist.
Nick
Regarding the error, it seems like you have modified the WF once deployed (is that #2 in your list?), hence the error you mention.
Versioning (or for this case, modifying a WF after it's been deployed) is something that will be improved in the coming version, but I don't think it will achieve what you need in #2 (if it is what I understood), as the same WF is used for every instance.
My project is converting a legacy fat-client desktop application into the web. The database is not changing as a result. Consequently, we are being forced to call external web services to access data in our own database. Couple this with the fact that some parts of our application are allowed to access the database directly through DAOs (a practice that is much faster and easier). The functionality we're supposed to call web services for are what has been deemed necessary for downstream, dependent systems.
Is this really how SOA is supposed to work? Admittedly, this is my first foray into the SOA world, but I have to think this is the complete wrong way to go about this.
I agree that it's the wrong approach. Calling your own database via a webservice should raise red flags in a design review, and a simple DAO is the way to go (KISS principle).
Now, if it's data that truly needs to be shared across your company (accounts, billing, etc) THEN it's time to consider a more heavy-duty solution such as SOAP or REST. But your team could still access it directly, which would be faster.
My team had the same thing happen with a web service that we wanted to call in batch mode. Rather than call our own SOAP endpoint, we instead set it up to call a POJO (plain old java object) interface. There's no XML transformation or extra network hop through an SOA appliance.
It's overkill to put an XML interface between MVC layers when your team owns the whole application. It may not be traditional SOA... but IMO it's traditional common sense. ;)
I've seen people try to jam SOA at too low a level and this may be such a case. I would certainly not equate DAO and SOA at the same level.
I agree with #ewernli
What is SOA "in plain english"?
IMHO, SOA makes sense only at the enterprise-level, and means nothing for a single application.
If I'm reading into your question correctly, your web services are for C/R/U/D data into the database. If so, providing C/R/U/D services directly to the database and its tables are likely too low level to be SOA services.
I'd look for services at a higher level and try to determine whether they are interesting at to the enterprise. If so, those are your services. I'd also ask myself whether my former desktop app is providing services (i.e. should you be looking to make your new app an SOA service itself rather than trying to force an SOA architecture into the desktop app at a low level.
Consequently, we are being forced to
call external web services to access
data in our own database.
Man, that gotta hurt. As far as services in SOA go,
a service is a repeatable logical manifestation of a business task - that means you are not implementing SOA if you are not 'service enabling' business processes. If you are putting some web services to select data out of your data base, all you got is a bunch of webservices, which would slowdown your applications which could have been faster by conventional data access patterns (like DAO)
When you equate SOA with Web services there is a risk of replacing existing APIs with Web services without proper architecture. This will result in identifying many services that are not business aligned.
Also, service orientation is a way of integrating a business as a group of linked services - so ask yourself is the organization making use of these atomic services to achieve further benefits?
Do a google search for SOA anti-patterns and you will find what are the different ways to end up with a pile of web-services instead of SOA.
SOA... SOA... is the bane of my existence, for just this reason. What, or what not, constitutes SOA? I support SOA products in my day job, and some people get it, some don't. SOA.. SOA is about wrapping discrete business services in XML. ZIP+4 validation services. Payment gateways. B2B messaging.
SOA CAN be used to decouple desktop apps from backend databases. Sometimes it doesn't make sense, sometimes it does. What almost NEVER makes sense is low-latency high-query-count logic. If you ever have to use an application in France directly connected to a database in California, you'll get what I mean. SOA pretty much forces you to then smartly about how you model and return your data (look into SDO - Service Data Objects). The devil's in the details though. Marshalling data to/from XML can be costly.
Good SOA design is all about separation of behavior and data.
I repeat behavior and data need to be separate or else you will have lots or problems whether its CORBA/SOAP/REST/XMLRPC or even plain old in-the-same-JVM-method calls.
Lots of people will talk about service end points, message handling, and contracts making SOA one of the more soporific areas of computing when its surprisingly not complicated.
If you are doing Java its really easy. Make POJOs for your domain objects with no weird state behavior and no weird collaborators and then make Service classes with the behavior. More often then not you can just use your DAO as the service (I mean you should have a thin layer over the DAO but if you don't need one....).
OOP lovers will disagree of this separation of data and behavior but this design pattern scales extremely well and is infact what most functional programming languages like Erlang do.
That being said if you are making a video game or something very state based then this design philosophy is a bad idea. BTW SOA is about as vacuous as the term enterprise.
Which part do you think is wrong? The part that you have to hit the web service, or the part you are hitting the database directly?
SOA is more of an API design guideline, not a development methodology. It's not an easy thing to implement, but the reward of reusability is often worth it.
See Service-Oriented Architecture expands the vision of Web services or any technical book on SOA. Simply wrapping function calls with web call does not make it a Service Oriented Architecture. The idea of the SOA is to make reusable services, and then you make higher level services (like website) by compositing or orchestrating underlying low-level services. At the very low level, you should focus on things like statelessness, loose coupling, and granularity. Modern frameworks like Microsoft's WCF supports wiring protocols like SOAP, REST, and faster binary side by side.
If your application is designed to run over the Internet, you should be mindful of the network latency issues. In a traditional client-server application that is deployed on a LAN, because the latency is sub 10 msec, you could hit the database every time you need the data without interrupting the user experience. However, on the Internet, it is not uncommon to have 200 msec latency if you go across proxies or oceans. If you hit the database 100 times, and that will add up to 20 seconds of pause. In SOA, you would try to pack the whole thing into a single document, and you exchange the document back and forth, similar to the way tax is filed using Form 1040 if you live in the US.
You may say that the latency issue is irrelevant because the web service is only consumed by your web application layer. But you could hit the web service from the browser using AJAX reload the data, which should give the user shorter response time.
Maybe I just expected "three-tier architecture" to deliver a little more than just a clean separation of responsibilities in the source code (see here)...
My expectations to such a beast that can safely call its self "three-tier architecture" are a lot higher... so, here they are:
If you were to build something like a "three tier architecture" system but this time with these, additional requirements and constraints:
Up and running at all times from a Users point of viewExpect when the UI gets replacedWhen other parts of the system are down, the UI has to handle that
Never get into a undefined state or one from which the system cannot recover automatically
The system has to be "pausable"
The middle-tier has to contain all the business logic
Obviously using an underlying Database, itself in the data-tier (if you like)
The business logic can use a big array of core services (here in the data-tier, not directly accessible by the UI, only through business logic tier facade)
Can be unavailable at times
Can be available as many parallel running, identical processes
The UI's may not contain any state other than the session in case of web UI's and possibly transient view baking models
Presentation-tier, logic-tier and data/core-services-tier have to be scalable independently
The only thing you can take for granted is the network
Note: The mentioned "core services" are heavy-weight components that access various external systems within the enterprise. An example would be the connection to an Active Directory or to a "stock market ticker"...
1. How would you do it?
If you don't have an answer right now, maybe read on and let me know what you think about this:
Sync considered harmful. Ties your system together in a bad way (Think: "weakest link"). Thread blocked while waiting for timeout. Not easy to recover from.
Use asynchronous messaging for all inter-process communication (between all tiers). Allows to suspend the system anytime you like. When part of the system is down, no timeout happens.
Have central routing component where all requests get routed through and core services can register themselves.
Add heartbeat component that can e.g. inform the UI that a component is not currently available.
State is a necessary evil: Allow no state other than in the business logic tier. This way the beast becomes manageable. While the core services might well need to access data themselves, all that data should be fed in by the calling middle tier. This way the core services can be implemented in a fire and forget fashion.
2. What do you think about this "solution"?
I think that, in the real world, high-availability systems are implemented using fail-over: for example, it isn't that the UI can continue to work without the business layer, instead it's that if the business layer becomes unavailable then the UI fails over to using a backup instance of the business layer.
Apart from that, they might operate using store-and-forward: e.g. a mail system might store a piece of mail, and retransmit it periodically, if it can't deliver it immediately.
Yep its the way most large websites do it. Look at nosql databases, Google's bigtable architecture etc.
1. This is the general approach I'd take.
I'd use a mixture of memcached , a nosql-cloud (couch-db or mongo-db) and enterprise grade RDBMS systems (core data storage) for the data layer. I'd then write the service layer ontop of the data layer. nosql database API's are massively parallel (look at couchdb with its ngingx service layer parallizer). I'd then provide "oldschool each request is a web-page" generating web-servers and also direct access to the service layer for new style AJAX application; both these would depend on the service layer.
p.s. the RDBMS is an important component here, it holds the authoritative copy of the all the data in the memchached/nosql cloud. I would use an enterprise grade RDBMS to do data-centre to data-centre replication. I don't know how the big boys do their cloud based site replication, it would scare me if they did data-cloud to data-cloud replication :P
Some points:
yYu do not need heartbeat, with nosql
the approach taken is that if content
becomes unavailable, you regenerate it
onto another server using the
authoratitve copy of the data.
The burden of state-less web-design
is carried to the nosql and memcached
layer which is infinitely scalable.
So you do not need to worry about
this. Just have a good network
infrastructure.
In terms of sync, when you are
talking to the RDBMS you can expect
acceptable synchronous response
times. Your cloud you should treat as
an asynchronous resource, you will
get help from the API's that
interface with your cloud so you
don't even have to think about this.
Advice I can give about networking
and redundancy is this: do not go for
fancy Ethernet bonding, as its not worth
it -- things always go wrong. Just
set up redundant switches, ethernet cards
and have multiple routes to all your
machines. You can use OpenBSD and
CARP for your routers, as they work
great - routers are your worst point of failure -- openbsd solves this problem.
2. You've described the general components of a web 2.0 farm, so no comment:D