Background
Our clients are using some proxy endpoints which call a Target endpoint using the Service Callout policy.
I am using Lookup Cache and populate Cache policies before and after a Service Callout policy.
Cache expires in 1 hour. The Service Callout policy makes the target endpoint request ( takes ~ 500msec ) when there is a cache miss otherwise the Lookup Cache policy returns from cache ( a cache hit within the hour ).
Scope of the is global.
Key space is not huge. May be around 200 unique cache keys.
Problem
I am trying to reduce response times for our clients. One way to possibly achieve this - is by eliminating the call to the target endpoint ( via Service Callout policy ) when our clients call these proxy endpoints. Basically always resulting in a cache hit.
Possible Solution?
Since its scope is global, I could then write a script ( batch hourly cron job ) which force refreshes this global cache for each . This will always trigger a cache hit for our clients.
Do you think this solution will work? Comments? Better alternatives?
Prefetch the cache for all possible cache entries by writing a batch script to hit the APIs (As you have mentioned). This does work well.
A step improvement can be made by using API-DN setup. Where your APIs would be deployed across multiple geographies. Your client would be served from the closest server to their location. As Apigee uses distributed cache there is no need to populate cache for each Message Processor node.
Related
I'm using the api http://exchangeratesapi.io/ to get exchange rates.
Their site asks:
Please cache results whenever possible this will allow us to keep the service without any rate limits or api key requirements.
-source
Then I found this:
By default, the responses all of the requests to the exchangeratesapi.io API are cached. This allows for significant performance improvements and reduced bandwidth from your server.
-somebody's project on github, not sure if accurate
I've never cached something before and these two statements confuse me. When the API's site says to "please cache the results", it sounds like caching is something I can do in a fetch request, or somehow on the frontend. For example, some way to store the results in local storage or something. But I couldn't find anything about how to do this. I only found resources on how to force a response NOT to cache.
The second quote makes it sound like caching is something the API does itself on their servers, since they set the response to cache automatically.
How can I cache the results like the api site asks?
To clear your confusion on the conflicting statements you're referencing:
Caching just means to store the data. Examples of where the data can be stored are in memory, in some persistence layer (like Redis), or in the browser's local storage (like you mentioned). The intent behind caching can be to serve the data faster (compared to getting it from the primary data source) for future requests/fetches, and/or to save on costs for getting the same data repeatedly, among others.
For your case, the http://exchangeratesapi.io/ API is advising consumers to cache the results on their side (as you mentioned in your question, this can be in the browser's local storage, if you're calling the API front front-end code, or stored in memory or other caching mechanisms/structures on the server-side application code calling the API) to that they can avoid the need to introduce rate limiting.
The project from Github you're referencing, Laravel Exchange Rates, appears to be a PHP wrapper around the original API - so it's like a middleman between the API and a developer's PHP code. The intent is to make it easier to use the API from within PHP code, and avoid having to make raw HTTP requests to the API and avoid processing the responses; the Laravel Exchange Rates handles that for the developer.
In regards to the
By default, the responses all of the requests to the exchangeratesapi.io API are cached
statement you're asking about, it seems the library follows the advice of the API, and caches the results from the source API.
So, to sum up:
http://exchangeratesapi.io/ is the source API, and it advises consumers to cache results. If your code is going to be calling this API, you can cache the results in your own code.
The Laravel Exchange Rates PHP library is a wrapper around that source API, and does cache the results from the source API for the user. If you're using this library, you don't need to further cache.
Is there a way to declare a variable in the application scope. This variable should stay in the memory and serves all the requests without the need to create it for each request. Something like session, but it is not tied to a specific user.
What Im looking for is something similar to php APC cache.
Session fields provide one possibility; another possibility is server fields:
http://docs.marklogic.com/xdmp:set-server-field
A server field is available to all requests for the appserver on the host.
Since this is tagged "marklogic" I will provide a MarkLogic non-standards answer.
The standards based answer is 'no' or 'question out of scope' because XQuery doesnt define an 'application scope' ( beyond the excecution of a the top level module's statement -- which in some context could be called 'application scope' )
In MarkLogic there is a supported feature called 'Sessions';
For web applications that coresponds to what is usually called "Session".
Sessions are created by the first call to xdmp:login() (explicitly or implicitly by running in the HTTP Server using a supported authentication).
Session 'state' is created on the first write to a session field (explicitly via xdmp:set-session-field, or implicitly in some cases(*)).
https://docs.marklogic.com/xdmp:set-session-field
Session Fields are a map of name (xs:string) to item()*
They last as long as the session lasts (see notes on expiration in link above).
Note that Sessions that span multiple HTTP requests require that the client side participate via proper HTTP Cookie handling ( ML server has no ability nor attempts to to track requests after the socket is closed except via the HTTP Cookie mechanism.)
If you are using a modern browser you need to not disable cookies.
If you are using a modern Web Client library this will likely be done for you.
If you are using one MarkLogics published SDK's (java, node, xcc) it will be done for you.
If you are going through a load balancer, Level 3 router, gateway, proxy ... those may need to be configured for "HTTP Session Affinity" (based on the cookies) to guarantee that cookies are passed through unchanged and that the SessionID cookie is mapped to HTTP affinity.(**)
Note: IT devices vary greatly in their capabilities and terminology.
What is often called "Session Affinity" may not relate to cookies, or
the right cookies. These can often be configured to work correctly but may
require custom code or configurations.
(*) Session state can be created implicitly in some cases by creating a multi-statement transaction.
(**)
https://docs.marklogic.com/guide/node-dev/extensions
Scenario:
Implement in-memory caching of master data in the WCF layer of an ASP.Net application for a web farm scenario
Data is cached on first access to the service layer ‘s, say, GetCountryList() method with cache’ expiry set to midnight. Let’s say the cache key is “CountryList_Cache”
All subsequent requests are served through cache’
If the Country list is updated using the master screen, then an additional call is made to invalidate the “CountryList_Cache” and fresh data is loaded into it
The next call now receives the updated country list
The above step is easy in a single server scenario, as step 3 only requires a cache expiry call to one server. The complexity increases when we have 2 or 3 load balanced web servers, because in that case the cache is updated (via master screen) on only one of the servers but has to be invalidated on all 3 servers.
Our proposed solution:
We intend to have an external service/ exe/ web page which would be aware of all load balanced servers (via a configuration file). In order to invalidate a specific cache, we would invoke this external component which in turn would invalidate the respective cache key on all the web servers and load then cache with latest data.
The problem:
Although the above approach would work for us, we do not think it is a clean approach for an enterprise class LOB application. Is there a better/ cleaner way of achieving the cache expiry across multiple servers?
Note:
We do not want to use distributed caching due to the obvious performance penalty, as compared to in-proc/ in-memory cache
Caching has been implemented using System.Runtime.Caching
We have worked with SQL dependency and used it in scenario of single web server
Comparing your design to Windows Azure In-Role Cache and AppFabric Cache.
In those products, the cache is stored in one or more servers (cache cluster). In order to speed up requests, they created Local Cache.
When local cache is enabled, the cache client stores a reference to
the object locally. This local reference keeps the object active in
the memory of the client application. When the application requests
the object, the cache client checks whether the object resides in the
local cache. If so, the reference to the object is returned
immediately without contacting the server. If it does not exist, the
object is retrieved from the server. The cache client then
deserializes the object and stores the reference to this newly
retrieved object in the local cache. The client application uses this
same object.
The local cache can be invalidation by time-out and/or notification
Notification-based Invalidation
When you use cache notifications, your application checks with the
cache cluster on a regular interval to see if any new notifications
are available. This interval, called the polling interval, is every
300 seconds by default. The polling interval is specified in units of
seconds in the application configuration settings. Note that even with
notification-based invalidation, timeouts still apply to items in the
local cache. This makes notification-based invalidation complementary
to timeout-based invalidation.
Say, for example, you are caching data within your ASP.NET web app that isn't often updated. You have another process running outside of the app which ocassionally updates this data, when you do this you would like the cached data to be cleared immediately so that the next request picks up the new data straight away.
The caching service is running in the context of your web app and not externally - what is a good method of calling into the web app to get it to update the cache?
You could of course, just hack a page or web service together called ClearTheCache that does it. This can then be called by your other process. Of course you don't want this process to be externally useable or visible on your web app, so perhaps you could then check that incoming requests to this page are calling localhost, if not throw a 404. Is this acceptable? Could this be spoofed at all (for instance if you used HttpApplication.Request.Url.Host)?
I can think of many different ways to go about this, mainly revolving around creating a page or web service and limiting requests to it somehow, but I'm not sure any are particularly elegant. Neither do I like the idea of the web app routinely polling out to another service to check if it needs to execute something, I'd really like a PUSH solution.
Note: The caching scenario is just an example, I could use out-of-process caching here if needed. The question is really concentrating on invoking code, for any given reason, within a web app externally but in a controlled context.
Don't worry about the limiting to localhost, you may want to push from a different server in future. Instead share a key (asymmetrical or symmetrical doesn't really matter) between the two, have the PUSH service encrypt a block of data (control data for example) and have the receiver decrypt. If the block decrypts correctly and the data is readable you can safely assume that only the service that was supposed to call you has and you can perform the required actions! Not the neatest solution, but allows you to scale beyond a single server.
EDIT
Having said that an asymmetrical key would be better, have the PUSH service hold the private part and the website the public part.
EDIT 2
Have the PUSH service put the date/time it generated the cipher text into the data block, then the client can be sure that a replay attack hasn't taken place by ensuring the date/time is within an acceptable time period (say a minute).
Consider an external caching mechanism like EL's caching block, which would be available to both the web and the service, or a file to cache data to.
HTH.
We are working on a web application that is distributed across 3 load-balanced web servers. A lot of the data we retreive via NHibernate is stored in System.Web.Caching.Cache.
System.Web.Caching.Cache does a lot to increase the responsiveness of an application, but there are a few issues that we don't exactly know how to resolve, such as
when a user requests data on server1 that data is cached on server1, but for their next request, the load balancer might direct them to server2. That data they requested on server1 is no longer available, and server2 will have to request it from the database again.
If the user does something on server1 to invalidate the cached data, the cache is flushed on server1. However the original cache is still available on server2 & server3, so when the user submits a subsequent request and they're directed to either of the other servers, they are going to be presented with invalid data.
We have applications that update data (such as performance data) on a regular basis. When the performance data is updated we want to flush this from the cache so when a user requests the data again, they're presented with the latest data. How can we get these applications to flush the cache on 3 web servers?
What are the best ways to resolve these issues?
Should we have cache stored on a separate server such as we could to for HttpContext.Session with a SessionState server?
Is there a way for us to set a Cache Dependency on the Cache in the other 2 servers?
Is it possible for us to implement a Cache Dependency on the actual database tables or rows? When these change the cache is flushed? -- or could we set up a database trigger to flush the cache somehow?
Yes, a multi-server environment exposes the weakness of the ASP.NET cache in that it is single-server only. You might want to look into using a distributed cache like AppFabric, which would give you a single logical cache that underlies all three web servers.
AppFabric can be used with NHibernate as a second-level cache - see http://sourceforge.net/projects/nhcontrib/files/NHibernate.Caches/ (although be aware that this question suggests the current implementation may not be up-to-date with the AppFabric v1 release).
You have a few options:
1) Use a distributed cache (such as distcache, velocity, or ncache)
2) Use a shared cache instance (something like memcached, for instance) that all of your web servers make use of.
NHibernate has second-level cache providers for all of these, which can be found here.