My Question: How can i query/list the contents of the cache.
So i have creates one cache from Management API->Environment Configuration.
Then i created a api proxy with no target and attached populate cache policy to it. e.g.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<PopulateCache async="false" continueOnError="false" enabled="true" name="populateBlacklistByIP">
<DisplayName>populateBlacklistByIP</DisplayName>
<FaultRules/>
<Properties/>
<CacheKey>
<Prefix>CLIENTIP-</Prefix>
<KeyFragment ref='request.queryparam.myvar'/>
</CacheKey>
<CacheResource>mycache</CacheResource>
<Scope>Global</Scope>
<!-- no expiry -->
<ExpirySettings>
<TimeoutInSec>432000</TimeoutInSec>
</ExpirySettings>
<Source>request.queryparam.myvar</Source>
</PopulateCache>
Now i invoke the operation and get 200 ok back. But no way to check if the value made to cache or not.
Please help.
Got My Answer from digging through the documentation.
Ref: http://apigee.com/docs/api-services/content/persistence.
Read this from url above:
Cache versus Key/value map
The PopulateCache policy does not persist cache entries. Entries are in memory until the configured expiration time. You can look up the value only until it expires.
One limitation is that you cannot find the list of keys that are currently in a cache.
When using KeyValueMap, the keys are persisted indefinitely. There are APIs available to retrieve the list of keys. There is no expiration time for the keys; you must explicitly delete them.
Related
we want to share ASP.NET Session state between our apps and services. We chose Elasticache/redis to achieve this. It was going well but we've run into a deadlock scenario.
Here's the deadlock sequence:
user navigates to page served by App 1
App 1 uses RedisSessionStateProvider, successfully fetches the Session in a few milliseconds
App 1 makes an HttpWebRequest to App 2, with the ASP.NET_SessionId cookie attached
App 2 also uses RedisSessionStateProvider, which attempts to fetch the Session from the same redis instance and times-out after ~ 2 minutes
Presumably App 1's RedisSessionStateProvider is holding a (write?) lock on the cache item containing the Session. As you can tell from my parlance, I'm no redis guru...
AFAICT Elasticache gives you no visibility onto situations like this, just performance-y graphs. And RedisSessionStateProvider is closed-source so I can't poke around there.
I also tried to get RedisSessionStateProvider to log (via the loggingClassName parameter) but nothing gets written by either App 1 or App 2 (my Log() method is called though).
To prove that it is the RedisSessionStateProvider deadlocking (rather than our own code deadlocking) I switched App 1 back to using InProc sessions and everything runs fine.
Does anyone have any suggestions? BTW our Session data is to all intents and purposes immutable, so there really is no need for it to be locked.
Many thanks,
Pete
EDIT: the sessionState config as requested. Note that the large operationTimeoutInMilliseconds value is so that we don't get exceptions whilst debugging the app. This will be changed to ~ 5000 in production.
<sessionState mode="Custom" customProvider="RedisSessionProvider">
<providers>
<add name="RedisSessionProvider"
type="Microsoft.Web.Redis.RedisSessionStateProvider"
host = "ec2-184-73-3-249.compute-1.amazonaws.com"
port = "6379"
ssl = "false"
throwOnError = "true"
retryTimeoutInMilliseconds = "2000"
applicationName = "PE"
connectionTimeoutInMilliseconds = "2000"
operationTimeoutInMilliseconds = "1800000"
</providers>
</sessionState>
This is not answer but it is not fitting in comment section.
At start of page asp.net page execution life cycle it calls GetItemExclusive which fetches session from store (in this case redis) and puts a lock on that session so other parallel request cannot modify session while this request is working. This lock has time out that is equivalent of request timeout that you can set using web.config like below.
<configuration>
<system.web>
<httpRuntime executionTimeout="10"/>
</system.web>
</configuration>
Now, page executes and depending on weather anything was modified or not modified in session it calls SetAndReleaseItemExclusive or ReleaseItemExclusive which releases the lock. If this request fails for some reason than it will retry depending on retryTimeoutInMilliseconds value. If retryTimeoutInMilliseconds is very less or same as operationTimeoutInMilliseconds then it might not retry at all. If SetAndReleaseItemExclusive or ReleaseItemExclusive is not completed successfully then basically your session will be locked for complete time of “executionTimeout” that you have set above which is in seconds. All other request will be blocked and won’t be able to access the session while it is locked. Lock will be released automatically when it reaches expiry.
Use web.config properties loggingClassName and loggingMethodName for configuring logging. You can find more details in web.config comments when you upgrade to above package. You can basically provide a public, static method which returns a TextWriter. Session state provider and StackExchange.Redis.StrongName both will use this TextWriter object to log details.
This will help us get more details about problem. Beware that enabling logging will reduce the performance.
Example of using logging:
namespace SSPWebAppLatest3
{
public static class Logger
{
public static TextWriter GetLogger()
{
return File.CreateText("C:\\Logger.txt");
}
}
}
Web.config:
<add name="MySessionStateStore" type="Microsoft.Web.Redis.RedisSessionStateProvider" host="127.0.0.1" accessKey="" ssl="false"
loggingClassName="Logger, SSPWebAppLatest3, Version=1.0.0.0, Culture=neutral ……."
loggingMethodName="GetLogger"/>
Please send me a reproducible test app with which I debug this further. You can also do the same as session state and output cache provider code is open source now. (https://github.com/Azure/aspnet-redis-providers)
We are in the process of totally rewriting our main API Proxy config and we discovered an issue with our new configuration (or maybe our existing one) relating to how API keys are being validated. Our current API uses the policy GetOAuthV1Info
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<GetOAuthV1Info enabled="true" continueOnError="false" async="false" name="APIKey-Validate">
<DisplayName>APIKey-Validate</DisplayName>
<FaultRules/>
<Properties/>
<AppKey ref="request.queryparam.apikey"></AppKey>
</GetOAuthV1Info>
Our new configuration uses the policy VerifyAPIKey
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<VerifyAPIKey async="false" continueOnError="false" enabled="true" name="Verify-Api-Key">
<DisplayName>Verify API Key</DisplayName>
<APIKey ref="request.queryparam.apikey"/>
</VerifyAPIKey>
On the surface both of these policies appear to work fine. However, after deploying the new config to our test environment some API keys were failing with a 401 Unauthorized error. Digging into those keys we discovered that they are assigned to a product that doesn't have access to the test environment. It appears that the GetOAuthV1Info step is not validating the environment..? The documentation for GetOAuthV1Info doesn't help as it doesn't talk about APIKeys at all (http://apigee.com/docs/api-services/content/authorize-requests-using-oauth-10a).
Fixing this particular issue is pretty straight forward in that we just need to allow those other products access to the test environment. However, this makes me wonder what the other differences are between these two policies? I'm very nervous now about deploying any changes to these API proxies because I don't know what else will break, or what other unforeseen issues will appear.
Is this a known limitation with the GetOAuthV1Info policy? Why does this even work at all? What are the other differences between these two policies that might bite me later?
The only difference that I'm aware of is that the variable names are assigned differently in the VerifyAPIKey Policy (it appends the policy type and name to the vairalbes like verifyapikey.verify_apikey-1.apiproduct.developer.quota.limit for example).
Both VerifyAPIKey and OAuth 1 does support restrictions by environment -- when I tested the GetOAuthV1 with an APIKey in an invalid environment and got this error:
OAuth Failure : Invalid API call as no apiproduct match found
Keep in mind that the convention for most projects seems to be either OAuth2 flows or the VerifyAPI so there is less information about the OAuth1 policies.
I have set up a custom variable in my developer app called sandbox.app_id so I can have different ids for different applications.
I am using the verify api key policy and this populate the verifyapikey.* variables.
I want an ExtractVariable policy similar to below:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="false" enabled="true" name="Extract-TLRG-App-ID">
<DisplayName>Extract TLRG App ID</DisplayName>
<FaultRules/>
<Properties/>
<Variable name="app_id">verifyapikey.verify-api-key.{environment.name}.app_id</Variable>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
<VariablePrefix>apigee</VariablePrefix>
</ExtractVariables>
Is this possible, or will I have to resort to javascript?
Basically you are trying to access custom application attributes from your extract variable policy. So you need to place this extract variable policy after your verify api key policy. A verify api key policy would load all the application attributes as flow variables for a valid key. You may not see the custom attributes as variables in the debug view. However correct way to access the variables is to use this naming format below:
verifyapikey.{your_policy_name}.app.{custom_attribute_name}
Note that the verify api key policy is mandatory, otherwise the application associated with the API call can not be identified.
I need to call a legacy API which uses GET.
My API proxy uses POST.
I tried using in AssignMessage:
<AssignTo type="request" createNew="false"/>
and
<Set> ... <Verb>GET</Verb>
But it still does a POST on the target API.
What is the proper way of converting?
Will the gateway automatically convert the POST form parameters into GET query parameters?
Is message.queryparam the same for both GET and POST?
When converting the Verb from POST to GET, the policy will NOT automatically convert the form parameters to query parameters. You will need to use the <Add> and/or <Remove> functionality of the AssignMessage policy to manipulate the message further. Example use in the AssignMessage policy to add the queryparams, referencing the formparams:
<Add>
<QueryParams>
<QueryParam name="q1">{request.formparam.q1}</QueryParam>
</QueryParams>
</Add>
Also, in your question you mentioned that the API Proxy accepts the request using method as POST. Then, you have a policy to set GET:
<Set> ... <Verb>GET</Verb>
But it still does a GET on the target API.
What's the problem? Isn't that what you are expecting? The request goes into the Apigee API Proxy as POST, the proxy converts the method (verb) to GET, and sends the request to the backend legacy API using GET.
Note: <AssignTo> is optional in the AssignMessage. Try leaving this out if the method is not being set properly. In its absence, the message at the current point in the flow will be modified.
Change this predefined variable to post
request.verb = "GET"
Note: If you do this and you have a flow condition based on request.verb="POST" that will not work well in the response. So you need to use another variable to use in the flow condition.
Here is the policy code that worked for me.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="changeverbassignmessage">
<DisplayName>ChangeVerbAssignMessage</DisplayName>
<FaultRules/>
<Properties/>
<AssignVariable>
<Name>request.verb</Name>
<Value>GET</Value>
<Ref/>
</AssignVariable>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
<AssignTo createNew="false" transport="http" type="request"/>
</AssignMessage>
My site allows anonymous users.
I saw that under heavy load anonymous users get sometimes profile values from other users.
I first delete my cookies and get a valid unique value in the cookie value .ASPXANONYMOUS. After a couple of requests I get a new value for .ASPXANONYMOUS which is already used by another user. I see in my loggs that there are always a couple of users who share the same value in .ASPXANONYMOUS.
I can see in the my logs that 2 or more users realy get the same cookievalue for .ASPXANONYMOUS even if they have different IP.
Here is the htttp traffic. In the second image the changing cookie is shown (You have to display the image full size do be able to read the log):
One of the many requests that work ok:
alt text http://img413.imageshack.us/img413/2711/log1.gif
Then there is this one request that changes the cookie
alt text http://img704.imageshack.us/img704/8175/log2.gif
Then the new cookie is used
alt text http://img704.imageshack.us/img704/3818/log3.gif
Just to be safe I removed dependency injection.
I dont use OutputCaching.
My web.config has this setting for authentication:
<anonymousIdentification enabled="true" cookieless="UseCookies" cookieName=".ASPXANONYMOUS"
cookieTimeout="30" cookiePath="/" cookieRequireSSL="false" cookieSlidingExpiration="true" />
<authentication mode="Forms">
<forms loginUrl="~/de/Account/Login" />
</authentication>
Does anybody have an idea what else I could log or what I should have a look at?
UPDATE
I saw now that the http-traffic I showed is perfectly valid. A changing value in .ASPXANONYMOUS is something that happens because the cookie gets refreshed. The value contains AnonymousID and a Timestamp.
This does not lead to users having the same value in .ASPXANONYMOUS under normal conditions.
The problem realy is, that whenever the cokies get set from the AnonymousIdentificationModule, then there is a chance that a couple of user get this cookie. Setting a cookie in my application doesnt have this strange sideefect.
I had the same problem and solution was to turn off output caching for the responses where you call SetCookie. Below are several links describing this
Don’t let your cookie being cached by accident!
ASP.NET Session Mix-up using StateServer (SCARY!)
Integrated Pipeline and the kernel-mode cache
Are you declaring any static variables in your code at all? I had this similar issue, and narrowed it down to that; at least for my situation.