I have enabled Global replication for cosmos db in prod. The uri of my cosmos db is my-cosmos-db-prod.document.azure.com:port . In Application Insight's Application Map, I am seeing traffic for 3 different endpoints.
my-cosmos-db-prod
my cosmos-db-prod-eastus
my-cosmos-db-prod-westus.
I don't understand why I am seeing traffic for first endpoint. Shouldn't I be seeing only regional endpoints? What region does the first endpoint represent?
Also I have enabled 5 regions but I am seeing traffic for only 2 regions(eastus, westus). For the service of my scale, I am pretty sure there has to be traffic in other regions as well. It has been just 12 hours since I added other regions and everything looks good on azure portal. Is there a delay in when we can actually start seeing traffic on newly added regions?
I am using cosmos db 3.17.1
This is how I have configured region preference in code.
.WithApplicationRegion(this.GetRegionPreference())
private string GetRegionPreference()
{
switch (this.serviceConfig.Region)
{
case "usea":
return Regions.EastUS;
case "uswe":
return Regions.WestUS;
case "asea":
return Regions.EastAsia;
case "asse":
return Regions.SoutheastAsia;
case "euno":
return Regions.NorthEurope;
case "euwe":
return Regions.WestEurope;
default:
return Regions.WestUS;
}
}
I agree with #Matias.
According to your description, the sdk you used is needed to set ApplicationRegion and this parameter decides which region your program will be deployed.
And to explain why you only see traffic for 3 different endpoints while you seems to set some other regions, I think it's based on the rule of cosmosdb. For lower latency, cosmosdb will use a closer endpoint to provide service.
To lower the latency, place the data close to where your users are.
Choosing the required regions depends on the global reach of your
application and where your users are located. Cosmos DB transparently
replicates the data to all the regions associated with your Cosmos
account.
Related
Update July 13, 2021
The links used below are now partially obsolete. Here is the new section on language differences.
Original post
On Azure Portal, in my App Insights / Logs view, I can query the app data like this:
app('my-app-name').traces
The app function is described in the article app() expression in Azure Monitor query.
Kusto.Explorer doesn't understand the app() function, which appears to be explained by the fact it is one of the Additional operators in Azure Monitor.
How can I query my App Insights / Logs with Kusto.Explorer? I cannot use cluster as it is one of the functions not supported in Azure Monitor.
Relevant doc: Azure Monitor log query language differences
Note on troubleshooting joins
(added December 16, 2021)
Pro-tip from Kusto team:
If you are querying application insights from Kusto.Explorer, and your joins to normal clusters fail with bad gateway or other unexpected error, consider adding hint.remote=left to your join. Like:
tableFromApplicationInsights
| join kind=innerunique hint.remote=left tableFromNormalKustoCluster
We have a private preview for Azure Data Explorer (ADX) Proxy that enables you to treat Log Analytics / Application Insights as a virtual cluster, query it using ADX tools and connecting to it as a second cluster in cross cluster query. Since its a private preview you need to contact adxproxy#microsoft.com in order to get enrolled. The proxy is documented at https://learn.microsoft.com/en-us/azure/data-explorer/query-monitor-data.
(disclaimer - I'm the PM driving this project).
Step 1 Connection String
Build your connection string from this template:
https://ade.applicationinsights.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<ai-app-name>
Fill in the subscription-id, resource-group-name, and ai-app-name from the portal. Here is an example image
Step 2 Add the connection to Kusto.Explorer
Open Kusto.Explorer, choose Add Connection, and paste your connection string into the Cluster connection field.
After you kit OK, Windows will prompt you to log in with your Azure Active Directory account. Once you have authenticated, Kusto.Explorer will display the Application Insights tables in the Connections panel.
So I have setup my cosmos in 2 regions (say West US and South Central US). I also have my app services running in these two region and connecting to cosmos. For app services running in each region I have configured my preferred region list. So for app service running WUS region, preferred list is in order [WUS, SCUS] and for app service running SCUS region, preferred list is in order [SCUS, WUS].
I want to verify if this configuration is working and my data was returned from cosmos region in order as i have mentioned. For example if accessed from WUS app service, verify if region chooses to execute the query was WUS and vice versa.
Is there any way to verify this?
NOTE : I am using spring-data-cosmosdb-2.1.2 to connect to cosmos.
Not sure if you can get that information from Response in code but you can make use of Cosmos Metrics in Azure portal.There you can filter the metrics on Region.
So, Attempt a request through App from region 1 and then verify in portal that expected Cosmos region served it. Test in same way for region 2.
Yes, just dump the Diagnostics returned in the response object and look for the FQDN to the endpoint. It will have a regional subdomain pre-pended to the URI.
I am planing to create few API calls based on Firebase Cloud Functions and DB.
Every call will access to Firebase DB, get data or modify data in Firebase DB and return relevant responce.
But i am afraid of DDOS atacks, or similar massive access attacks.
By my usecase scenairio I am sure that user does not need more than 25 API calls in one minute.
Is it possible to set such limit to my Firebase Cloud Functions/DB ?
How can i protect my Firebase from overusing by unconscious individuals.
private mylock:boolean = true;
if(this.mylock == true){
do DB dquery//
this.mylock = false;
setTimeout(function(){
this.mylock = true;
}, 3000);
}
You could also look into Cloudflare to protect against attacks https://www.cloudflare.com/integrations/google-cloud/
But you might be going over the top, worrying about this... in a tin foil hat kind of way, it is quite rare for this to happen. I would simply put a budget alert in your billing section of Google Cloud that will flag up any large usage.
Is there any way to write a security rule or is there any other approach that would make possible only for currently connected (not authenticated) user to write/read certain location - admin should also be able to write/read?
Can a rule be written that disallows users to read of complete list of entries and let them read only entry that matches some identifier that was passed from client?
I'm trying to exchange some data between user and Node.js application through Firebase and that data shouldn't be able to read or write by anyone else other than user and/or admin.
I know that one solution would be that user requests auth token on my server and uses it to authenticate on Firebase and that would make it possible to write rule which prevents reads and writes. However, I'm trying to avoid user connecting to my server so this solution is not first option.
This is in a way session based scenario which is not available in Firebase but I have
some ideas that could solve this kind of problem - if implemented before session management:
maybe letting admin write into /.info/ location which is observed by client for every change and can be read only by active connection - if I understood correctly how .info works
maybe creating .temp location for that purpose
maybe letting admin and connected client could have more access to connection information which would contain some connection unique id, that can be used to create location with that name and use it inside rule to prevent reading and listing to other users
Thanks
This seems like a classic XY problem (i.e. trying to solve the attempted solution instead of the actual problem).
If I understand your constraints correctly, the underlying issue is that you do not wish to have direct connections to your server. This is currently the model we're using with Firebase and I can think of two simple patterns to accomplish this.
1) Store the data in an non-guessable path
Create a UUID or GID or, assuming we're not talking bank level security here, just a plain Firebase ID ( firebaseRef.push().name() ). Then have the server and client communicate via this path.
This avoids the need for security rules since the URLs are unguessable, or close enough to it, in the case of the Firebase ID, for normal uses.
Client example:
var fb = new Firebase(MY_INSTANCE_URL+'/connect');
var uniquePath = fb.push();
var myId = uniquePath.name();
// send a message to the server
uniquePath.push('hello world');
From the server, simply monitor connect, each one that connects is a new client:
var fb = new Firebase(MY_INSTANCE_URL+'/connect');
fb.on('child_added', newClientConnected);
function newClientConnected(snapshot) {
snapshot.ref().on('child_added', function(ss) {
// when the client sends me a message, log it and then return "goodbye"
console.log('new message', ss.val());
ss.ref().set('goodbye');
});
};
In your security rules:
{
"rules": {
// read/write are false by default
"connect": {
// contents cannot be listed, no way to find out ids other than guessing
"$client": {
".read": true,
".write": true
}
}
}
}
2) Use Firebase authentication
Instead of expending so much effort to avoid authentication, just use a third party service, like Firebase's built-in auth, or Singly (which supports Firebase). This is the best of both worlds, and the model I use for most cases.
Your client can authenticate directly with one of these services, never touching your server, and then authenticate to Firebase with the token, allowing security rules to take effect.
The AppFabric 1.1 client documentation discusses assigning a list of DataCachServer endpoints to the DataCacheFactoryConfiguration. Most of the examples show the list consisting of a single or perhaps two different cache servers. If the cluster consists of n servers should the client register each of the servers? Does it matter what order the servers are registered in? For example, if I have 50 servers in my web tier, and 5 servers in my cache tier, do each of the 50 web servers register all 5 caching servers? Here is sample code:
// Declare array for cache host(s).
DataCacheServerEndpoint[] servers = new DataCacheServerEndpoint[5];
servers[0] = new DataCacheServerEndpoint("Cache01", 22233);
servers[1] = new DataCacheServerEndpoint("Cache02", 22233);
servers[2] = new DataCacheServerEndpoint("Cache03", 22233);
servers[3] = new DataCacheServerEndpoint("Cache04", 22233);
servers[4] = new DataCacheServerEndpoint("Cache05", 22233);
// Setup the DataCacheFactory configuration.
DataCacheFactoryConfiguration factoryConfig = new DataCacheFactoryConfiguration();
factoryConfig.Servers = servers;
// Create a configured DataCacheFactory object.
DataCacheFactory mycacheFactory = new DataCacheFactory(factoryConfig);
// Get a cache client for the cache named "default".
DataCache myDefaultCache = mycacheFactory.GetCache("default");
Can each web server register identically, and will the load be balanced across the caching tier? If a registered server becomes unavailable is the next one tried in sequence, or is it randomized? Links to supporting documentation would be helpful.
Related to load balancing, Jason Roth wrote the following [is there other documentation available]?
App fabric client is smart client and it can directly contact the server which ever server has your data. The application need not worry about load balancing. This is done using the routing client.
Based on some testing, and letting Jason Roth's comment sink in, I think the DataCacheServerEndPoint is used by the "smart client" to retrieve the list of cache cluster members when the GetCache method is called on the DataCacheFactory. The DataCache object is the thing that is smart--and it is smart in the sense that if the server used in the DataCacheServerEndpoint instantiation goes offline or otherwise becomes unavailable, the smart client still has access to the other cluster members. Therefore the purpose of a list of more than one DataCacheServerEndpoint is to provide redundancy when calling the GetCache method.
The advice is that the DataCache object should follow a singleton pattern, and not be instantiated on each request for data from the cache. Which is why there is no need to loadbalance or provide a VIP for the individual DataCacheServerEndpoints.
Instantiate as many DataCacheServerEndPoints as needed to ensure at least one is up at all times--there is no need to add every member of the cache cluster unless that is the only way to ensure at least one is up.
When it comes to administering boxes in the cache cluster (for instance, applying monthly patches), consider minimizing the cache thrashing and rebalancing by administering a single box at a time, rather than attempting to administer groups of boxes in "waves".