I have enabled multi-write enabled for the Cosmos account in azure portal. I don't understand if it is mandatory to set the ApplicationRegion using the SDK as well? If it is mandatory, what is the purpose of this property? I see the below documentation, but it is still not clear to me.
Documentation
The purpose for ApplicationRegion is in scenarios where the SDK detects the region is no longer responsive, it will use that information to know which is the next closest region to failover to. If you do not set this value the SDK will failover to the next highest region listed in the portal or by failoverPriority in your Cosmos account configuration which may not be the closest.
Related
How to add Azure custom Policy for Azure Data Factory to only use Azure Key Vault during the Linked Service Creation for fetching the Data Store Credentials instead of credentials being put up directly in ADF Linked Service. Please suggest ARM or PowerShell methods for the policy implementation.
As of yesterday, the Data Factory Azure Policy integration is available which means you can now find some built-in policies that can be assigned to ADF.
One of those is exactly what you're asking for as you can see in the image below. You can find more information here
Edit: Based on your comment, I'm editing this answer with the info you want. When it comes to custom policies, it's pretty much up to you to come up with them and create what fits your needs. In your particular case, I've created one policy that does what you want, please see here.
This policy will audit your data factory linked services and check if they're using a self-hosted integration runtime. Currently, that check is only done for a few types of integration runtimes (if you look at the policy, you can see 5 of them) which means that if you want to check more types of linked services, you'll need to add them to the list of allowed values and select them when assigning the policy definition.
Bear in mind that for some linked services types, such as Key Vault, that check won't make sense since that service can't use a self-hosted IR
Does Cosmos DB automatically set PreferredLocations? For example when new regions are added/deleted.
Or users have to set it themselves?
According to the docs, the most optimal endpoint will be chosen by the SQL SDK to perform write and read operations: https://learn.microsoft.com/en-us/azure/cosmos-db/tutorial-global-distribution-sql-api#connecting-to-a-preferred-region-using-the-sql-api
This is based on the configuration of your account, region availability. If you don't specify the PreferredLocation property, then all requests (read and write) will be served from your account's current write region.
Hope this helps :)
PreferredLocations must be specified for high availability. EnableEndpointDiscovery along with PreferredLocations allows you to leverage Cosmos DB failover capabilities.
When the value of this EnableEndpointDiscovery is true, the SDK will automatically discover the current write and read regions to ensure requests are sent to the correct region based on the regions specified in the PreferredLocations property. Default value is true indicating endpoint discovery is enabled.
When will autopilot available for existing database and containers?
When will autopilot available to be configured by ARM template?
Is there ETA for these features? We need to decide whether to wait for the official autopilot or implement it by ourselves if it takes long time before GA.
ARM support for AutoPilot, as well as migration capability for existing database and container resources will be available when this feature goes GA.
We are not disclosing the exact date for GA but it will be fairly soon.
Thanks.
This is now GA, the ARM template below works for me.
Create an Azure Cosmos DB account for Core (SQL) API with autoscale
I still get 'Error updating offer for resource dbs/... {"code":500,"body":{"code":"InternalServerError","message":"Message: {\"Errors\":[\"An unknown error occurred while processing this request. If the issue persists, please contact Azure Support:
Existing container, UK South
In Cosmos Db, I am using a document level Time to Live (TTL) and Cosmos does not appear to be expiring documents. Does this feature work in Cosmos Db using MongoDB API? If it does, what am I missing?
I am using Cosmos Db with the MongoDB API.
A "ttl" field is set in each document for my collection.
In Azure, Time to Live is set to "On (no default)" for my collection.
I am doing this without the emulator because the emulator defaults to the SQL API. In the emulator, I see "_ts" set and I do not see this field in Azure.
I can switch to collection level expiration by setting Time to Live to "On" and documents expire as expected. When I do this, my "ttl" field is ignored and the value I set for "second(s)" in Azure is followed. I still see my "ttl" field in the document.
Although I don't see a "_ts" field in my documents, an article about indexing mentions that it is a reserved property. This makes be think that it is set behind the scenes and it is not returned in queries.
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-indexing
"_ts is a Cosmos DB-specific field and is not accessible from MongoDB clients. It is a reserved (system) property that contains the timestamp of the document's last modification."
Update:
I checked the MongoDB support page (https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support) and it indicates that collection level TTL is available and says nothing about document level.
Azure Cosmos DB supports a relative time-to-live (TTL) based on the timestamp of the document. TTL can be enabled for MongoDB API collections through the Azure portal.
Update:
My Azure Portal Preview Features now show this:
I got document level Time to Live working in Cosmos Db using MongoDb API. I had to ask help from Microsoft support to get this working. Response from Microsoft Big data team was following.
Before enabling Document level TTL feature , I would like to
clarify following about Document TTL feature details here.
The TTL feature is controlled by TTL properties at two levels -
Collection level and the Document level.
Right now per Document level TTL for MongoDB accounts are not
available by default. However, we can enable this feature for specific
customers and this feature is set at an account level.
TTL is at a document level but the feature is enabled at an account
level which means for all collections under the account, if there is a
document with a TTL set, it will take effect. For other collections,
if the TTL value is not set for each document, it would not be
affected.
You needs to have an index on the _ts field for this to work.
To summarize this : - This feature works at Cosmos DB account level.
We need to enable Document TTL feature in Cosmos DB backend on our
side.
Firebase's realtime database SDK has callback methods built in which allow the client to listen for database changes. (e.g. as per https://firebase.google.com/docs/database/android/start/)
Does Azure have any functionality like that in either the SQL Database or DocumentDB products? If not, why not? And what is the best way to achieve this functionality using Azure?
Thanks.
Edit context: I'm building a cross-platform mobile app using Xamarin.
Microsoft just announced a new feature in DocDb called Change Feed which sounds like what you are looking for. It is enabled by default and included in the cost of DocDb.
From their announcement: "With Change Feed support, DocumentDB provides a sorted list of documents within a DocumentDB collection in the order in which they were modified. This feed can be used to listen for modifications to data within the collection and perform actions..."
Documentation on how to work with the change feed can be found here: https://learn.microsoft.com/en-us/azure/documentdb/documentdb-change-feed
Azure's SQL Database service now has Extended Events (currently in Preview). More info here.
DocumentDB does not have events you can subscribe to. You would need to build your own mechanism for taking action when modifying your database contents.
EDIT: Cosmos DB (formerly called DocumentDB, when this answer was originally posted) provides a change feed you can subscribe to.
As for "best way to achieve this functionality" - there is no best way. It's whatever you decide to build, based on your app, chosen frameworks, etc.