We have a need to pre-populate Cosmos DB containers with some static json files. This is a requirement for both local developer Cosmos DB emulator environments, and also Azure DevOps deployments. Ideally those two scenarios would use the same approach of course.
One way I was thinking was to have static json documents in our git repo, and to have a dotnet core command line tool that would connect to Cosmos DB and insert one or more docs to a specified DB and container, per invocation of the console app.
I found this tool which seems like a good fit:
https://github.com/azure/azure-documentdb-datamigrationtool
https://learn.microsoft.com/en-us/azure/cosmos-db/import-data
However, this targets .NET Framework 4.5, and therefore cannot be used easily by our Mac and Linux developers. So one option would be to have a go at migrating that tool to dotnet core.
I also found these bash scripts that seem relevant:
https://github.com/Krumelur/AzureScripts/blob/master/cosmosdb_create_document.sh
Use bash, Azure CLI and REST API to access CosmosDB - how to get token and hash right?
I.e. Windows users could use WSL to run these.
However, I think a dotnet core console app would be the ideal solution here. It seems like an obvious simple tool to want, so was wondering if there is anything already out there.
Or maybe am I thinking about this problem the wrong way?
There isn't anything out there today but the first link you referenced there to the DMT running on .NET 4.5 is being transitioned to a new maintainer, Solliance, who is going to be ported over to .NET Core but there is no ETA as of yet.
The only thing I can suggest is to roll your own app to read from blob storage and insert into Cosmos. The other possible option too is to use Azure Data Factory and create/update a job with new endpoint and keys when you roll a new environment.
Related
I have a Xamarin Forms app that I have switched from using only restful API to using local SQLite DB, that will sync using this Dotmim.Sync - DB Sync'ing Framework (which is great!). I am using EF Core 3 in the Xamarin forms project to interact with SQLite.
My questions are around running migrations or just database updates on SQLite. I have read several blogs and forums about different approaches, but they all are several years old, from EF Core 1 to EF Core 2, and lots of work around.
https://forums.xamarin.com/discussion/101805/xamarin-android-entity-framework-core-2-and-migrations - this talks about running migrations by add console application
https://medium.com/#yostane/entity-framework-core-and-sqlite-database-migration-using-vs2017-macos-28812c64e7ef - this add's a Dot Net Cli Tool Reference. I tried this and got errors, but it also EF Core 1.1
https://www.algoworks.com/blog/xamarin-forms-and-entity-framework-core/ - gave me more questions then answers.
There are more links but I, these prove my point of all the different approaches taken.
What I am trying to achieve is:
When a user opens the app, it checks if there is DB version update, or the app updated and knows it has to update the database.
When there is a DB update, Dotmim.Sync Framework is in control of Provisioning and Deprovisioning the database on the client (Xamarin Forms). Which gives the opportunity to run EFCore Migrations or SQL Sricpts to update the SQLite store.
Option 1
I would like to go the EF Core Migration path, since the Master DB is updated with Migrations, The App would be able to leverage the same scripts. If running migrations on Xamarin forms at runtime is possible that would be great.
Q1. Can EF Core run mirgations in Xamarin Forms apps during runtime, same as it would in .netcore/netframework project?
Option 2
If running scripts needs to be the path thats fine, which i have explored and means I need to be able to SQLite schema compares Tool 1 and tool 2, I tried the tool one on SQLite db3 and it error'ed. Wasn't willing to pay for Tool 2 unless without verification it works.
Q2. Is there a SQLite compare tool to generate schema changes?
Q3. Or what is database update path for sqlite deployed on clients, that would be better, i.e drop the DB and just recreate it? then re sync the DB, to load back all user specific data. The chances of lost data would be minimal, the sync logic in the App will be frequent, not sure after every user data change event, but possible if we need to make sure the server never out of sync.
I hope I made this clear. Thanks
You'll need to create the dummy console app as mentioned. Here are instruction for installing the tools. After that, the normal Migrations workflow should work on Xamarin (Be sure to let us know if it doesn't).
Add-Migration MyMigration -P MyNetStandardClassLibrary -S DummyNetCoreApp
You can apply migrations at runtime using this method:
myDbContext.Database.Migrate();
I am in the process of converting our legacy custom database deployment process with custom built tools into a full fledged SSDT project. So far everything has gone very well. I have composite projects that can deploy a base database as well as projects that deploy sample and test data.
The problem I am having now is finding a solution for running some sort of code that can call a web service to get an activation code and add it to the database as the final step of the process. Can anyone point me to a hook that I might be able to use?
UPDATE: To be clearer I am doing this to make it easier to maintain and deploy our sample and test data to a local machine. We can easily use Jenkins to activate the sites when they are deployed nightly to our official testing environments. I'm just hoping to be able to do this in a single step to replace the homegrown database deploy tool that we use now.
In my deployment scenario I wrapped database deployment process in some powershell scripts which do necessary prerequisites. For example:
powershell script is started and then it stops some services
next it run sqlpackage.exe or preproduced sql deployment scripts
finally powershell script starts services.
You can pass some parameters from powershell to sql scripts or sqlpackage.exe as sqlcmd variables. So you can call webservice first, then pass activation code as sqlcmd variable and use the variable in postdeployment script.
Particularly if it's the final step, I'd be tempted to do this separately, using whatever tool you're using to do the deployment: Powershell scripts, msbuild, TFS, Jenkins, whatever. Presumably there's also a front-end of some description that gets provisioned this way?
SSDT isn't an eierlegende Wollmilchsau, it's a set of tools for managing database changes.
I suspect if the final step were "provision a Google App Engine Instance and deploy a Python script", for example, it wouldn't appear to be a natural candidate for inclusion in an SSDT post-deploy script, and I reckon this falls into the same category.
How do I apply EF7 migrations on an Azure database?
According to this link, you simply tick a box in the Publish Profile settings. Well, I don't have that checkbox - I'm not sure if the profile configuration has changed since then but I don't even have a databases section.
According to this link, EF7 doesn't support database initializers and you have to use nuget package manager or k (dnx) migrations. I'm not sure about the nuget option, so going with the dnx option: how do I connect to my Azure (hosted) project/website using a dnx console window or the Package Manager Console in VS?
Are there any other options (hopefully easier!) for doing this?
Here's the 'new' way:
_context.Database.EnsureCreated();
_context.Database.Migrate();
Simple.
My Azure database somehow had some migrations applied, but nothing in the __EFMigrationsHistory table, so I dropped all other tables and then ran all the migrations to get it back to where I wanted it.
I've managed to apply the migrations by changing my connection string on my local project, opening the firewall on Azure to my IP address, and running a dnx . ef migration apply command.
However, that doesn't seem like a good solution to me: now I have to store my live connection string in my dev project and keep switching between the two. There must be a better way...?
I am trying to write a script to deploy asp.net application from Clear Case. I am using Clear Case Remote Client.
How will i start? what is the easiest way?
CCRC is for accessing code from a "web" ClearCase snapshot view.
Being a light ClearCase installation, you:
won't have all the cleartool command which would allow to detect new content (new versions on files) to be updated
won't have the easy integration you could have with TeamCity, or Jenkins, or Hudson, ... since they all rely on a cleartool command.
TeamCity, for instance, has still a pending ticket on CCRC support.
For you, since you don't want/need to use those schedulers anyway, you can start by using the CCRC CLI (rcleartool) in order to:
update your ccweb view
check if the update has gotten any new versions
deploy your app if it has gotten anything new.
rcleartool update [-username user-name][-ser/ver server-url][-pas/sword user-password]
[-print] [-ove/rwrite | -nove/rwrite | -ren/ame]
[pname ...]
Jenkins currently follows a similar path to plan for CCRC support: ticket 5192:
(and neither Jenkins nor Hudson support CCRC yet)
I'm thinking about which is better the calling of rcleartool as external tool, or develop a teamapi (or as they call now cmapi) based pure java extension.
More details on this IBM article:
"Continuous integration with IBM Rational ClearCase Remote Client"
In this general architecture schema for CI with CCRC, my suggestion above (rcleartool update) is illustrated by the link between the CM server and the build server.
Personally I'd start by not re-inventing the wheel.
Team City is one such product that can do what you're asking about
http://www.jetbrains.com/teamcity/
Does anyone have any best practices around deploying database changes in an EF 4.1 code-first solution? I know MS does not currently support database migrations for EF 4.1, but obviously people are going to need to do this from time to time.
Thanks
Once you deployed database to production you must do incremental changes. It means that before you deploy next version you must prepare two databases in your dev box:
Database with DB schema currently deployed in production - you should be able to get this from source control so always correctly label / tag your production releases
Database with new DB schema
Once you have two databases you can use some tool to make difference SQL script for you. I have experience with both:
Visual Studio 2010 Premium / Ultimate Database tools
Red Gate SQL Compare
These tools are for SQL server.
Once you have difference script you can test it on your dev box. Be aware that some more complicated changes cannot be created by difference script and require you to create custom migration script for example with storing data existing data in temporary tables while refactoring real table. Also if you use some new seed data in your new version you must add them manually into script or use Data Compare tools (also offered by both products).
After that you can plan outage of your production application, database backup and running upgrade script.