I am interested in knowing how can I integrate a repository with Azure Machine Learning Workspace.
What have I tried ?
I have some experience with Azure Data Factory and usually I have setup workflows where
I have a dev azure data factory instance that is linked to azure repository.
Changes made to the repository using the code editor.
These changes are published via the adf_publish branch to the live dev instance
I use CI / CD pipeline and the AzureRMTemplate task to deploy the templates in the publish branch to release the changes to production environment
Question:
How can I achieve the same / similar workflow with Azure Machine Learning Workspace ?
How is CI / CD done with Azure ML Workspace
The following workflow is the official practice to be followed to achieve the task required.
Starting with the architecture mentioned below
we need to have a specific data store to handle the dataset
Perform the regular code modifications using the IDE like Jupyter Notebook or VS Code
Train and test the model
To register and operate on the model, deploy the model image as a web service and operate the rest.
Configure the CI Pipeline:
Follow the below steps to complete the procedure
Before implementation:
- We need azure subscription enabled account
- DevOps activation must be activated.
Open DevOps portal with enabled SSO
Navigate to Pipeline -> Builds -> Choose the model which was created -> Click on EDIT
Build pipeline will be looking like below screen
We need to use Anaconda distribution for this example to get all the dependencies.
To install environment dependencies, check the link
Use the python environment, under Install Requirements in user setup.
Select create or get workspace select your account subscription as mentioned in below screen
Save the changes happened in other tasks and all those muse be in same subscription.
The entire CI/CD procedure and solution was documented in link
Document Credit: Praneet Singh Solanki
The Main part
I am trying to create a comfortable workflow for myself. I just want to know is there any way to automate deploys from GitHub Repo to Firebase hosting when I commit changes in the repo's main branch at my desktop and push origin to GitHub?
...possibly using "GitHub actions"...
Context and what I've tried
As on 15-Jan-2022 there are two ways I found of deploying website files to firebase hosting.
The de facto firebase cli way, where you use firebase deploy command to push files to firebase hosting.
The automated way using github actions, where the site is deployed automatically every time a pull request is created.
The first one is tedious, the second one is not my cup of cake. Especially because the workflow I want to setup is not too much git oriented, I don't use "pull requests" as far as I understand. I just commit changes directly in the main branch.
(That is so because, I am not a programmer, I just trying to create a blogging site using the Firebase hosting + GitHub repo)
I also referred https://medium.com/firebase-developers/the-comprehensive-guide-to-github-actions-and-firebase-hosting-818502d86c31
Use Deploy to live & preview channels via GitHub pull requests. It will work if you use pull requests to review changes or if you push commits directly to the primary branch.
I would recommend that you use pull requests though. It's a great flow to double check your changes and with the Firebase action it will deploy a temporary preview of the the changes so you can validate everything looks correct.
I have two projects for dev and prod. I want to be able to run a script to copy dev config to prod.
Firebase Remote Config has an API for programatically updating Remote Config. But as far as I can tell, you need to init admin with a project-specific service account. It seems like I would need two admin instances, but I'm not sure that's possible?
I'm wondering if someone has done this before and has an example script. Thanks!
See docs:
https://firebase.google.com/docs/remote-config/automate-rc
There is no Firebase Admin SDK for Flutter, so you'll have to implement this on a different platform that is supported. For a list of these platforms and instructions on setting it up, see the documentation on adding Firebase to a server.
For these platforms that the Firebase Admin SDK targets, you can create multiple instances of the FirebaseApp class, and initialize each of them with different credentials and project configuration. For examples of how to do this, see the documentation on initializing multiple apps.
Using Elastic Beanstalk console, I'm trying to create a super basic Node.js environment using the preconfigured platform and sample application.
When I try to configure more options and then modify Rolling updates and deployments, I only have two options for Deployment policy. Why don't I have the option to select Rolling or Rolling with additional batch? What else would I need to do either within Elastic Beanstalk or in another service (resource) to be able to do this?
Region is N. California and IAM user is full admin.
The Configuration present needs to be changed from the Low cost (Free Tier eligible) to either High availability or Custom Configuration. It would still be good to know how to change after the application has already been created using the Low cost preset.
I am working on a ASP.net MVC4 project where a same project needs to be deployed to many clients on daily basis, each client will have its own domain / sub domain and a separate app pool and db (MSSSQL).
Doing each deployment manually could take at least 1-2 hours if everything goes well. Is there anyway using which I can do this in some automated way?
Moreover, we also need to update all of the apps when a new version is released.. may be one by one or all of them at same time. However, doing this manually could take weeks and once we have more clients then it will not possible doing this update manually.
The update involves, suspending app for some time, taking a full backup of files and db, update application code/ files in app folder, upgrade db with a script and then start app, doing some diagnosis script to check if update was successful or not, if not we need to check what went wrong?
How can we automate this updates? Any idea would be great on how to approach this issue.
As a developer for BuildMaster, I can say that this scenario, known as the "Core Version" pattern, is a common one. If you're OK with a paid solution, you can setup your deployment plans within the tool that do exactly what you described.
As a more concrete example, we experience this exact situation in a slightly different way. BuildMaster has a set of 60+ extensions that rely on a specific SDK version. In our recent 4.0 release, we had to re-deploy every extension because of breaking API changes within the SDK. This is essentially equivalent to having a bunch of customers and deploying to them all at once. We have set up our deployment plans such that any time we create a new release of the SDK application, we have the option to set a variable that says to build every extension that relies on the SDK:
In BuildMaster, the idea is to promote a build (i.e. an immutable object that travels through various environments like Dev, Test, Staging, Prod) to its final environment (where it becomes the deployed build for the release). In your case, this would be pushing your MVC application to its final environment, and that would then trigger the deployments of all dependent applications (i.e. your customers' instances of your application). For our SDK, the plan looks like this:
For your scenario, you would only need the single action, "Promote Build". As I mentioned before, any dependents would then be promoted to their final environments, so all your customer deployments would kick off once that action is run during deployment. As an example, our Azure extension's deployment plan for its final environment looks like this (internal URLs redacted):
You may have noticed that these plans are marked "Shared", which means every extension we have has the exact same deployment plan, but utilizes different variables to handle the minor differences like names, paths, etc.
Since this is such an enormous topic I could go on for ages, but I think that should be sufficient for your use-case if you wanted to try it out.
There are others but you could setup Team Server Foundation to deploy automated builds.
http://msdn.microsoft.com/en-us/library/ff650529.aspx
I find the easiest way to do this from an MVC project is to create a publish profile.
This is done by right-clicking your project selecting publish and then configuring it to your needs.
Then from TFS you create a new build definition, this kicks of a wizard which takes you through it.
There are quite a few options which would be too long to go into for every scenario.
The main change I usually find the most important is to set an MSBuild Argument to deploy with the publish profile.
This can be found at Process > Advanced > MSBuild Arguments.
Once this is configured correctly it's a simple case of right-clicking and queue new build to build and deploy.
You wil need different PublishProfile/Build configuration per deployment environment.
For backups I use a powershell script which can be called manually or from TFS.
You also have a drop folder in TFS which keeps a backup of x many releases.
The datbases are automatically configured via Sql server to backup, TBH I didn't set that up it was a DB admin guy who is also involved with releases.
From a dev testing side I use jMeter (http://jmeter.apache.org/) to run some automated scripts that check that users can login and view certain screens, just to confirm nothing major has gone wrong. However there is usually a testing team to run more detailed tests, again not setup by me.
All of the above will probably take you sometime to setup but in the long run it will literally save you weeks of time over a year.
A free alternative to TFS is http://www.cruisecontrolnet.org/, I have used this in the past too and is pretty good.
You can automate your .Net deployments with Beanstalk, which will give you a way to trigger deployments with a single click, watch progress, manage permissions and see history of deployments. Check out this guide on the topic:
http://guides.beanstalkapp.com/deployments/deploy-dotnet.html
I hope you will find it useful.
P.S. - I work at Beanstalk.