Is there a methodology/pattern to redirect Azure ML to non-ACR image repositories like jfrog artifactory for all supported Azure ML functionalities?
Potential solution that we are exploring is to build custom images on Artifactory that does not require Azure ML customization, in that scenario AML only pulls and runs pipelines.
The only supported way in AzureML to build a derived image is ACR (with 2 exceptions like local build and image build on AzureML Compute).
Already built image can pulled from any registry, so if you manage the image separately you can use them as is in AzureML.
Related
I am interested in knowing how can I integrate a repository with Azure Machine Learning Workspace.
What have I tried ?
I have some experience with Azure Data Factory and usually I have setup workflows where
I have a dev azure data factory instance that is linked to azure repository.
Changes made to the repository using the code editor.
These changes are published via the adf_publish branch to the live dev instance
I use CI / CD pipeline and the AzureRMTemplate task to deploy the templates in the publish branch to release the changes to production environment
Question:
How can I achieve the same / similar workflow with Azure Machine Learning Workspace ?
How is CI / CD done with Azure ML Workspace
The following workflow is the official practice to be followed to achieve the task required.
Starting with the architecture mentioned below
we need to have a specific data store to handle the dataset
Perform the regular code modifications using the IDE like Jupyter Notebook or VS Code
Train and test the model
To register and operate on the model, deploy the model image as a web service and operate the rest.
Configure the CI Pipeline:
Follow the below steps to complete the procedure
Before implementation:
- We need azure subscription enabled account
- DevOps activation must be activated.
Open DevOps portal with enabled SSO
Navigate to Pipeline -> Builds -> Choose the model which was created -> Click on EDIT
Build pipeline will be looking like below screen
We need to use Anaconda distribution for this example to get all the dependencies.
To install environment dependencies, check the link
Use the python environment, under Install Requirements in user setup.
Select create or get workspace select your account subscription as mentioned in below screen
Save the changes happened in other tasks and all those muse be in same subscription.
The entire CI/CD procedure and solution was documented in link
Document Credit: Praneet Singh Solanki
Amplify does not support CLI option --profile. It always uses profile specified when application was generated. Different team members use different AWS profiles.
How to change/configure/use profile different than profile used during application generation?
Aim is to publish changes from different computer. Final goal is to use CI server to publish application to different regions.
Amplify does not work like "other" development tools where tool is detached from git. Amplify goes hand in hand with Git and requires initialization after cloning. Running amplify init and choosing existing environment (which is pushed by other developer), it is possible to select different AWS profile.
I know using command line it can be get by running feature:list -i but is there any API/JSON available to fetch this?
You can use jolokia and hawtio to retrieve that information. Quite easily. I believe you can easily add the hawtio repo from the native karaf repos in features (repo-add hawtio). Then you need to install jolokio, hawtio, and the karaf web console. From the karaf webconsole alone you can see a full list of features, but I find the hawtio interface to be a god send.
A REST API can be installed without the need for Hawtio, which uses jolokia for accessing the bundle list under the hood.
The jolokia project provides web applications called agents serving a REST API. For quick experiments you can deploy the war jolokia-war-unsecured into the hot deploy folder of a running karaf instance. This installs a A REST web service at e.g. http://localhost/jolokia-war-unsecured/ which does not require any authentications.
I am trying to setup cloudify in an OpenStack installation using this offline guide.
This guide does not specify much about cloud platform so I have assumed it can be used on OpenStack environment. I am using simple manager blueprint YAML file for bootstrapping .
I have the following questions:
Can I use fabric 1.4.2 with cloudify 3.4.1 ?
If not, I am unable to find wagon-file for fabric 1.4.1.wgn file
Architecture: Can I use CLI inside a network to bootstrap a manager within that network? And this network lies inside OpenStack environment. Can cloudify CLI machine, cloudify Manager and application reside within one network inside openstack? If so, how? Because we would like to test it inside one single network.
(Full disclosure: I wrote the document you linked to.)
Yes you can.
You can find all Wagon files for all versions of the Fabric plugin here: https://github.com/cloudify-cosmo/cloudify-fabric-plugin/releases
Yes.
Usual story, new to artificatory and looking for a jump start.
Can anyone point me at a decent how-to post for using Artifactory (free version) with Jenkins in a deployment pipeline?
I figure I'm likely to want to:
setup several repos for dev thru production (any standards for this?)
have jenkins publish artifacts to the first repo using the artifactory plugin - limiting the amount of builds kept in artifactory.
promote builds from one repo to next as release to next environment - again deleting older builds
I just need a good guide/example to get started...
Did you check the User Guide? It covers all your questions perfectly well. Here are:
Creating repositories (re the standards - the preconfigured repos reflect the standards).
Working with Jenkins
Jenkins Release Management
Keep in mind that all the promotion functionality is available in Pro version only. That includes both simple move operation (not recommended for promotion pipeplines) and extremely powerful release management, based on build integration with Jenkins Artifactory plugin.