We're a little aghast at how time consuming it is to develop syntactically correct ARM templates from scratch.
The Portal helps, but pushes out non-development ready templates (pretty hard to find what the bug is when all the templates use 'name' for the resource name, versus maybe something more verbose like ('microsoftStorageAccountResourceName', microsoftStorageAccountResourceLocation, microsoftStorageAccountResourceTags, etc.).
We understand that there are many ways to deploy -- but if at all possible, we'd like some assurances that ARM is the current preferred way and will continue to be the preferred primary means of scripting deployments via VSTS -- or is it sliding towards a different -- maybe more programmatic -- approach (eg: Powershell, CLI, other).
We're asking this because it looks like we will have to invest significant effort to create a resource library for this organisation (to decrease the need for all projects to become proficient at ARM deployment) -- and would prefer to do it using an approach that will be preferred by developers over the coming years, for maintainability objectives.
Thanks for any insight on which approach to recommend as the best investment.
Templates are going to be around for the foreseeable future... it really depends on whether you want to orchestrate the deployment yourself (imperative deployments using CLI, PS, SDK) or you want ARM to orchestrate the deployment (via templates). Happy to chat in offline if you want to discuss more - email bmoore at microsoft.
Writing this now one year after the original post: The answer to 'ARM Templates are still the preferred deployment mechanism?' probably depends on who you ask. "Preferred" by Microsoft according to their product strategy may be meant differently than preferred by actual users that, well, feel the pain of vendor strategy decisions. I had started with an Azure automation book that used PS scripting only; I was lead (maybe mis-lead?) then to the ARM Template deployment model, mainly by the Microsoft web documentations, but found out that those templates need so much rework that writing a PS script, or even writing an ARM Template from scratch, seems to be a more efficient way to go. In fact, I am confused at the moment about what the 'Best Practice' is, i.e. what method other developers actually use. Is there a community-established opinion on this matter, now in August 2019? Or is it all VSTS / 3rd-party IDEs nowadays?
Related
We are evaluating and looking for a workflow engine which support .NET-Core and I'd really appreciate the community input. I would like to hear on the suggestion based on your guys implementation stories.
My main evaluation criteria, so far, are below;
open source and OEM friendly license
production installations (success stories are a great help)
technical support available
open standards support - BPMN
dynamic creation/assembly of the workflow based on input
embeddable
Currently I am evaluating Elsa, Workflow Core, Argo, and Airflow. Elsa seems like a good candidate as well but never used it.
Do you guys have any successful deployments on Elsa workflow engine?
Full disclosure: I am the project lead of Elsa, but I will try and be as objective as I can.
Elsa does not currently support BPMN, so if this is a hard requirement then Elsa might not be suitable for your project. At least not until it implements BPMN in the future.
As for technical support, there is no official paid support available as of yet, but the community is very friendly & helpful, though still relatively small.
Dynamic creation based on input is possible since you can programmatically define workflows. But you cannot update workflows while it executes (which would be more or less similar to being able to update your C# program statements as your program runs). Not sure if this is what you are looking for or not?
Other than that, Elsa is OEM friendly, runs in production successfully at several companies that I know of and is embeddable.
We're in the initial stages of a large project, and have decided that some form of automated UI testing is likely going to be useful for us, but have not yet sorted out exactly how this is going to work...
The primary goal is to automate a basic install and run-through of the app, so if a developer causes a major breakage (eg: app won't install, network won't connect, window won't display, etc) the testers don't have to waste their time (and get annoyed by) installing and configuring a broken build
A secondary goal is to help testers when dealing with repetitive tasks.
My question is: Who should create these kinds of tests? The implicit assumption in our team has been that the testers will do it, but everything I've read on the net always seems to imply that the developers will create them, as a kind of 'extended unit test'.
Some thoughts:
The developers seem to be in a much better position to do this, given that they know control ID's, classes, etc, and have a much better picture of how the app is working
The testers have the advantage of NOT knowing how the app is working, and hence can produce tests which may be much more useful
I've written some initial scripts using IronRuby and White. This has worked really well, and is powerful enough to do literally anything, but then you need to be able to write code to write the UI tests
All of the automated UI test tools we've tried (TestComplete, etc) seem to be incredibly complex and fragile, and while the testers can use them, it takes them about 100 times longer and they're constantly running into "accidental complexity" caused by the UI test tools.
Our testers can't code, and while they're plenty smart, all I got were funny looks when I suggested that testers could potentially write simple ruby scripts (even though said scripts are about 100x easier to read and write than the mangled mess of buttons and datagrids that seems to be the standard for automated UI test tools).
I'd really appreciate any feedback from others who have tried UI automation in a team of both developers and testers. Who did what, and did it work well? Thanks in advance!
Edit: The application in question is a C# WPF "rich client" application which connects to a server using WCF
Ideally it should really be QA who end up writing the tests. The problem with using a programmatic solution is the learning curve involved in getting the QA people up to speed with using the tool. Developers can certainly help with this learning curve and help the process by mentoring, but it still takes time and is a drag on development.
The alternative is to use a simple GUI tool which backs a language (and data scripts) and enables QA to build scripts visually, delving into the finer details of the language only when really necessary - development can also get involved here also.
The most successful attempts I've seen have definitely been with the latter, but setting this up is the hard part. Selenium has worked well for simple web applications and simple threads through the application. JMeter also (for scripted web conversations for web services) has worked well... Another option which is that of in house built test harness - a simple tool over the top of a scripting language (Groovy, Python, Ruby) that allows QA to put test data into the application either via a GUI or via data files. The data files can be simple properties files, or in more complex cases structured (something like YAML or even Excel) data files. That way they can build the basic smoke tests to start, and later expand that into various scenario driven tests.
Finally... I think rich client apps are way more difficult to test in this way, but it depends on the nature of the language and the tools available to you...
In my experience, testers who can code will switch jobs for a pay raise as developers.
I agree with you on the automated UI testing tools. Every place I've worked that was rich enough to afford WinRunner or LoadRunner couldn't afford the staff to actually use it. The prices may have changed, but back then, these were in the high 5-digit to low 6-digit price tags (think of the price of a starter home). The products were hard to use, and were usually kept uninstalled in a locked cabinet because everyone was afraid of getting in trouble for breaking them.
I worked over 7 years as an application developer before I finally switched to testing and test automation. Testing is much more challenging than coding, and any automation developer who wants to succeed should master testing skills.
Some time ago I put my thoughts on skill matrices in a couple of blog posts.
If interested to discuss:
http://automation-beyond.com/2009/05/28/qa-automation-skill-matrices/
Thanks.
I think having the developers write the tests will be of the most use. That way, you can get "breakage checking" throughout your dev cycle, not just at the end. If you do nightly automated builds, you can catch and fix bugs when they're small, before they grow into huge, mean, man-eating bugs.
What about the testers proposing the tests, and the developers actually writing it ?
I believe at first it largely depends on the tools you use.
Our company currently uses Selenium (We're a Java shop).
The Selenium IDE (which records actions in Firefox) works OK, but developers need to manually correct mistakes it makes against our webapps, so it's not really appropriate for QA to write tests with.
One thing I tried in the past (with some success), was to write library functions as wrappers for Selenium functions. They read as plain english:
selenium.clickButton("Button Text")
...but behind the scenes check for proper layout and tags on the button, has an id etc.
Unfortunately this required a lot of set up to allow easy writing of tests.
I recently became aware of a tool called Twist (from Thoughtworks, built on the Eclipse engine), which is a wrapper for Selenium, allowing plain English style tests to be written. I am hoping to be able to supply this to the testers, who can write simple assertions in plain English!
It automatically creates stubs for new assertions too, so the testers could write the tests, and pass them to developers if they need new code.
I've found the most reasonable option is to have enough specs such that the QA folks can stub out the test, basically figure out what they want to test at each 'screen' or on each component, and stub those out. The stubs should be named such that they're very descriptive as to what they're testing. This also offers a way to crystalize functional requirements. In fact, doing the requirements in this fashion are particularly easy, and help non-technical people really work through the muddy waters of their own though process.
The stubs can be filled in via a combination of QA/dev people. This allows you to CHEAPLY train QA people as to how to write tests, and they typically slurp it up as it furthers their job security.
I think it depends mostly on the skill level of your test team, the tools available, and the team culture with respect to how developers and testers interact with each other. My current situation is that we have a relatively technical test team. All testers are expected to have development skills. In our case, testers write UI Automation. If your test team doesn't have those skills they will not be set up for success. In that case, it may be best for developers to write you UI automation.
Other factors to consider:
What other testing tasks are on the testers' plate?
Who are your customers and what are their expectations related to quality?
What is the skill level of the development team and what is their willingness to take on test automation work?
-Ron
95% of my time I program ASP.NET (MVC) web sites.
Should I care about MSBuild?
We use MSBuild with CruiseControl.Net to manage the builds of most of our big ASP.NET projects. For every commit of one member of the team, a build is launched. It helps us detect
quickely incompatibilities before moving a feature to "staging" or "production".
I think it is really usefull when working with a team on the same ASP.NET project or if you are working alone on a big project.
That depends on your development environment.
If you have other folks that do deployment of your systems, and they take care of the build and deployment environment, then MSBuild probably won't be necessary for your work.
On the other hand, if you need to configure the build script to understand special situations that your code comes up with, then you will definitely need to understand MSBuild scripts.
Even for a one-man shop, it's a useful tool to know, especially if you are configuring a continuous integration server like Hudson.
No. Until you have to.
Its not absolutely necessary to know MS Build, but it is useful to know.
It might not be needed for all kind of projects, but it is extremely useful when you are working on a huge code base with automated custom build solution/ nightly build/developer builds so on and so forth.
It's unlikely, unless you choose to use it, or you start to make use of Team Foundation Server's Team Build.
Your development processes need to get to a certain complexity before automated builds really deliver their true value and/or if you find need for automatic deployment (including database changes if applicable).
The coming Visual Studio 2010 is going to make it far easier to use, but for now it retains a fairly steep learning curve which you can avoid by using alternatives, or commercial products (e.g. Visual Build Pro, Final Builder etc).
The nice thing is that it is part of the .Net framework, so it's already available as long as you have the framework installed (which it probably is).
So, in short, not really. It's something very useful and powerful though, setting up deployments using MSBuild can be very, very useful.
What should a developer know about MsBuild?
Every developer should know it exists and it's basic capabilities. If know it exists you won't duplicate its features and will know what it can do for you, when you need it.
Minimum:
As an exercise, build your project through the command line: msbuild myproj.sln
Know the role of continuous integration
A little more than minimum:
Hack your csproj (or vbproj) with a message task, so it outputs something during clean.
All done. When you need to know more, you'll figure it out.
My company (a large organization) is developing a "road-map" for evolving their rather old, tangled confederation of systems to an SOA model. A few people are pushing hard for using Websphere Integration Developer and Websphere Process Server as the defacto platform for developing future applications...because they feel IBM is a stable vendor, the tools are made for the enterprise, they drank the "business agility" BPEL kool-aid, etc.
Does anyone have positive or negative thoughts on this platform? Do the GUI tools help eliminate monotonous/redundant coding...or just obscure things and make things harder to maintain? Basically, do the benefits justify the complexity?
My experience with the IBM Java tool set is pure pain. Days to install lots of different versions of different components all incompatible with each other, discover a bug in component A get told to update to see if it fixes, updating component A breaks component B and C, get told to update these etc.
I find Eclipse with out the IBM extensions far more stable and quicker and provides more features (as its stable versions are a couple releases ahead of WID/RAD).
I would advise against going the IBM way for development tools. As for process server I have less experience but the people in my team using it seemed to enjoy it as much as I enjoyed WID. not a lot.
So far I havent been impressed by any tools with the "SOA" and/or "BPM" labels on them. My "roadmap" would be very very iterative to see some results with the archetecture as fast as possible while trying to grab some of the easy fruits. That way you gain your feel for what works for you and your people.
I would never let any vendor push me anywhere in the "scuplturing" of the architecture.
I agree with other users complaining about WID. The only reason we are using WID is that a decision was made a while back to use IBM products across the board by our sales department.
That's right, our sales department made the decision to use IBM products.
Development has been painful and frustrating. We have lots of stability problems with Process Server, sometimes it doesn't want to start or shutdown properly. Yeah you can easily draw processes in the IDE, but most any toolset provides that functionality these days. It is nothing special or unique to WID or IBM. IBM is a few iterations behind mainstream.
There are plenty of open source implementations out there that offer great support. Checkout JBoss or RedHat, they are pretty good. If that doesn't float your boat, you can always use Apache tools.
Walter
Developers don't choose WID, WMB, or WPS. Managers do, because IBM is a "stable vendor".
Look at JBoss, or K.I.S.S.
WID/WPS is actually pretty simple. The original intention was for analysts and business people to "compose" services (DO NOT LET THEM DO THIS!) so the UI is simple and easy.
Most of the work will be in defineing and implementing the back end services which depending on the platform will mostly involve wrapping existing code in SOA service.
The most important thing to bear in mind is that SOAP is technoligy and SOA is an architecture and a state of mind.
There is a zen to a succesful SOA implementation. Its all about "business services", if you have a service that you cannot describe to a business user in less than six words you have done it wrong! Ideally the service name alone should be enough to describe the functionality of the service.
If you end up with a service called "MyApp.GetContactData" described as "get name, addresses tel fax etc." then you are there. If You have a service called MyAppGetFaxNoFromOldSys" described as "Retrieve current-fax-nmbr from telephony table in legacy system" you are doomed!
Incidently most of the Websphere tooling for WS* is pretty nice. But I would recommend the very wonderful SAOPUI tool from http://www.eviware.com which is very good for compsing/reading WSDL based messages and also function as a useful test client or server.
Do the GUI tools help eliminate monotonous/redundant coding...or just obscure things and make things harder to maintain? Basically, do the benefits justify the complexity?
As a Developer, I find the tools at varying levels of being bug free. 6.0.1 was a pain, 6.2 is so much better. But once you develop with the tool, there is minimal effort to maintain it. I develop in hours what java developers take days to do. It is also easy to maintain as changes can be made very quickly. I cannot answer your question from the perspective of an architect or a Manager but i would agree with comments of some others here.
I am evaluating WF for use in line of business applications on the web, and I would love to hear some recent first-hand accounts of this technology.
My main interest here is in improving the maintainability of projects and maybe in increasing developer productivity when working on complex processes that change frequently.
I really like the idea of WF, however it seems to be relatively unknown and many older comments I've come across mention that it's overwhelmingly complex once you get into it.
If it's overdesigned to the point that it's unusable (or a bad tradeoff) for a small to medium-sized project, that's something that I need to know.
Of course, it has been out since late 2006, so perhaps it has matured. If that's the case, that's another piece of information that would be very helpful!
Thanks in advance!
Windows Workflow Foundation is a very capable product but still very much in its 1st version :-(
The main reasons for use include:
Visually modeling business requirements.
Separating your business logic from the business rules and externalizing rules as XML files.
Separating your business flow from your application by externalizing your workflows as XML files.
Creating long running processes with the automatic ability to react if nothing has happened for some extended period of time. For example an invoice not being paid.
Automatic persistence of long running workflows to keep resource usage down and allow a process and/or machine to restart.
Automatic tracking of workflows helping with business requirements.
WF comes as a library/framework so most of the time you need to write the host that instantiates the WF runtime. That said, using WCF hosted in IIS is a viable solution and saves a lot of work. However the WCF/WF coupling is less than perfect and needs some serious work. See here http://msmvps.com/blogs/theproblemsolver/archive/2008/08/06/using-a-transactionscopeactivity-with-a-wcf-receiveactivity.aspx for more details. Expect quite a few changes/enhancements in the next version.
WF (and WCF) are pretty central to a lot of the new stuff coming out of Microsoft. You can expect some interesting announcements during the PDC.
BTW keeping multiple versions of a workflow running takes a bit of work but that is mostly standard .NET. I just did a series of blog posts on the subject starting here: http://msmvps.com/blogs/theproblemsolver/archive/2008/09/10/versioning-long-running-workfows.aspx
About visually modeling business requirements.
In theory, this works quite well with a separation of intent and implementation. However, in practice, you will drop quite a few extra activities on a workflow purely for technical reasons, and that sort of defeats the purpose as You have to tell a business analyst to ignore half the shapes and lines.
Related question: When to use Windows Workflow Foundation? My answer there:
You may need WF only if any of the
following is true:
You have a long-running process.
You have a process that changes frequently.
You want a visual model of the process.
For more details, see Paul Andrew's
post: What to use Windows Workflow Foundation for?
Please do not confuse or relate WF
with visual programming of any kind.
It is wrong and can lead to very bad
architecture/design decisions.
So, if you have such requirements, then WF is a good candidate. Of course it is relatively complex, but mention that the problems that is trying to solve is also complex (and sometimes very complex). IMHO, it is very complex for example to dehydrate/rehydrate objects that have event handlers attached (with events that can be triggered when the object is not in memory).
I can not judge what you mean by "small to medium-sized project", but in general I would say that if your project has at least two requirements from the above list, then you can consider WF as a solution.
We've used WF in a large-ish SharePoint application and I can say it's OK. It has lots of power and flexibility. and, as Kevin mentions, once you grok the underlying concepts of workflows, you can do pretty much anything you want with it.
On the other hand, it has some really serious issues, like lack of versioning, which can really hurt your application in the future. We've been forced to deploy up to 3 parallel versions of the same workflow named xxx-v1, xxx-v2 and xxx-v3 to keep older instances running and have new instances use the updated versions. A real pain in the ass. Oh, and there are also some really non-intuitive concepts in there (correlation tokens, wtf??)
We had a project at work that I was involved in using Workflows.
The idea (from management), was that us programmers would write the Workflow Activities along with the "engine" and framework. Then non-programmers would take care of all the rest by compiling their own Workflows into dlls which the engine would automatically load.
Management was sold on this idea of non-programmers using Workflow to help develop software, and it was pretty much a complete waste of time. The problem we were trying to solve with this project was relatively complex and we knew from the very beginning that the software would have to be modified almost constantly (its calculations were dependent on other companies and governements).
The end result was that we were unable to make the Workflow modules generic enough for anyone else to use. So the programmers were the ones who were forced to work with the Workflows, and all the Workflows did was get in our way.
I've been using Workflow 4.0 for the last few months and although mostly impressed, I've found it extremely hard to learn.
For the most recent version (that comes with .NET 4.0 RC), there is next-to-no documentation on the web, in any books or no training courses available. I've only found articles relating to the now defunct 3.0 version. Even the MSDN documenation is light on the ground.
The workflow designer is not as intuitive as it should be by any means so learning is very hard. I've had to rely on answers from a single person on StackOverflow (thanks by the way Maurice!) - and I would be stuffed without his help.
So in summary, I think it has potential but you would be quite mad to learn it yet - wait for more training, documentation and books otherwise you will be going into it blind!
Last year we completed a working application with WF, now used as the backbone of an unbelievably huge system which is used by a very big bank for its mortgage process. The pe process has many steps starting from customer application to approval of credit.
Although it was a success, there were so many problems and crisis all along the way. And it wont worth the trouble for any smaller size projects.
I consider MS WF as a low-level workflow library rather than a fully fledged enterprise workflow product such as K2. It will enable you to build a workflow enabled application, but is not in itself a workflow application. My experiance of it in this capacity has been positive, although we have had to build a lot of our own infrastructure around it (a pub/sub framework, a worlkflow lifetime manager etc). A lot of the documentation out there is fairly simplistic and does not cover building up an enterprise workflow application based on MS WF.
Hard to learn. Quite flexible. Not to be confused with a visual tool for end users, only for programmers. Not sure if I like the dependancy property approach.
It really depends on what you want to do with it. I've only used it a little, but compared to more mature products like MetaStorm (I know technically it's a BPM, but there is still a workflow component), Process Choriographer and IBM MQ workflow, there's no comparison. It's just not mature enough. On the other hand it's free where the others are not and can probably get the job done. I don't know if I would place a multi-million dollar operation on it, but with smaller ones, I'd give it another shot. The real hurdle you are going to face is the change in thought process it requires. If you don't have developers that have worked with state systems before that can be a real hurdle.
Brian, I can't reply to your comment, but anyway, by versioning i mean making changes to the underlying code of the workflow without breaking already running instances, and gracefully applying updates to existing workflows. I'm not sure about 'stock' WF, but at least in SharePoint environment there's no concept of workflow versions so new versions have to be deployed as completely different workflows which becomes a maintenance nightmare.
This has nothing to do with 'rehydration', rehydration is the process by which you bring a 'dormant' workflow back to activity after some event or change in state. That is handled transparently by the workflow runtime.
WF ist integrated into SharePoint (WSS 3.0), and i have created quite a few workflows for various SharePoint-Websites, so i can tell about my experience of WF in SharePoint. Compared with other workflow-frameworks WF scores well. It's stable (i haven't experienced any mysterious errors), workflows are fairly easy to design (thank to the workflow-designer in Visual Studio) and you can use not only sequential but also state-machine workflows.
It's not perfect, of course, and a developer will definitly need some time to understand the concept (of i.e. the Activity Model); but it's definitely useable - even for "small tasks".
Never tried WFF, but I remember reading this article about WFF by Leon Bambrick where he basically says the whole genre of software development tools is nonsense. Might help you decide one way or the other.