I am deploying a Spring Boot application to AWS Elastic Beanstalk using AWSCLI. I want to put an external application.properties file containing customer specification configurations (database credentials, etc.) in the same directory of the application. The application should pick up this properties file. How can I accomplish that? Are there any alternatives?
Spring Cloud Config
This project allows you to use an external, centralized configuration repository for one or more applications. You don't need to rebuild your application if a property changes. You can simply change the property in your configuration repository and even push the changes to all of your applications.
See this Getting Started Guide.
This above approach is what I would recommend for the scenario you described. I would not bother with having a separate directory on the application server for your configuration files. Spring Cloud Config is a great approach as it solves the problem you described and a few more.
You can alternatively specify them in an application-prod property like
server.port=5000
spring.datasource.url=jdbc:mysql://${RDS_HOSTNAME}:${RDS_PORT}/${RDS_DB_NAME}
spring.datasource.username=${RDS_USERNAME}
spring.datasource.password=${RDS_PASSWORD}
spring.jpa.hibernate.ddl-auto=create
and specify an environment variable called SPRING_PROFILES_ACTIVE with the value prod making the beanstalk smart enough to pick values from out there.
We have an site made up of several hundred ASP.NET 4 web apps. Currently our production config is specified in config transforms and resides along with the source code for each application. We deploy to staging and production environments using WebDeploy packages.
What are the options for securing the configuration?
I can think of the following:
encrypt the config files and build tool/s that use certs to decrypt in prod
store and deploy the configuration transform files separately
extend the system.configuration to read config from env. vars (easy for appSettings, more involved for custom config sections).
Is there a standard approach for this and perhaps some tools I am unaware of?
We have 3 components that we want to map to our domain like this:
a static website: example.com
an asp.net app used for account settings: example.com/account
some .net webservices used in other apps: example.com/web
We want to use AWS's Elastic Beanstalk to host the entire structure, preferably in the same environment.
And if we were to have all the components in the same Visual Studio project, it would be quite easy. But the static website is prone to a lot of changes and we don't want those changes directly pushed to production to mess with the other 2 components.
The best scenario for us would be to have 3 separate VS projects, each of them publishing separately and independently to the EBS environment:
the static websites VS project should be able to publish in the root
w/o interfering with the /account and /web folders
the ASP.NET user account VS project should be able to publish
directly to /account without messing with the root files
the .NET webservices VS project should be able to publish directly to
/web without messing with the root files
My questions:
Is this possible? And if there is a way, how should we deal with
versioning?
Should we use other AWS services that can help us reach our goal architecture?
Our fallback plan is to host the static website in S3 and use subdomains for the apps: account.example.com and web.example.com.
For your requirements I would suggest using S3 for static website, and Elastic Beanstalk for other 2 components.
Here one more decision you have to take is either to use a single Beanstalk Environment or Multiple. This decision you can take based the requirements for scalability of each service running in Benastalk.
Another important service you can use is AWS Cloudfront, to be used infront of all 3 components, which can be configured to do the url routing for example.com/ and also cache static content in edge locations. This will also help to setup SSL certificates if needed (Which is free for CloudFront). Also if you use Cloudfront you don't need subdomains, which works best for web apps since it don't need CORS with preflight requests.
I have an application that is installed at several different client's servers. They each have different web.config files and different virtual folders. At the moment I am compiling, manually copying over, setting up IIS, changing web.config and adding virtual folders for each install and also again when updating.
I simply don't know how to deploy using something like Web Deploy or Deployment Package that will let me create different config files or how to manage virtual folders (I would assume I would simply deploy empty folders and would still have to do this part manually). I can handle setting up IIS and virtual folders from the start but I want each client to be able to download new versions and install them without my input (as some Clients are funny about remote access).
You can setup build configurations for each environment. Typically, you get Debug and Release out of the box. I like to replace those with Development, Staging, and Production configurations, which allows a different web.config per environment.
Of course, when publishing, you still have to make sure you select the correct configuration.
I'm not a fan of Web Deploy and other schemes because web-servers, unlike desktops, all tend to have a unique configuration.
In my case, all of our web applications are deployed with custom-written VBScripts (much more pleasant than Batch files and without needing to relearn PowerShell). VBScript (with its default COM object library) provides a compelling platform for writing deployment scripts. And if you can't stand the syntax you can use JScript using the same tools. Bonus: Visual Studio still provides Windows scripting IntelliSense despite it not being an advertised feature).
My most recent deployment script is simple: it's a VBScript (invoked by VS's Post-build command-line) that uses 7-Zip to pack up the web application's files, then generates an ftp.exe batch file (then runs ftp.exe itself) that uploads the files to the server, it also generates an uploads a file called "Unpack.cmd" which calls 7-Zip to extract the files into the right place. The only manual step is executing Unpack.cmd on the server, but that can be done with Remote PowerShell, for example.
Why do you need to customise your IIS configuration separately from the application? Can't you put everything in your web.config file under <system.webServer>?
Currently we deploy compiled ASP.Net applications by publishing the web site locally and emailing a zip file to the system administrator with a (usually) lengthy set of instructions for deployment. This is because the first time we deployed an ASP.Net application to a customer the dev and test IIS instance were the same, and we were unable to deploy the site twice to the same machine. This set the tone for deployment on all subsequent projects.
I am now evaluating our deployment methods and am looking specifically at the built in deployment tools; specifically I'm looking at custom installation tasks and using as much of the standard installer functionality as I can (mostly the user interface).
Secondly, I'm looking at merging deployments and automatic updates.
How do you go about deploying sofware in your organisation? What tools do you use, and what problems do you come across most frequently?
We have dedicated DEV, TEST, STAGE, and PRODUCTION servers.
We also have a dedicated build machine which runs Cruise Control.
Cruise Control is configured for a Continuous Integration build, which runs after code is checked in. It is also configured for separate Development, QA, Stage, and Production tasks.
To deploy to development, the code is first retrieved from SVN and built, then the "Precompiled Web" folder is copied to the development web site, and the web service project is copied to the development application server. Cruise Control is also configured to "tag" the source code before the build starts so we can reproduce the build at a later time, or branch from the tag if we need to do a hot fix.
To deploy to QA, the files are copied from the development machines to the QA machines.
Likewise, to deploy to Stage the files are copied from the QA machines to the Stage machines.
Finally, to deploy to production, the files are again copied from the Stage machines to the Production machines.
To configure each environment, we have a custom tool which is part of each environment's Cruise Control task that modifies connection strings, "debug=true|false", "customErrors=Off|RemoteOnly", and other environment-specific settings.
So each environment can be deployed with a button push from the Cruise Control dashboard.
One caveat is that we currently have the production database password configured in the Cruise Control config file...it would be nice move it elsewhere!
Lastly, let me add that even though our production machines are in a dedicated hosting facility, the servers are accessible from our Cruise Control machine, which makes it very easy to do a production deployment. The only manual step is to encrypt the web.config files and remove the "AppOffline.html" file that Cruise Control puts up.
Let me know if this helps, or if you have any questions.
Thanks!
A couple things that I have done is the following:
1) Use a Web Deployment Project in order to compile and clean the build as well as handing web.config section replacement if the config changes between environments.
2) Use NAnt to do all of the building, archiving, and copying in a repetitive manner.
The Web Deployment Project ends up creating a MSBuild file which can be used in place of NAnt; however, I came from a Java background and used Ant all of the time so NAnt is my preference in .Net. If you add in the NAnt Contrib tasks, you will be able to deploy not only the files but also handle items such as your source control (incase it is not part of the default tasks) and Sql Script Execution for changes.
Currently I use both of the options together. I have my NAnt build file call the Web Deployment Project through MSBuild. With the configuration manager setup for each environment, it allows me to manage the web.config section replacements automatically and still have fairly decent control over my copying and archiving of a release.
Hope this helps.
We use web deployment projects, and the VS 2008 projects to create an .msi from the output of the webdeployment & other projects. A normal windows app called 'setup' is used to do a lot of the db creation and preliminary stuff, rather than trying to customise the setup projects with custom steps. It is a lot easier to do this yourself than trying to customise the MS code. This windows app then calls the correct .msi files that the user needs.
Team foundation build runs every evening to rebuild the solution and copy everything to a 'Release CD' directory which anyone can access and do testing on the latest 'release'.
To be honest TFS build is a bit overboard for a small team like ours, and I only use it because its what I am used to.
In a previous company we used this http://www.finalbuilder.com/ and I can recommend it for ease of use and for the amount of software supported.
1) Build project with MSBUILD
2) FTP files to Production Environment
3) Copy / Paste manually to each web server
For intranet sites, we use CruiseControl in conjunction with SVN to have the site rebuilt automagically.
Theoretically you could extend this model over a VPN if you could map a drive remotely to a client's intranet. Or a more quick and dirty solution might be to use a tool like SyncBack to sync the remote folder containing the compiled DLLs for the site.
Deploy Web Applications Using the Copy Web Tool
Text from Microsoft Training Kit Book Web Based Development
Web Setup Projects are useful if you are providing a Web application to many users (for example, allowing people to download the application from the Web and install it). If you are responsible for updating a specific Web site for your organization, it’s impractical to log on to the Web server and install a Windows Installer package each time you make an update. For internal applications, you can edit the Web application directly on the Web server. However, changes you make are immediately implemented in your production Web application, and this includes any bugs that might be there. To enable yourself to test a Web application, you can edit a local copy of the Web application on your computer and publish changes to the production Web server using the Copy Web tool. You can also use the Copy Web tool to publish changes from a staging server to a production Web server, or between any two Web servers. The Copy Web tool can copy individual files or an entire Web site to or from a source Web site and a remote Web site. You can also choose to synchronize files, which involves copying only changed files and detecting possible versioning conflicts in which the same file on both the source and remote site have been separately edited. The Copy Web tool cannot merge changes within a single file; only complete files can be copied.