Update a production Sqlite database after publish - sqlite

We are trying to update the schema of an Sqlite database during continuous integration. Here is a simplified version of our scenario.
The deployment script publishes the project.
cd App
App> dotnet publish <args>
That creates the following directory structure.
artifacts/
App.dll
Database.sqlite
The-rest-of-the-publish-output
web.config
App/
Migrations/
Program.cs
project.json
Startup.cs
The deployment script then runts the migrations.
App> dotnet ef
--assembly ..\artifacts\App.dll
--startup-assembly ..\artifacts\App.dll
database update
The issue is that we receive the following message:
Unexpected value '..\artifacts\App.dll' for option 'assembly'
We have also tried other ways to run dotnet ef database update on the compiled project but have not been able to determine how to update the database in the artifacts directory.

In our current scenario, we do it like this on Startup.cs:
if (migrateDb)
{
try
{
using (var serviceScope = app.ApplicationServices.GetService<IServiceScopeFactory>()
.CreateScope())
{
serviceScope.ServiceProvider.GetService<ApplicationDbContext>()
.Database.Migrate();
}
}
catch { }
}
while on development migrateDb resolves to false so we can add/remove and apply migrations as desired. On production that will resolve to true for convenience.
There may be well more valid/appropiate options, usually depends on the context and/or project needs. This is just one way of many.

Related

Flyway Gradle Plugin with Cloud Spanner - No database found to handle url

I'm trying to setup Flyway for Google Cloud Spanner (beta) using the flyway gradle plugin but it gets the error below when executing ./gradlew flywayinfo.
> Error occured while executing flywayInfo
No database found to handle jdbc:cloudspanner:/projects/<my-project>/instances/<my-instance>/databases/<my-db>
build.gradle
plugins {
id 'java'
id 'org.flywaydb.flyway' version '7.13.0'
}
...
dependencies {
implementation(
'org.flywaydb:flyway-gcp-spanner:7.13.0-beta'
)
}
flyway {
url = 'jdbc:cloudspanner:/projects/<my-project>/instances/<my-instance>/databases/<my-db>'
}
The values in the url correspond to my project and instance names.
I've also tried:
using a service account key in the end of the URL
adding the com.google.cloud:google-cloud-spanner-jdbc:2.3.2 JDBC driver dependency (implementation)
I'm behind a proxy but I have set it in my gradle.properties with systemProp.http.proxyHost and systemProp.http.proxyPort (also for https)
Using Flyway CLI and the API programmatically works.
It seems like the error comes from flyway implementation here. Your issue seems somewhat similar with https://github.com/flyway/flyway/issues/3028.
Consider opening a new issue here: https://github.com/flyway/flyway/issues

Entry point not found in .NET Core 2.0 DLL

I couldn't find anything that explains this -- for some reason my .NET Core 2.0 ASP.NET application does not run as a DLL via:
dotnet MyProject.Web.dll
And instead I get the exception:
Unhandled Exception: System.MissingMethodException: Entry point not found in assembly 'MyProject.Web, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
namespace MyProject.Web
{
public class Program
{
public static void Main(string[] args)
{
LoadDependencies();
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseStartup<Startup>()
.Build();
host.Run();
}
private static void LoadDependencies()
{
DependencyLocator.Instance.DefineIfUndefined<IDataProvider, DataProvider>();
}
}
}
It runs fine as a standalone executable (when targeting a "Console Application" in the project's config), but now that I'm trying to deploy to a server that needs it to run via the dotnet command (as a DLL, i.e. "dotnet .\MyProject.Web.dll"), it seems to be having issues. I get the above exception on both my server and my local development box.
I'm kind of blown away that it cannot locate the Main method -- it's declared as static and in Program.cs. Am I missing something?
(EDIT: To clarify, the DLL I'm trying to run against the "dotnet" command is from the target compiling as a "Console Library," since my server is explicitly asking for a DLL, since they will not run executables).
OK, so this is annoying and will hopefully help someone else out.
My host wants to specifically run DLL's thru .NET Core ONLY. They do not allow for executables to be run.
Because DLL's are frequently built as "Class Library" output types on the project, I assumed that this was the workflow necessary to build it. However, I found out that whenever you build your project as a "Console Application," it builds a DLL in addition to an EXE. So, in the above example, MyProject.Web.exe and MyProject.Web.dll are both built when the output type is "Console Application."
MyProject.Web.dll that comes from "Console Application" is different than MyProject.Web.Dll that comes from "Class Library." The one that comes from "Class Library" will NOT have an entry point that can be discovered on it, which will lead to the problem above.
So, if you're getting this error, look for the DLL that ships with your EXE of the same name -- that's the actual DLL you'll want to run in your dotnet console (i.e. dotnet MyProject.Web.dll)

Unable to update database to match the current model. Failing only on self hosting project

The following code:
Database.SetInitializer
(new MigrateDatabaseToLatestVersion<Db, Migrations.Configuration>(true));
using (var C = new Db())
{
Console.WriteLine(C.Usuarios.Count());
}
Works on a console test project but on the other console with self-hosting it fails with the Unable to update database to match the current model... migration error
Obviously the migrations are up to date since the other project runs fine and they both do the same configuration since the database model and the migration configuration are on a separated library
I tracked down the problem to the Newtonsoft.Json library.
The package Microsoft.AspNet.WebApi.Client depends on the version 6.0.4 of this library which seems to have conflicts with Entity Framework.
Just upgrading the Newtonsoft.Json with Install-Package Newtonsoft.Json solves the problem

Can't deploy precompiled, merged webapp to Azure

I'm trying to deploy an ASP.NET web application to Azure. It's hybrid Web Forms, MVC, and WebAPI, and there are a TON of aspx/ascx files, such that they really need to be precompiled or every deploy will render the site sluggish for awhile.
I am trying to deploy via SCM integration with GitHub via kudu, with precompiled views, all merged to a single assembly.
Note that:
Deploy works fine with precompilation disabled.
Deploy works fine from Visual Studio
Build works fine if I copy the msbuild command from the Azure log, replace the relevant paths, and run it locally on my Windows 8.1 machine.
I've set up the Advanced Precompile settings as:
Don't allow precompiled site to be udpatable
Don't emit debug information
Merge all pages and control outputs to a single assembly = AppViews.dll
Here's the .deployment file for Azure
[config]
project = WebSite/WebSite.csproj
SCM_BUILD_ARGS=/p:Configuration=Release;PublishProfile=azure-prod /v:n
You notice I'm sending the verbosity /v to "normal" for extra diagnostic information.
Here is info I get toward the tail of the deployment log:
AspNetPreCompile:
D:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v \ -p D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\Source -c D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir
GenerateAssemblyInfoFromExistingAssembleInfo:
Creating directory "obj\Release\AssemblyInfo".
D:\Windows\Microsoft.NET\Framework\v4.0.30319\Csc.exe /out:obj\Release\AssemblyInfo\AssemblyInfo.dll /target:library Properties\AssemblyInfo.cs
AspNetMerge:
Running aspnet_merge.exe.
D:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\aspnet_merge.exe D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir -w AppViews.dll -copyattrs obj\Release\AssemblyInfo\AssemblyInfo.dll -a
aspnet_merge : error 1003: The directory 'D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir' does not exist. [D:\home\site\repository\WebSite\WebSite.csproj]
Done Building Project "D:\home\site\repository\WebSite\WebSite.csproj" (Build;pipelinePreDeployCopyAllFilesToOneFolder target(s)) -- FAILED.
Build FAILED.
It looks like aspnet_compiler.exe runs, but doesn't do what it's supposed to, which is why the TempBuildDir directory (supposed to be the output of the compiler) does not exist in time for the AspNetMerge target. Contrast that with my system, where that directory DOES in fact exist, containing the marker aspx/ascx/etc. files, static content, a PrecompiledApp.config file, and a whole mess of stuff in the bin directory.
aspnet_compiler.exe has an -errorstack flag but it's not clear to me how I could get MSBuild to add this just via the .deployment file, or even if that app is really even throwing an error.
I could just deploy via Visual Studio, but I would really like to take advantage of the SCM integration so I can just push to my prod branch and let it go. Any suggestions?
I replied on https://github.com/projectkudu/kudu/issues/1341, but copying my answer here in case someone lands here...
Way back, we had found that aspnet_compiler.exe was not working within Azure Websites due to how it dealt with the profile folder. We made a change at the time that's a bit of a hack but got us going: we turned it into a no-op, by pointing HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\aspnet_compiler.exe to our own dummy exe (D:\Program Files (x86)\aspnet_compiler\KuduAspNetCompiler.exe).
But trying it now, it appears to work correctly today, likely thanks to improvements in the Azure Websites hosting environment. So we will try getting rid of this hack and doing a full test pass to make sure it doesn't cause any major regressions. If all goes well, we can get that into production, which should enable those scenarios.
In the short term, you may be able to work around this by having your build script:
copy aspnet_compiler.exe from D:\Windows\Microsoft.NET\Framework\v4.0.30319 into your own site files, but under a different name (e.g. aspnet_compiler2.exe)
convince msbuild to use that one
Note: This GitHub issue on projectkudu will eventually make this solution obsolete, but for the meantime, that issue is filed as Backlog, and this works right now.
Thank you thank you David Ebbo. With this information, I was able to bootstrap my build to work for the short term.
First, I downloaded the aspnet_compiler.exe from the Azure instance using the Diagnostic Console available at https://{WEBSITE_NAME}.scm.azurewebsites.net/DebugConsole and added that to my own repository. This way there's no question about any difference between 32/64-bit, etc. I renamed it to azure_aspnet_compiler.exe in my repository.
Second, the AspNetCompiler task doesn't give you the option to change the tool name. It's hardcoded, but as a virtual property so it's overrideable. So I had to create my own task class, and package it in its own assembly, which I built in Release mode and also included in my repository.
public class AzureAspNetCompiler : Microsoft.Build.Tasks.AspNetCompiler
{
private string _toolName = "aspnet_compiler.exe";
protected override string ToolName
{
get { return _toolName; }
}
public string CustomToolName // Because ToolName cannot have a setter
{
get { return _toolName; }
set { _toolName = value; }
}
}
Next I needed to replace the AspNetPreCompile task in MSBuild, but I couldn't figure out how to do that directly. But that task wasn't doing anything anyway, so why not just run right after it?
I added this to the top of my Website.csproj file to import the DLL containing the AzureAspNetCompiler class. Note that the path is relative to the Website.csproj file I'm editing.
<UsingTask TaskName="AzureBuildTargets.AzureAspNetCompiler"
AssemblyFile="..\DeploymentTools\AzureBuildTargets.dll" />
Then I added this right below it, which is basically stealing the MSBuild target definition of AspNetPreCompile from C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Transform\Microsoft.Web.Publishing.AspNetCompileMerge.targets, with some of the property setting stuff near the top of it left out (because the original task will do that for us anyway.) Just take note of the ToolPath and CustomToolName values at the bottom of the (renamed) AzureAspNetCompiler element.
<PropertyGroup>
<!--Relative to solution root apparently-->
<LocalRepoDeploymentTools>.\DeploymentTools</LocalRepoDeploymentTools>
<AzureAspnetCompilerPath>$([System.IO.Path]::GetFullPath($(LocalRepoDeploymentTools)))</AzureAspnetCompilerPath>
</PropertyGroup>
<Target Name="NoReallyAspNetPreCompile" AfterTargets="AspNetPreCompile">
<AzureAspNetCompiler
PhysicalPath="$(_PreAspnetCompileMergeSingleTargetFolderFullPath)"
TargetPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)"
VirtualPath="$(_AspNetCompilerVirtualPath)"
Force="$(_AspNetCompilerForce)"
Debug="$(DebugSymbols)"
Updateable="$(EnableUpdateable)"
KeyFile="$(_AspNetCompileMergeKeyFile)"
KeyContainer="$(_AspNetCompileMergeKeyContainer)"
DelaySign="$(DelaySign)"
AllowPartiallyTrustedCallers="$(AllowPartiallyTrustedCallers)"
FixedNames="$(_AspNetCompilerFixedNames)"
Clean="$(Clean)"
MetabasePath="$(_AspNetCompilerMetabasePath)"
ToolPath="$(AzureAspnetCompilerPath)"
CustomToolName="azure_aspnet_compiler.exe"
/>
<!--
Removing APP_DATA is done here so that the output groups reflect the fact that App_data is
not present
-->
<RemoveDir Condition="'$(DeleteAppDataFolder)' == 'true' And Exists('$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data')"
Directories="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data" />
<CollectFilesinFolder Condition="'$(UseMerge)' != 'true'"
RootPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)" >
<Output TaskParameter="Result" ItemName="_AspnetCompileMergePrecompiledOutputNoMetadata" />
</CollectFilesinFolder>
<ItemGroup Condition="'$(UseMerge)' != 'true'">
<FileWrites Include="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\**"/>
</ItemGroup>
With this in place, everything works as I would expect it to.

Grails and SQLite

Trying to get SQLite working with grails...stuff I've found on the web seems a little dated - references to ivy and plugins and such, but based on these:
http://stackoverflow.com/questions/1199512/grails-sqlite
http://bigohno.blogspot.com/2010/01/groovy-on-grails-sqlite.html
http://maven-repository.com/artifact/org.xerial/sqlite-jdbc/3.6.17
I've been able to get it working in a test environment...oddly, when I "prod war" my grails app and deploy to tomcat it fails with:
Dialect class not found: hibernate.SQLiteDialect
Here's my setup:
in conf/hibernate added a class for the SQLiteDialect. This .java was taken from here http://code.google.com/p/hibernate-sqlite/
Then in my DataSource.groovy I have:
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory'
}
// environment specific settings
environments {
development {
dataSource {
// SQLite
// !!!see also BuildConfig for Dependancies!!!
dbCreate="update"
url='jdbc:sqlite:C:\\sqlite-shell-win32-x86-3080100\\rss_1.db'
logSql="true"
dialect="hibernate.SQLiteDialect"
driverClassName="org.sqlite.JDBC"
readOnly="true"
}
}
production {
dataSource {
// SQLite
dbCreate="update"
url="jdbc:sqlite:/opt/sqlite/dbs/rss/1/rss_1.db"
logSql="true"
dialect="hibernate.SQLiteDialect"
driverClassName="org.sqlite.JDBC"
readOnly="true"
showsql="false"
}
}
}
and in BuildConfig.groovy I have:
dependencies {
runtime 'org.xerial:sqlite-jdbc:3.6.17'
}
I also jar'd up the .java dialect class and put in in lib - some posts said this helped. I also put sqlite-jdbc-3.7.15-M1.jar in lib.
Now when I run-app in my dev environment it runs fine...but when I deploy to tomcat I get the dialect error.
Is there something special I need to do to the prod environment for the dialect?
Here's how to setup SQLite with Grails:
Download SQLite from http://www.sqlite.org/download.html , extract and save to a directory. You may also want to create directories for your databases.
Download SQLite JDBC jar from https://bitbucket.org/xerial/sqlite-jdbc and put the jar in your grails lib directory.
Download a SQLIte dialect...google search as there are many, but you may reference https://github.com/gwenn/sqlite-dialect or https://gist.github.com/virasak/54436
In grails, create a class in src/java and put your dialect code in.
I also jar'd this class up and put the jar in lib.
Setup your grails datasource, e.g.,:
dataSource {
// SQLite
dbCreate="update"
url="jdbc:sqlite:/opt/sqlite/dbs/rss/1/rss_1.db"
logSql="true"
dialect="SQLiteDialect"
driverClassName="org.sqlite.JDBC"
}
NOTE: Depending on whether your sqlite dialect class is in a package, you may need to prefix the package name to the dialect above (mine was not).
In BuildConfig.groovy, add a dependency to sqlite jdbc, like so:
dependencies {
runtime 'org.xerial:sqlite-jdbc:3.6.17'
}
That's what worked for me!

Resources