I'm working on a node.js project and I would like to use some kind of database to store application data. I've searched for various DBMS and I've selected SQLite because I want my project to be able to run without installing heavy DBMS and because I want it to run both on linux and windows. however, all modules that we found, that connect node with SQLite are either very old, or cannot be installed due to bugs.
so, I want to ask if nowadays, 2012, there is a completely functional node.js module for SQLite.
I did a project last July (2012) in Node.JS v0.8.8, using the node-sqlite module (see https://github.com/developmentseed/node-sqlite3) and it worked just fine.
And, I successfully deployed and used this module on Heroku. However, the deployed portion of the project only involved reading out of the SQLite database – not writing. I suspect that writing into a SQLite deployed on Heroku would cause some issues, because it involves modifying the SQLite file in the file system, which AFAIK Heroku doesn't let you do.
Related
I will like to deploy an app that uses sqlite to Heroku. Googling for how to do this, I come across the official documentation of sqlite on hereku here
And the amazing thing is that, the documentation seems to be suggesting you should not use sqlite on heroku and then went further to tell you how to use Postgres instead!!!
Is it really impossible to use sqlite on heroku and have your data not transient?
Heroku has a database offering of its own, based on the Postgres database so it is necessary to switch so as to avoid the file-based SQlite. To connect to a Heroku database, you will need to use the Heroku CLI. That is the recommendation, as you have noted.
You can run heroku addons:add <your database choice, say the free tier one for example>. It is as simple as that.
If you try to run heroku config:get DATABASE_URL you will notice that a postgres://<> (now postgresql) has been created for you.
I've played around with SQLite on my Windows desktop before, but I've never used it in a web context and their web documentation is maddenigly unspecific about things like this. Essentially, I have free student webspace from my school (a /public/html/ directory) and I'd like to use it for a small data-driven web project. SQLite is my only (free) option and I'm wondering how to... "install" it in this context? More specifically, is it safe just to throw the files in the html directory and call it a day?
If it helps, I'm pretty sure it's Linux environment (they run Apache).
See if it's already installed. Try the following in a bash session:
touch test.db
sqlite3 test.db
If not, you'll have to download and install sqlite3.
Useful tip: if you've been trying to run the 'sqlite' binary, it probably doesn't exist. It's called 'sqlite3'.
SQLite is an embedded database; it is typically not installed, but compiled into whatever program is used to access the database.
All the typical server-side extension languages (PHP, Ruby, Python, Perl etc.) have support modules for SQLite.
You have to check which one of those are installed on the server, and whether SQLite support has been enabled for them.
Does anyone have any best practices around deploying database changes in an EF 4.1 code-first solution? I know MS does not currently support database migrations for EF 4.1, but obviously people are going to need to do this from time to time.
Thanks
Once you deployed database to production you must do incremental changes. It means that before you deploy next version you must prepare two databases in your dev box:
Database with DB schema currently deployed in production - you should be able to get this from source control so always correctly label / tag your production releases
Database with new DB schema
Once you have two databases you can use some tool to make difference SQL script for you. I have experience with both:
Visual Studio 2010 Premium / Ultimate Database tools
Red Gate SQL Compare
These tools are for SQL server.
Once you have difference script you can test it on your dev box. Be aware that some more complicated changes cannot be created by difference script and require you to create custom migration script for example with storing data existing data in temporary tables while refactoring real table. Also if you use some new seed data in your new version you must add them manually into script or use Data Compare tools (also offered by both products).
After that you can plan outage of your production application, database backup and running upgrade script.
I'm accustomed to writing desktop applications in C#.NET. Where I have a nice little solution folder which is also under version control. So at any time, on any computer I can check out whatever version of my software I want run the compiler and have a working copy of my program.
Now I'm looking into developing websites, where the files and data are a lot more dispersed. I'm using ASP.NET, but really my question is more general and could apply to any website framework.
I'm trying to understand the proper work-flow between developing my website, a version control server, and the actual live website that users will see. Obviously this can vary a lot depending on the type and scale of the website, but I'm only considering a pretty simple site. I'm just getting started with this stuff.
The diagram below shows my current idea. All the source files for the site would be stored on a subversion server, which I would check out onto my local computer. My local computer would have a local database which I would use for development of the site. Next I would publish to a test version on my hosted server, which would point to a separate test database. This test database may periodically be replaced by a copy of the live database.
If all goes well I would then publish to a beta version of the site which points to the live data. Users could then check out the beta version to provide feedback. Finally if there are still no problems the source files for the live site would be updated.
Does this make sense? Does anyone have any comments on how this could be improved? Are there any good books or online tutorials available on developing these kind of workflows?
Also the one thing that I'm really not sure about is how to manage changes to the actual schema of the database. I figure with each version I could generate a SQL script that can be use to update the Test and Live databases on the host. However, I'd also like to be able to easily setup a new database for any version of my site with out having to run every update SQL script for every version up to the desired version. Is the best solution to use an ORM like NHibernate or Subsonic so I could always generate my database schema directly from my code?
I currently use a workflow very similar to this. It's not flawless, but for the most part it works well. It sounds like you pretty much have the important parts figured out.
Where I work we have a powerful web server. Our subversion repos also live on this server. For myself, as a LAMP developer, I do all of my development on my local Linux machine (or VM) with a local MySQL database, operating on a local working copy.
At all times I maintain two primary branches of the application: A Dev branch (trunk) and a Live branch (which reflects the current production version). My repo looks like this:
/repo/trunk/ [My current active development version]
/repo/archive/ [All other versions not in active development live here]
/repo/archive/2010.12/ [This happens to be the current production version]
On the web server we maintain three separate instances of the software: Live (or Production), Beta, and Dev. The following should illustrate how we use these versions to
Dev - Always points to our development version at `/repo/trunk'. Uses a non-vesioned config file to point to the development database. Development is never physically done here however; instead we work on our local machines, where our working copies point to the Trunk, and do all our development testing on our local machines. After several commits from multiple developers, though, it's important to test on the Dev server to make sure we're on the right track and no one is breaking someone else's changes.
Live - Always points to the most recent stable production version. In this case, it points to /repo/archive/2010.12/. This version is world-accessible.
Beta - Most of the time, Beta will be a mirror of whats on Live and points to the same production version in our repository (/repo/archive/2010.12). Anytime we need bug fixes on Live, we make them here, test them, and then commit & update live. (We also merge them into Trunk, if necessary.)
When the version in Trunk is deemed complete and ready for testing, I create a new archive branch in the repository for the upcoming new release (i.e. 2011.01) by svn copying the existing production branch. Then I merge the Trunk version into the new branch and commit, so we have a version that mirrors whats currently on Dev. Of course now active development for the next release can continue on Dev, while we Beta test this new Archived version on Beta. When beta testing is complete, we commit any fixes and switch Live to the new version (/repo/archive/2011.01).
Now, you've probably figured out that the database merging is a bit trickier. We use MySQL and, to my knowledge, there's no suitable versioning system for MySQL. So with each migration (VM to Dev, Dev to Beta, Beta to Live) I'll first make a backup of the current database, then make the necessary changes. Personally I use a commercial version of SQLYog which allows me to synchronize the database schema (super handy).
Summary of environment.
Asp.net web application (source stored in svn)
SQL Server database. (Database schema (tables/sprocs) stored in svn)
db version is synced with web application assembly version. (stored in table 'CurrentVersion')
CI hudson server that checks out web app from repo and runs custom msbuild file to publish/package app.
My msbuild script updates the assembly version of the web app (Major.Minor.Revision.Build) on each build. The 'Revision' is set to the currently checked out svn revision and the 'Build' to the hudson build number (incremented on each automated build).
This way i can match the app to a specific trunk revision also get other build stats from the hudson build number.
I'd like to automate the collecting of migration scripts (updated sprocs etc) to add to the zip package.
I guess by comparing the svn revision of the db that has yet to be deployed to, to the revision being deployed, i can find what db files have changed in the trunk since the last deployment to that database/environment.
This could easily be achieved by manually calling the svn diff -r REVNO:REVNO command to list changed .sql files. These files could then manually have to be added to the package.
It would be great if this could be automated.
Firstly i'd imagine I'll have to write a custom task to check the version of the db that has yet to be deployed to. After that I'm quite unsure.
Does anyone have any suggestion on how this would be achieved through an msbuild task either existing or custom?
Finally I'll have to autogen a script to add to the package that updates the database version table so as to be in sync with the application.
Integrating SQL changes into an automated build/deploy process is HARD. I know, because I've tried to to it a couple times with limited success. What you're trying to do is roughly on the right track, but I would argue that it's actually a bit too complicated. In your proposal, you suggest collecting the specific SQL scripts that need to be applied to your DB at build/package time. Instead, you should package all your delta scripts (for the entire history of your database) with your project, and calculate the deltas that actually need to be applied when you deploy -- that way, your deployable package can be deployed to environments with databases of differing versions. There are two implementation pieces you need to achieve this:
1) You need to package your deltas into your deployable package. Note that you should package deltas -- not static files that create the schema in its current state. These delta scripts should be in source control. It's okay to keep the static schema in source control as well, but you will have to keep it in sync with the deltas. You can actually use a tool like Red Gate's SQLCompare or the VS Database version to generate (most) deltas from the static schema. To get the deltas into your deployable package, and given that you're using svn -- you may want to look into svn:externals as a way to "soft link" the delta scripts into your web project. Your build script can then simply copy them into your deployable package.
2) You need a system that can read the list of delta files, compare them to an existing database, determine which deltas need to be applied to that database, and then apply the deltas (and update the bookkeeping information, like the database version). There is an open-source project (sponsored by ThoughtWorks) called dbdeploy that accomplishes this. I've had some success with that tool personally.
Good luck -- this is a tough nut to crack (correctly).
Have a look at SQL database projects. In VS 2010 they have been enhanced quite a bit and have built in deployment capabilities that can sync your DEV database to other environments.
Here are a few good links about DB projects in vs 2010:
http://msmvps.com/blogs/deborahk/archive/2010/05/02/vs-2010-database-project-building-and-deployment.aspx
http://weblogs.asp.net/gunnarpeipman/archive/2009/07/29/visual-studio-2010-database-projects.aspx
Try SQL Examiner:
http://www.sqlaccessories.com/Howto/Version_Control.aspx
You can automate script collecting with SQL Examiner command-line tool.
The solutions available today that target a .NET/SQL Server stack are:
DBUp (open source)
ReadyRoll (deeper Visual Studio integration,
auto-generation of scripts)
The latter product is one that we're actively developing here at Redgate.