I am using a c program to write/delete 1-2MB of data periodically (10min) to sqlite3 database. The program also act as a read only database for my node.js web server to output Restful APIs. (I can not use node.js modules because node.js web server is on different machine)
In documentation its mentioned that in client/server architechture RDBMS might be good but that point is not put strongly
I am using a c program to act as a server to answer web servers request as well as other processes on different machine. The system require small data (~2-5Mb) frequently (every 5min).
If it is not good to use sqlite as client server database How can I convince my manager?
If its okay then why do they not have a standard server plugin?
When the SQLite documentation speaks about a client/server architecture, it says:
If you have many client programs accessing a common database over a network
This applies to whatever program(s) that access the database directly. (In the case of SQLite, this would imply that you have a networked file server, and that multiple clients access the database file directly over the network.)
If you have a single process that accesses the database file, then you do not have a client/server architecture as far as the database is concerned.
If there are other clients that access your server program, this has no effect on your database.
Related
I am new to oracle coherence
Basically we have some data and we wanted some java/bpel webservice to get those data from coherence cache instead of database. [we are planning to load all those data to cache server]
So we have below questions before we start this solution.
Webservice we are planning to start is going to be just java would be fine.
And all operations are reading only.
Question
1. IS it Coherence needs to be stand alone server ? (down load from oracle and install it separately and run the default cacheserver) ?
2.If so we are planning to do the pre loading of data from database to cache server by using code ? i hope thas possible ? Any pointers would be helpful ?
3.How does the webservice connect with Coherence server if webservice running in different nmachine vs coherence server running ?
(OR)
Is it mandatory that webservice and coherence should run in the same machine ?
If webservice can run in different machine how does the webservice code connects to coherence server (any code sample , url would be helpful) ?
Also what is that coherence comes with weblogic ? Is it not fit for our applications design i assume ?!!!! then what type of solution we go for weblogic with coherence ?
FYI : Our goal is simple we want to store the data in cache server and have our new webservice to retrieve the data from Cache servere instead of database(because v are planning to avoid database trip )
Well, you questions are very open and probably have more than 1 correct answer. I'll try to answer all of them.
First, please take into consideration, that Coherence is not a free tool and you have to pay for a license.
Now, to the answers:
Basically, coherence has 2 parts: proxy and server. The first is the responsible to routing your requests and the second for hosting the data. You can run both together in the same service but this has pros&cons. One Con is that your services will be very loaded and the memory will be shared between two kind of processes. A pro is that is very simple to run.
You can preload all the data from the DB. For that you have to open the Coherence and write your own code. For that, you need to define you own cachestore (look for that keyword in coherence docs) and override the loadAll method.
As far I remember Coherence comes together with Weblogic. That says the license for the one is the license for the second and they come in the same product. I'm not familiar with weblogic, but I suppose is a service of the package. In any case, for connecting to coherence you can refer to Configuring and Managing Coherence Clusters
The coherence services can run in different machines, in different network and even in different places of the world if you want. Each, proxy or server, consumer and DB, could be in a different network. Everything can be configured. You have to say you weblogic server where the coherence proxy will be, you'll set in the coherence proxy/server the addresses of them and you'll configure your coherence server for finding out his database by configuration. Is a bit complicated to explain everything here.
I think I answered before.
Just take into consideration coherence is a very powerful tool but very complicated to operate and to troubleshoot. Consider the pros/cons of accessing directly your DB and think about if you really need it.
If you have specific questions, please don't hesitate. Is a bit complicated to explain everything here since you're trying to set up one of the complicated system I ever seen. But is excellent and I really recommend it.
EDIT:
Basically Coherence is composed by 2 main parts: proxy and server. The names are a bit confusing since both are servers, but the proxy serves to the clients trying to perform cache operations (CRUD) while the "servers" serve to the proxies. Proxy is the responsible for receiving all the requests, processing them and routing them, according to their keys, to the respective server who holds the data or who would be the responsible for holding it if the operation requires a loading act. So the answer to your questions is: YES, you need at least one proxy active in your cluster, otherwise you'll be unable to operate correctly. It can run on one of your machines on into a third one. Is recommended to hold more than 1 proxy for HA purposes and proxies can act as servers as well (by setting the localstorage flag to true). I know, is a bit complicated and I recommend to follow oracle docs.
Essentially, there are 2 types of Coherence installation.
1) Stand-alone installation (without a WebLogic Server in the mix)
2) Managed installation (with Weblogic Server in the mix)
Here are a few characteristics for each of the above
Stand-alone installation (without a WebLogic Server in the mix)
Download the Coherence installation package and install (without any dependency on existing WebLogic or FMW installations)
Setup and Configure the Coherence Servers from the command-line
Administer and Maintain the Coherence Servers from the command-line
Managed installation (with Weblogic Server in the mix)
Utilize the existing installation of Coherence that was installed when WebLogic or FMW was installed
Setup and Configure the Managed Conherence Servers to work with WebLogic server
Administer and Maintain the Managed Coherence Servers via the WebLogic console
Note the key difference in terminology, Coherence Servers (no WL dependency) vs. Managed Coherence Servers (with WL dependency)
I'm developing an application that works distributed, and I have a SQLite database that must be shared between distributed servers.
If I'm in serverA, and change sqlite row, this change must be in the other servers instantly, but if a server were offline and then it came online, it must update all info equal other servers.
I'm trying to develop a HA service with small SQLite databases.
I'm thinking on something like MongoDB or ReThinkDB, due to replication works fine and I have got data independently server online I had.
There are a library or other SQL methodology to share data between servers?
I used the Raft consensus protocol to replicate my SQLite database. You can find the system here:
https://github.com/rqlite/rqlite
Here are some options:
LiteReplica:
It supports master-slave replication for SQLite3 databases using a single master (writable node) and one or many replicas (read-only nodes).
If a device went offline and then it came online, the secondary/slave dbs are updated with the primary/master one incrementally.
LiteSync:
It implements multi-master replication so we can write to the db in any node, even when the device is off-line.
On both we open the database using a modified URI, like this:
“file:/path/to/app.db?replica=master&bind=tcp://0.0.0.0:4444”
AergoLite:
Blockchain based, it has the highest level of security. Stores immutable relational data, secured by a distributed consensus with low resource usage.
Disclosure: I am the author of these solutions
You can synchronize SQLite databases by embedding SymmetricDS in your application. It supports occasionally connected clients, so it will capture changes and sync them when a server comes online. It supports several different database platforms and can be used as a library or as a standalone service.
You can also use CopyCat, which support SQLite as well as a few other database types.
Marmot looks good:
https://github.com/maxpert/marmot
From their docs:
What & Why?
Marmot is a distributed SQLite replicator with leaderless, and eventual consistency. It allows you to build a robust replication between your nodes by building on top of fault-tolerant NATS Jetstream. This means if you are running a read heavy website based on SQLite, you should be easily able to scale it out by adding more SQLite replicated nodes. SQLite is probably the most ubiquitous DB that exists almost everywhere, Marmot aims to make it even more ubiquitous for server side applications by building a replication layer on top.
I'm designing a new web project and, after studying some options aiming scalability, I came up with two database solutions:
Local SQLite files carefully designed for a scalable fashion (one new database file for each X users, as writes will depend on user content, with no cross-user data dependence);
Remote MongoDB server (like Mongolab), as my host server doesn't serve MongoDB.
I don't trust MySQL server at current shared host, as it cames down very frequently (and I had problems with MySQL on another host, too). For the same reason I'm not goint to use postgres.
Pros of SQLite:
It's local, so it must be faster (I'll take care of using index and transactions properly);
I don't need to worry about tcp sniffing, as Mongo wire protocol is not crypted;
I don't need to worry about server outage, as SQLite is serverless.
Pros of MongoDB:
It's more easily scalable;
I don't need to worry on splitting databases, as scalability seems natural;
I don't need to worry about schema changes, as Mongo is schemaless and SQLite doesn't fully support alter table (specially considering changing many production files, etc.).
I want help to make a decision (and maybe consider a third option). Which one is better when write and read operations is growing?
I'm going to use Ruby.
One major risk of the SQLite approach is that as your requirements to scale increase, you will not be able to (easily) deploy on multiple application servers. You may be able to partition your users into separate servers, but if that server were to go down, you would have some subset of users who could not access their data.
Using MongoDB (or any other centralized service) alleviates this problem, as your web servers are stateless -- they can be added or removed at any time to accommodate web load without having to worry about what data lives where.
I have a client who is interested in hiring my company to do a small, custom, multi-user contact database/crm. They don't care about the technology I use, as long as the product is hosted inside their organization (no "cloud" hosting). Now the problem is that their IT department refuses to host any application developed by an outside company on their servers, additionally they will not allow any server not serviced by them inside of their network.
The only means of sharing data that IT would allow is a windows network share...
I was thinking to do the application as a fat client in Adobe Air, and let all users access a shared sqlite database, but then I read a lot of negative things about this approach.
So I'm asking you - Are there people out there who have actually tried this ?
What are your experiences ?
You can use an MS-Access 2007+ (accdb) file.
Of course there are many database engines with much more features and much better SQL syntax, but if you are looking for a file-based database system that can be accessed simultaneously by multiple processes on a shared Windows drive, then an accdb file is as good as you're going to get I think.
Also note that another popular embedded database, SQL Server Compact Edition, cannot be used on shared drives (at least not by multiple processes from different machines).
References:
Share Access Database on a Network Drive:
http://office.microsoft.com/en-us/access-help/ways-to-share-an-access-database-HA010279159.aspx#BM3
SQL Server CE Cannot be used on a shared drive:
SQLCE 4 - EF4.1 Internal error: Cannot open the shared memory region
The ways sqlite locks databases means you have to be careful if there's a chance you'll have multiple sources trying to access the database. You either have to develop a waiting method, or a timeout, or something
We are looking at creating a custom ASP.NET application for a client, however they are a nonprofit and thus budget is limited.
We typically develop ASP.NET web and desktop apps to connect to a central SQl Server 200X database, ie with a full version of SQL Server, running on networked Windows Server. In this case we won't have a full version available.
Are there any issues with using SQL Server Express in this sort of arrangement? IIS and SQL Server Express would be running on the same physical server, serving up pages over the local Intranet to users.
Any real differences to be aware of in regards to development of the app itself or deployment? This will be a fairly standard app, with SQL mainly being used for a datastore with tables and SPs, nothing really SQL Server specific beyond that.
SQL Server Express edition should be fine for this scenario. It has all the core features of the full product, but as you said you are only really using it for data storage and some SPs, then you will not need any of the additional functionality available in the other versions (ie. reporting and analysis services). There are some limitations to the express version (the biggest being that the maximum database size is 4GB), but they should not really affect you unless your are building a very busy ASP.Net application.
Some public-facing websites use SQL Server Express as the database server (DotNetKicks being the only one I can remember at the moment) without issue.
The exact list of unsuported features in Express is at SQL Server Express Features:
Database mirroring
SQL Mail
Online restore
Fail-over clustering
Database snapshot
Distributed partitioned views
Parallel index operations
VIA protocol support
Mirrored media sets
Log shipping
Partitioning
Parallel DBCC
Address Windowing Extensions (AWE)
Parallel Create Index
Hot-add memory
Enhanced Read Ahead and Scan
Native HTTP SOAP access
Indexed views (materialized views)
SQL Mail and Database Mail
Partitioned views
Online Index Operations
SQL Server Agent and SQL Server Agent Service
SSIS, SSAS, OLAP/Data Mining
The SQL Server Express with Advanced Services Features supports a "subset of Reporting Services features".
In addtion there are the operational restrictions:
Express will use onyl one CPU core
Express will not grow the buffer pool over 1 GB no matter how much RAM you have
Express will not allow any database to grow over 4GB and will not put online (restore, attach) databases that are already over 4 GB.
The key problems you may run into are the operational restrictions (one core, 1 GB ram, 4GB each database) and the lack of SQL Agent, preventing any sort of job scheduling.
You should not really run into anything, its actually a full featured product that MS SQL Express
Here's a really basic comparison from Microsoft.