Excuse for English
I using SQLite and test it. Insert multi-million row for testing speed. and deletes rows after any insert.
But i know my database size is 33.0 MB..... now database is empty. but size on disk is 33 MB.
WHY?
can you help me?
The VACUUM command rebuilds the entire database. There are several
reasons an application might do this:
Unless SQLite is running in "auto_vacuum=FULL" mode, when a large amount of data is deleted from the database file it leaves behind
empty space, or "free" database pages. This means the database file
might be larger than strictly necessary. Running VACUUM to rebuild the
database reclaims this space and reduces the size of the database
file.
https://www.sqlite.org/lang_vacuum.html
Related
I am testing MariaDB for a possible replacement of a MySQL data warehouse. This data warehouse is rebuilt nightly from a legacy database.
The basic process is to generate XML documents/files from the legacy database and then use DROP TABLE, create table from DDL, LOAD XML LOCAL INFILE 'xml file'. A few of the XML files are 60-100 megabytes (about 300K rows). On MySQL these tables take a couple minutes. On MariaDB these tables take significantly longer (e.g. 83 megabyte XML file requires 16 minutes on MariaDB, MySQL less than 1 minute), and the times seem to grow exponentially with file size.
I have read and followed the KB topic How to Quickly Insert Data Into MariaDB and have tried the suggestions there with no real change. Since the MariaDB tables are dropped and recreated immediately before the LOAD XML LOCAL INFILE, several performance improvements should be triggered.
I have not tried LOCK TABLE yet.
What can I try to improve performance? Don't want to return to MySQL but this issue is a deal killer.
Environment is RHEL 8, MariaDB 10.5.16
Used DISABLE KEYS/LOAD.../ENABLE KEYS with no apparent benefit.
Increased max_allowed_packet_size with no effect.
I have an MS access DB with 35 linked tables, a few queries and another 35 reports.
The database has no physical tables in it as all data for its tables are coming from the linked back-end MS Access database. The problem now is, the front-end database size is above 1 GB. How and why? And how can I fix it?
First, please try HansUp's suggestion.
But, if that doesn't shrink it as much as you expect, try this:
Make a file called decompile.bat. In it, put the command:
"C:\Program Files (x86)\Microsoft Office\Office14\MSACCESS.EXE" "C:\Your\Path\To\YourFrontEnd.accdb" /nostartup/decompile
Edit the paths to suit. Put this batch file in the same directory as your front-end.
To use:
Run decompile.bat by double-clicking on it in Windows Explorer
In Access:
Hit Alt-F11 to go to the Visual Basic Editor
Click Debug, then Compile
Save, then exit the Visual Basic Editor
In the main Access window, click Database Tools, Compact and Repair Database
When finished, exit Access
You should notice that your front-end is dramatically smaller.
That's a very late addition but I wonder why didn't anyone point at the cause of the proplem! Microsoft says the cause is:
If you do not release a recordset's memory each time that you loop
through the recordset code, DAO may recompile, using more memory and
increasing the size of the database.
And the solution:
Use the Close method of the Recordset object to explicitly close the
recordset's memory when you no longer need the recordset. If the
database has increased in size because you did not use the Close
method of the Recordset object, you can reduce the size of the
database by running the Compact and Repair utility (on the Tools
menu).
https://learn.microsoft.com/en-us/office/troubleshoot/access/prevent-database-bloat-dao
I've used the "Compact and repair tool" from the "Database tools" ribbon once and it did reduced the size of -an almost empty- database from 2.0 GB (Max. Size) to just 3.41 MB!
I just ran across this. We use access databases to store project data. For the most part they stay less than a megabyte. Then I was getting issues from a client that storage cost was going up on a server because some databases were growing to the tens and hundreds of megabytes. So I search my stuff and found a database at a gig!
I'm using Office 7, and I'm pretty sure it is still in 365.
Solution:
From the Office button in access, (upper left). Go to Manage -> Backup Database. For me, it copied that 1 gig database to a 720k database. And, of course, it opened fine and all the data was there. I found that Compact and Repair did not come anywhere close to the results of backup
I don't know why Microsoft lets this occur, I could not find a reasonable explanation for it.
I'm going to end-up with a rather large database CubeSQLite in the cloud and cloned on the local machine. In my current databases I already have 185 tables and growing. I store them in 6 SQLite databases and begin by attaching them together Using the ATTACH DATABASE command. There are views that point to information in other databases and, as a result, Navicat won't open the SQLite tables individually. It finds them to be corrupted, although they are not and are working fine.
My actual question is this:
Considering the potential size of the files, is it better/faster/slower to do it this way or to put them all into one really large SQLite DB?
I have a executable jar file that inserts data into a SQLite database. This insertion, however, is taking so longer than I expected. I thought I might be able to create another copy of this jar file to HELP the first one.
The reason why this process is so slow is not because my CPU is working 100%, but because the process itself is time-consuming.
By the way, during this process no row will be deleted. it's just INSERT and UPDATE.
I have this error in my application log:
sqlite3.OperationalError: database or disk is full
As plenty of disk space is available and my SQLite database does not appear to be corrupted (integrity_check did not report any error), why is this happening and how can I debug it?
I am using the Lustre filesystem (with flock set), and until now, it worked perfectly.
Versions are:
Python 2.6.6
SQLite 3.3.6
It's probably too late for the original poster, but I just had this problem and couldn't find an answer so I'll document my findings in the hope that it will help others:
As it turns out, an SQLite database actually can get full even if there's plenty of disk space, because it has a limit for the number of pages in a database file:
http://www.sqlite.org/pragma.html#pragma_max_page_count
In my case the value was 1073741823, which meant that in combination with a page size of 1024 Bytes the database maxed out at 1 TB and returned the "database or disk is full" error.
The good news is that you can raise the limit; for example double it by issuing PRAGMA max_page_count = 2147483646;.
The limit doesn't seem to be saved in the database file, though, so you have to run it in your application every time you open the database.
By default, SQLite uses /tmp temporary directory (not the memory). If /tmp is too small you will get disk full. In that case change the temporary directory like that: export TMPDIR=<big file system>.
I had same problem too.
Your host or PC's storage is full so delete some files in your system then problem is gone.