When I call sqlite3_open_v2() on a zero byte file with flag SQLITE_OPEN_READWRITE, the function returns zero. Then the "PRAGMA quick_check;" statement returns a row containing string "ok". So zero-byte file is considered a valid database? Seems a little counter-intuitive?
Well I don't think it's "valid". If it's zero bytes, it's nothing, other than a reference. Yes you can open it. But that's not working if the file has >= 1 byte.
So as long as the file is 0 bytes, you can open it; once you start making changes, it will become an SQLite file.
If you open a non-SQLite file and try to make changes, it will give you an error message:
Error: file is encrypted or is not a database
The (only) way to create a new, empty database is to attempt to open a non-existing file.
A zero-sized file is considered the same; it's just an empty database.
There are some settings (such as the page size) that must be set inside an opened database connection, but that would affect the structure of the database file.
Therefore, SQLite delays actually writing the database structure until it is actually needed.
Related
I got an error when trying to query an sqlite file in R saying Error in rsqlite_fetch(res#ptr, n = n) : database disk image is malformed likely indicating that the sqlite3 file is somehow malformed. Running pragma integrity_check returns ok.
The file size is 76GB. There is one table main containing 172 columns with an index.
This was meant to create an easier to access method for a series of unstructured files and so is intended only to be read only. I tried reconstituting it several times, but it always seems malformed.
Not sure what else to look at or to do. Any help would be appreciated!
A passage in the file documentation caught my eye:
## We can do the same thing with an anonymous file.
Tfile <- file()
cat("abc\ndef\n", file = Tfile)
readLines(Tfile)
close(Tfile)
What exactly is this anonymous file? Does it exist on disk, or only in memory? I'm interested in this as I'm contemplating a program that will potentially need to create/delete thousands of temporary files, and if this happens only in memory it seems like it would have a much lesser impact on system resources.
This linux SO Q appears to suggest this file could be a real disk file, but I'm not sure how relevant to this particular example that is. Additionally, this big memory doc seems to hint at a real disk based storage (though I'm assuming the file based anonymous file is being used):
It should also be noted that a user can create an “anonymous” file-backed big.matrix by specifying "" as the filebacking argument. In this case, the backing resides in the temporary directory and a descriptor file is not created. These should be used with caution since even anonymous backings use disk space which could eventually fill the hard drive. Anonymous backings are removed either manually, by a user, or automatically, when the operating system deems it appropriate.
Alternatively, if textConnection is appropriate for use for this type of application (opened/closed hundreds/thousands of times) and is memory only that would satisfy my needs. I was planning on doing this until I read the note in that function's documentation:
As output text connections keep the character vector up to date line-by-line, they are relatively expensive to use, and it is often better to use an anonymous file() connection to collect output.
My C is very rusty, so hopefully more experienced people can correct me, but I think the answer to your question "What exactly is this anonymous file? Does it exist on disk, or only in memory?" is "It exists on disk".
Here is what happens at C level (I'm looking at the source code at http://cran.r-project.org/src/base/R-3/R-3.0.2.tar.gz):
A. Function file_open, defined in src/main/connections.c:554, has the following logic related to anonymous file (with an empty description), lines 565-568:
if(strlen(con->description) == 0) {
temp = TRUE;
name = R_tmpnam("Rf", R_TempDir);
} else name = R_ExpandFileName(con->description);
So a new temporary filename is generated if no file name was supplied to file.
B. If the name of the file is not equal to stdin, the call R_fopen(name, con->mode) happens at line 585 (there some subtleties with Win32 and UTF8 names, but we can ignore them now).
C. Finally, the file name is unlinked at line 607. The documentation for unlink says:
The unlink() function removes the link named by path from its
directory and decrements the link count of the file which was
referenced by the link. If that decrement
reduces the link count of the file to zero, and no process has the file open, then all resources associated with the file are
reclaimed. If one or more process have the
file open when the last link is removed, the link is removed, but the removal of the file is delayed until all references to it have
been closed.
So in effect the directory entry is removed but file exists as long as it's being open by R process.
D. Finally, R_fopen is defined in src/main/sysutils.c:135 and just calls fopen internally.
Is there a way that I can always append new text into the beginning of a text file in Qt? i'm using QFile::Append to do it.
file.open(QFile::Append | QFile::Text)
You can't, see the documentation at http://doc.qt.io/qt-5/qiodevice.html:
QIODevice::Append 0x0004 The device is opened in append mode, so that all data is written to the end of the file.
The problem is even worse, a file is usually stored sequentially on disk, appending (better: inserting) at the start of a file would involve moving all data towards the end of the file, thus a reorganization of filesystem blocks. I'm not sure such a filesystem exists, but if, I guess it would only allow insertion of a multiple of the filesystem block size into a file.
I am using file upload mechanism to upload file for an employee and converting it into byte[] and passing it to varBinary(Max) to store into database.
Now I what I have to do is, if any file is already uploaded for employee, simply read it from table and show file name. I have only one column to store a file and which is of type VarBinary.
Is it possible to get all file information from VarBinary field?
Any other way around, please let me know.
If you're not storing the filename, you can't retrieve it.
(Unless the file itself contains its filename in which case you'd need to parse the blob's contents.)
If the name of the file (and any other data about the file that's not part of the file's byte data) needs to be used later, then you need to save that data as well. I'd recommend adding a column for the file name, perhaps one for its type (mime type or something like that for properly sending it back to the client's browser, etc.) and maybe even one for size so you don't have to calculate that on the fly for each file (useful when displaying a grid of files and not wanting to touch the large blob field in the query that populates the grid).
Try to stay away from using the file name for system-internal identity purposes. It's fine for allowing the users to search for a file by name, select it, etc. But when actually making the request to the server to display the file it's better to use a simple integer primary key from the table to actually identify it. (On a side note, it's probably a good idea to put a unique constraint on the file name column.)
If you also need help displaying the file to the user, you'll probably want to take the approach that's tried and true for displaying images from a database. Basically it involves having a resource (generally an .aspx page, but could just as well be an HttpHandler instead) which accepts the file ID as a query string parameter and outputs the file.
This resource would have no UI (remove everything from the .aspx except the Page directive) and would manually manipulate the response headers (this is where you'd set the content type from the file's type), write the byte stream to the client, and end the response. From the client's perspective, something like ~/MyContent/MyFile.aspx?fileID=123 would be the file. (You can suggest a file name to the browser for saving purposes in the response headers, which you'd probably want to do with the file's stored name.)
There's no shortage of quick tutorials (some several years old, it's been around for a while) on how to do this with images. Just remember that there's essentially no difference from the server's perspective if it's an image or any other kind of file. All the server needs to do is send the type in the response headers and write the file's bytes to the client. How the client handles the file is up to the browser. In the vast majority of cases, the browser will know what to do (display an image, display via a plugin a PDF, save a .doc, etc.).
As part of a Classic ASP Project the user should be able to download a file - which is dynamicly extracted from a zip archive and sent via Response.BinaryWrite() - by simply calling "document.asp?id=[some id here]".
Extracting and sending is not the problem but I need to delete the extracted file after the download finished. I never did any ASP or VBA before and I guess that's why I stuck here.
I tried deleting the file right after Response.WriteBinary() using FileSystemObject.DeleteFile() but this results in a 404-Error on the client-side.
How can I wait till the download finished and then do additional actions?
Edit: This is how my code looks like:
'Unzip a specified file from an archive and put it's path in *document*
set stream = Server.CreateObject("ADODB.Stream")
stream.Open
stream.Type = 1 ' binary
stream.LoadFromFile(document)
Response.BinaryWrite(stream.Read)
'Here I want to delete the *document*
I suspect that the point you are calling the DeleteFile method the file you are trying delete is currently locked by something else, the question is what?
Try including:-
stream.Close()
after your BinaryWrite. Also make sure you've done a similar thing to the component you've used to extract the file. If the component doesn't offer any obviouse "close" methods they trying assigning Nothing to the variables referencing them.
Is it not possible to stream the file into memory, then binary write the stream to the browser, this way the file is never created on the server and there is no need to delete it.
I found a solution: The extracted files are saved in a special directory and everytime a user runs the document.asp it checks this directory for files older than one hour and deletes them.
I think it's the simplest way to manage, but furthermore I would prefer a solution where the document is deleted after downloading.