I have a remote MonetDB server running and I want to bulk upload a csv file as it is much faster.
Based on the params in MonetDB.R, there is a csvdump=TRUE option but I don't think it works when you are trying to do this against a remote server. The server has to be local.
https://rdrr.io/github/MonetDB/monetdb-r/man/dbWriteTable.html
First, am I correct that I can't do this and if not, is there a workaround? I have a dataframe with +5M rows so it takes a long time with insert statements rather than using COPY INTO.
When I try using csvdump=TRUE against the remote server, it can't find the csv file because it is local to computer that called the dbWriteTable command.
I think you are right. As a workaround either use explicit COPY INTO ON CLIENT SQL statements or first use some file transfer tool to copy the file to the remote server before calling dbWriteTable.
It reads from MonetDB's documentation on COPY INTO:
FROM files ON SERVER
With ON SERVER, which is the default, the file name must be an
absolute path on the system on which the database server (mserver5) is
running. ...
Interestingly enough pymonetdb, the Python driver for MonetDB, uses ON CLIENT for bulk loads. From the pymonetdb's doc:
File Uploads and Downloads
Classes related to file transfer requests as used by COPY INTO ON
CLIENT.
You might want to file an issue for the MonetDB R-driver project to have similar behavior as pymonetdb.
I have an application written in ASP Classic and connecting to a FoxPro database which intermittently throws this error message, found in the IIS log file:
14|80004005|[Microsoft][ODBC Visual FoxPro Driver]Unable to create temporary work files.
The line referenced is a SELECT statement against the FoxPro database. The error does not occur every time this particular SELECT statement is run.
The ever-helpful MSDN Article concerning this error suggests that the problem is either due to permissions or to disk space. Permissions are at least semi-functional because the problem doesn't always happen. Disk space is also not a problem as there is enough free space on the drive (8GB) proportional to the size of the database in question (about 500MB).
What else can I look for?
Is the FoxPro ODBC driver cleaning up after itself?
Look in your TEMP folder (defined by the %TEMP% environment variable, in my case), and you may see a large number of files matching the expression [A-Z0-9]{8}\.TMP (e.g. LVAK00AQ.TMP).
Per this ancient tome, FIX: TMP File Errors If ALTER TABLE Runs Same Time As ODBC DLL:
Visual FoxPro and the VFP ODBC driver both use the same naming
convention and algorithm for temporary (.tmp) file creation. If both
programs run concurrently, there is a conflict in the processes
attempting to access the same file or same file name. This conflict
creates different error messages.
Visual FoxPro 5.0x uses a tempfile naming scheme based on the system
clock. These names are generated for internal use and many times the
filename is never actually created on the disk. However, there are
many circumstances when FoxPro does create the temporary file on disk,
so the name generation scheme could cause two processes or two
instances of the run-time in the same process to generate the same
temporary file name. If both processes try to create a temporary file
on disk later, only the first one succeeds.
In my experience, the file naming pattern is based on a small unit of time, and with the alphanumeric scheme limited to 8 characters, it will start again from 0 within a short period of time (not sure if hours or days).
If the temp file needs to be written to disk, and the file already exists from a previous execution, then instead of overwriting the previous file, the driver will throw the error "Unable to create temporary work files".
Have you given folder and folder contents change permissions to IUSR_machinename & IWAM_machinename?
I'm an accidental DBA. We have oracle database running on windows server.
I created an oracle directory object named 'Result' and mapped it one of the local drives (C:\appResult) on the database server. The front end application creates a report and writes the file to 'Result' and a file thus gets written to C:\appResult. Now, we want to change the file path 'C:\appResult' to some other server in the network (another server in the network). What is the best way to achieve this?
Use the CREATE OR REPLACE DIRECTORY... variant of this command:
CREATE OR REPLACE DIRECTORY RESULT AS '\\ServerName\ShareName\SomeOtherDir'
Share and enjoy.
i'm new to Berkeley db, i have installed the version "db-4.8.30.NC.tar.gz" but now i would like to find the configuration
information by using the configuration file.
I've read in the documentations that this file is named DB_CONFIG and it exists in the
database home directory.
In my system, i have uzipped the tar file under /usr/db-4.8.30.NC but i still haven't found
the DB_CONFIG file.
Well, i'm trying to find where the DB_CONFIG file is located in Unix, but i can't find it.
May you please help me?
Thanks, in advance
You (or a system administrator) writes the DB_CONFIG file by hand to modify any of the DB_ENV environment variables that can be over-ridden at runtime.
The DB_CONFIG file is stored in the db_home directory; the docs/programmer_reference/env_naming.html file has full details, but you can either pass an explicit db_home parameter in the DB_ENV->open() call or rely on the DB_HOME environment variable to locate the DB_CONFIG file, if any exists. The environment variable approach might be nice if the system administrator would reasonably want to move the storage around as they wish; the specific path name approach might be nice if you don't want to bother your system administrators with details of managing your storage.
Where is the SQLite database stored i.e. directory path on windows 7 when created ?
A SQLite database is a regular file. It is created in your script current directory.
.databases
If you run this command inside SQLite
.databases
it lists the path of all currently connected databases. Sample output:
seq name file
--- --------------- ----------------------------------------------------------
0 main /home/me/a.db
There is no "standard place" for a sqlite database. The file's location is specified to the library, and may be in your home directory, in the invoking program's folder, or any other place.
If it helps, sqlite databases are, by convention, named with a .db file extension.
If you are running Rails (its the default db in Rails) check the {RAILS_ROOT}/config/database.yml file and you will see something like:
database: db/development.sqlite3
This means that it will be in the {RAILS_ROOT}/db directory.
When you call sqlite3_open() you specify the filepath the database is opened from/saved to, if it is not an absolute path it is specified relative to your current working directory.
It depends on how you initialized the database. If you used the command line shell for SQLite (e.g. sqlite3 ex1) to create the database, it'll be a path from the root of your local machine. If you used a Python script to create the database, it'll be a path from your project.
To check the former, run the command line shell:
sqlite3
sqlite> .databases
To check the path in your project, you can print the path in the connection. For example:
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATABASE = 'db'
def get_db_connection():
print(os.path.join(BASE_DIR, DATABASE, "database.db"))
conn = sqlite3.connect(os.path.join(BASE_DIR, DATABASE, "database.db"))
conn.row_factory = sqlite3.Row
return conn
In my case I think it was an access issue. I saved the SQLite files to "C:/Program Files (x86)/sqlite". I CD'd there, ran sqlite3, and created a database called test.db:
As you can see, I ran .database, which told me the .db file was created in the same directory, so I went to confirm in File Explorer, and it wasn't there:
Curiously the database was working correctly in spite of this.
It was only through trial-and-error that I discovered that I could save in some locations, but not others. It appears to me that SQLite can't save to locations that require elevation. In my case, moving from Program Files to My Documents made the issue go away.
I find it quite irritating that SQLite doesn't just tell me "access denied" instead of trying to be clever and saving to some location that I can't even find.
In Windows machines (Windows 2010), by default, the new SQLite database files will be stored in the same folder where Sqlite3.EXE application is stored in your machine. However , we can create a new folder in Windows and within sqlite> prompt, you may use the .cd to change to the new working directory.
It is a good idea to give a .db file extension to the new database files that you create (even though it is not mandatory to have any file extension)
The SQLite command, .databases will show the default database "main" or currently created or currently opened database or all "attached" database files with file path. The .attach is useful to attach more than one database file to the current connection when we want to work with tables belonging to different databases.
Regards,
Biju Joseph N.,
Houston TX, USA (January 12, 2023)
the database path will be displayed, when using .databases
SQLite is created in your python directory where you installed the python.
SQLit Database is simply a file where your local data is stored on your local machine
In Windows 10 if in the prompt command the path where you start sqlite is
C:\users\USER_NAME
You can find it in the user home folder.
The .db file is stored where you start the sqlite command.
I hope this solve the issue