"Database is locked" error in SQLite over a Mac network - sqlite

I have created a simple database using SQLite (actually PySQLite). It works fine when I'm querying or writing to the database from the local machine (ie program and database file on the windows machine drive). However when I copy the database file to my network drive (a time capsule), then Windows machines, although they can see the files and have full read/write access to the drive, give me a "SQL Error: database is locked" even when performing a simple select!
Queries work fine over the network from Macs.
There is no fancy multi-access going on - only one machine has the database open. Seems like some weird Mac networking issue. Happens in either the Python program, or in the SQLite3 command line. I am using SQLite 3.6.14.2.
Anybody seen this problem? Any way of fixing it? Don't really want to get heavy with MYSQL because this is a simple single-user program, but i'd like to use it from multiple machines.
Thanks.

I don't know if it can be done on MAC, on Debian I have to mount the samba directory with the nobrl option.
From mount.cifs(8):
nobrl
Do not send byte range lock requests to the server. This is
necessary for certain applications that break with cifs
style mandatory byte range locks (and most cifs servers do
not yet support requesting advisory byte range locks).

Read the sqlite FAQ: http://www.sqlite.org/faq.html#q5
"People who have a lot of experience
with Windows tell me that file locking
of network files is very buggy and is
not dependable. If what they say is
true, sharing an SQLite database
between two or more Windows machines
might cause unexpected problems."
So it doesn't work on Windows, it doesn't tell about MAC.

Possibly it fails to lock the file over the network, I think you use SMB protocol so the bugginess comes with the package. If you would like to use SQLite over the network see SQLite Network for alternatives.

I've had a similar problem and I solved it by installing a newer sqlite version. Since Python 2.6 the problem has disappeared too because it uses a newer sqlite dll.

Thank you Carlos. Cherrytree depends on SQLite, and for some reason it recently stopped working with my samba-mounted SQLite database file, complaining about a locked database. Adding "nobrl" to my ubuntu fstab entry solved the problem.
//192.168.3.122/Files /mnt/Files cifs username=public,password=asdf,rw,noperm,nobrl 0 0

Related

Azure Data Factory - Self-Hosted Integration Runtime - ODBC driver mystery

We are using a Self-Hosted Integration Runtime for Azure Data Factory.
On that machine there was installed an Exasol ODBC driver of version 6. We wanted to upgrade the driver, deleted an old one and installed a new driver of version 7.
Weird thing is that now in Exasol logs we can see that Data Factory is sometimes connecting via driver version 7, and sometimes via driver version 6.
I made an experiment and deleted Exasol ODBC driver from the machine completely. After that Data Factory still was able to connect to Exasol using the driver I just deleted.
Looks like drivers' DLLs are cached somewhere. What can it be?
Update 1
I captured following actions in Process Monitor when Data Fatory connected to Exasol with ODBC driver of version 6:
Where these C:\Config.Msi\3739be5*.rbfASolution-6.1\ODBC\ DLLs may come from? There is no C:\Config.Msi\ directory on the machine.
Update 2
I noticed that when I test connection via Microsoft Integration Runtime Configuration Manager on the machine or in Data Factory Linked Service, then connection is always performed with ODBC driver of version 7.
But when I test connection via Data Factory Dataset, then in some cases connection is done with ODBC driver of version 6.
You could check the registry but clean at your own risk. An alternative might be the SysIternals tools, Process Monitor or Process Explorer which might help you get to the bottom of this. Install them on the SHIR VM if you are allowed to. Process Explorer in particular is a bit like SQL Profiler (if you've ever used that) so will be able to tell you which registry keys external processes are using. It will give you a lot a lot of information so you will have to make judicious use of timestamp and filtering. The proposed steps:
Start a trace using Process Monitor
Start a pipeline using the Exasol driver
Wait til it completes (or at least you know it has started)
Stop the Process Monitor trace Spend time going through the millions
of records it has captured, trying to filter down, or search for your
process
An alternative would be to build a clean SHIR and install only the new driver. Then swap it in for the old one. You may have to get the new SHIR added to the firewall if this is an issue for you.
Honestly I would propose both of these approached in parallel for a production problem. Procmon / Process Explorer can be quite labour and time expensive but should help you get to the bottom of the issue. Building a cleaner SHIR is probably a safer option in the long-term, but requires new infrastructure.
It may sound silly, but rebooting the server where SHIR is working solved the problem.
We noticed, that this server was running for more than 30 days, and decided to reboot it. Maybe restarting Integration Runtime service itself would also help, but we didn't do it.
Thanks to everyone for you help.

Cosmos DB Emulator won't start on Windows 10

Iā€™m recieving this error when I try to start Cosmos Emulator 2.7.2.0:
Tried it on a fresh win10 machine, and it's working here. Any ideas on what might course this?
I tried all the suggestions in this article.
Looking at the etl file didn't give my anything either.
It's the first time I try to install it on my machine.
-- UPDATE --
I opened the dmp file, and found this:
Loading Dump File [C:\Users\xxx\AppData\Local\CrashDumps\Microsoft.Azure.Cosmos.StartupEntryPoint.exe(1).12596.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available
Symbol search path is: srv*
Executable search path is:
Windows 10 Version 18363 MP (12 procs) Free x64
Product: WinNt, suite: SingleUserTS
18362.1.amd64fre.19h1_release.190318-1202
Machine Name:
Debug session time: Mon Jan 20 09:35:26.000 2020 (UTC + 1:00)
System Uptime: not available
Process Uptime: 0 days 0:00:02.000
................................................................
..................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(3134.41b0): Unknown exception - code c000000d (first/second chance not available)
For analysis of this file, run !analyze -v
ntdll!NtWaitForMultipleObjects+0x14:
00007ff9`562fcc14 c3 ret
0:000> .ecxr
rax=0000000000000000 rbx=0000000000000000 rcx=0000003629dfd270
rdx=00000000e6bc7d4e rsi=0000000000000000 rdi=0000000000000000
rip=000000006748b0ec rsp=0000003629dfd190 rbp=0000000000000000
r8=000001ba53134730 r9=0000000000000000 r10=0000000000000026
r11=000001ba6d7662e0 r12=0000000000000004 r13=000001ba6d7974d0
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl zr na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
msvcr80!_invalid_parameter+0x6c:
00000000`6748b0ec 488d4c2440 lea rcx,[rsp+40h]
Seems like something is wrong with my version of msvcr80
This may be caused by corrupted performance counters on your machine.
To fix your performance counters try the following.
Open cmd as administrator
Run "lodctr /R" (must use capital R)
If this doesn't work, see this link here that shows other options to reset your counters. Older article but works the same on Windows 10.
https://answers.microsoft.com/en-us/insider/forum/insider_wintp-insider_perf/possible-performance-counter-problem-on-win10/3a5c22cb-1425-4d26-99e7-4ec46940b9a1
btw, one more option here, is to run the emulator in a Docker container. The docs on that are in that article you referenced in your question.
Hope this is helpful.
For me this confusing error message simply meant that I needed to use another port! Or fix problems with the default 8081 one (read update 2 way below).
Here's the text of the error so that it's searchable:
Error: multiple attempts to restart one of the Azure Cosmos Emulator
processes were detected. The emulator will shutdown. If the problem
persist please try uninstalling the Azure Cosmos Emulator, remove the
CosmosDBEmulator directory from your %%LOCALAPPDATA%% and reinstall.
You can also reach out to the Azure Cosmos team using the 'Feedback'
link in the Data Explorer browser window.
Details:
In my case this happened by itself after Windows 10 updated 2004. I swear something breaks in Cosmos DB emulator every feature update!
For me there were no crash dumps and rebuilding performance counters didn't help. In the Event Viewer for System I've noticed this error A LOT:
Unable to bind to the underlying transport for 127.0.0.1:8081. The IP Listen-Only list may contain a reference to an interface which may not exist on this machine. The data field contains the error number.
No solutions for this error helped, so I first used netstat -ao to see if any program is already listening at 8081. Nope.
Then I used the "Port listener" utility from the www, and it turned out it can't listen to 8081 either, not even under admin rights!
Since I couldn't fix this 8081 problem at all & turning off Windows Firewall didn't help, I simply started the Cosmos DB emulator at a different port and it worked!
C:\Program Files\Azure Cosmos DB Emulator>Microsoft.Azure.Cosmos.Emulator.exe /Port=18001
If you've configured cosmos db to be started automatically on system startup, make sure to update that script as well.
In my case this cmdline got me into the Data explorer, but it wasn't actually working. The "Explorer" part of it was "fetching offers" infinitely. I had to again uninstall Cosmos DB emulator, clean this folder: %LOCALAPPDATA%\CosmosDBEmulator, reboot, install again without starting at 8081, and only start at the new port.
Cosmos DB devs, you're awesome people, but this is too much hassle. And if you can't use the port just say so, you don't have to be cryptic!
UPDATE:
This worked, but I forgot to update the start up script to use the new port, and noticed that after one more reboot the cosmos db started on 8081 without issues. What??! But it wasn't just any old reboot - I already did several of them before and they didn't help. This reboot may have been special in that it came after Windows found one more update (kb4576478), probably realizing it's now needed since I'm at version 2004, installed it, and THAT reboot (or the kb4576478 update specifically) fixed the 8081 problem, whatever it was. Yikes!
UPDATE 2:
Had another port-related problem with another tool. Ugh. Found this solution (see "General workaround") https://superuser.com/a/1568476/251491 This page's accepted answer also mentions the magic of multiple reboots.
Finally got it working!
By using WinDbg and opening the dmp files %LOCALAPPDATA%\CrashDumps, I found out that a dll called perf-MSSQL$SQLEXPRESS-sqlctr10.1.2531.0.dll was the root cause.
By deleting this in the %WINDIR%\System32 the error was "fixed" and I can now run the emulator.

Can not tnsping but can sqlplus connect

I have a question hopefully someone could explain it to me. I have an Oracle 11g installed properly on the server. From a workstation, I have installed the oracle client which tnsname.ora pointed to the Oracle database. I can ping to the server where oracle db was installed. I can sqlplus connect to the database and I can even connect to the database programmatically using C#. However, I can not tnsping to it. I got error Message 3511 not found when trying to do so. So my question is: in what scenario tnsping is used and how do I make tnsping works? In other word, if my tnsping does not work but I still can connect to the database programmatically using C#, what are the concerns that I should be worrying about? I am new to Oracle.. Thanks!
I have tried to do the following actions without success although they are the most popular answers to similar scenario similar to mine.
1) Set Oracle_Sid to my oracle sid
2) Set Oracle_Home to my oracle home directory
What I did to solve the issue:
1) Uninstall oracle client
2) Reinstall the oracle client with the newest version 11.2.xx.
I believe, re-installing the oracle client of earlier version would just work as well.
However, since I have to re-install the client, I might want to get the newest version.
The cause is probably improper installation at the first time as we have an disk image of pre-installed application to automatically prep a new computer.
Not able tnsping does not seem to impede with other Oracle operations (at least not that I know of).
If someone knows what is the limitation of not able to tnsping but can sqlplus connect, ping... , I would appreciate if you could share.
Either your $Oracle_Home value is not set or your default sid is not set in $ORACLE_SID
Try this :
set ORACLE_SID=mysid
export ORACLE_SID
tnsping mysid
and see what it comes back with.

Cygwin SVN: E200030: SQLite disk I/O error

When I use Subversion in Cygwin to update some repository, some directories update with success, while some other one gets a failure with the error message:
svn: E200030: sqlite: disk I/O error
When doing svn update again for the same repository, a different directory can get the same error. Sometimes, there is a SVN instruction after the above error message.
This happened due to a change someone wanted in Cygwin's SQLite package. I was the maintainer of that package when this question was asked, and I made the change that caused this symptom.
The change was released as Cygwin SQLite version 3.7.12.1-1, and it fixed that one person's problem, but it had this bad side effect of preventing Cygwin's Subversion package from cooperating with native Windows Subversion implementations.
What Happened?
The core issue here is that Subversion 1.7 changed the working copy on-disk format. Part of that change involves a new SQLite database file, .svn/wc.db. Now, in order to implement SQLite's concurrency guarantees, SQLite locks the database file while it is accessing it.
That's all fine and sensible, but you run into a problem when you try to mix Windows native and POSIX file locking semantics. On Windows, file locking almost always means mandatory locking, but on Linux systems ā€” which Cygwin is trying to emulate ā€” locking usually means advisory locking instead.
That helps understand where the "disk I/O error" comes from.
The Cygwin SQLite 3.7.12.1-1 change was to build the library in "Unix mode" instead of "Cygwin mode." In Cygwin mode, the library uses Windows native file locking, which goes against the philosophy of Cygwin: where possible, Cygwin packages call POSIX functions instead of direct to the Windows API, so that cygwin1.dll can provide the proper POSIX semantics.
POSIX advisory file locking is exactly what you want with SQLite when all the programs accessing the SQLite DBs in question are built with Cygwin, which is the default assumption within Cygwin. But, when you run a Windows native Subversion program like TortoiseSVN alongside a pure POSIX Cygwin svn, you get a conflict. When the TortoiseSVN Windows Explorer shell extension has the .svn/wc.db file locked with a mandatory lock and Cygwin svn comes along and tries an advisory lock on it, it fails immediately. Cygwin svn assumes a lock attempt will either succeed immediately or block until it can succeed, so it incorrectly interprets the lock failure as a disk I/O error.
How Did We Solve This Dilemma?
Within Cygwin, we always try to play nice with Windows native programs where possible. The trick was to find a way to do that, while still playing nice with Cygwin programs, too.
Not everyone agreed that we should attempt this. "Cygwin SQLite is part of Cygwin, so it only needs to work well with other Cygwin programs," one group would say. The counterpartisans would reply, "Cygwin runs on Windows, so it has to perform well with other Windows programs."
Fortunately, we came up with a way to make both groups happy.
As part of the Cygwin SQLite 3.7.17-x packaging effort, I tested a new feature that Corinna Vinschen added to cygwin1.dll version 1.7.19. It allowed a program to request mandatory file locking through the BSD file locking APIs. My part of the change was to make Cygwin SQLite turn this feature on and off at the user's direction, allowing the same package to meet the needs of both the Cygwin-centric and Windows-native camps.
This Cygwin DLL feature was further improved in 1.7.20, and I released Cygwin SQLite 3.7.13-3 using the finalized locking semantics. This version allowed a choice of three locking strategies: POSIX advisory locking, BSD advisory locking, and BSD/Cygwin mandatory locking. So far, the latter strategy has proven to be completely compatible with native Windows locking.
Later, when Jan Nijtmans took over maintenance of Cygwin SQLite, he further enhanced this mechanism by fully integrating it with the SQLite VFS layer. This allowed a fourth option: the native Windows locking that Cygwin SQLite used to use before we started on this journey. This is mostly a hedge against the possibility that the BSD/Windows locking strategy doesn't cooperate cleanly with a native Windows SQLite program. So far as I know, no one has ever needed to use this option, but it's nice to know it's there.
Alternate Remedy
If the conflict you're having is between Cygwin's command line svn and the TortoiseSVN Windows Explorer shell extension, there's another option to fix it. TortoiseSVN ships with native Windows Subversion command-line programs as well. If you put these in your PATH ahead of Cygwin's bin directory, you shouldn't run into this problem at all.
Having encountered the same problem, it appears (in my case at least) to be an interaction with TortoiseSVN. Disabling TortoiseSVN's status icon cache (Settings > Icon Overlays > Status cache "None" > Apply) has everything working just fine for me.
(That obviously doesn't resolve the underlying problem, which appears to be due to the SQL package that Cygwin's Subversion package relies on changing its mode of access. As I write, there's active [if slow] discussion on the Cygwin mailing list about how to resolve this.)
ldd /usr/bin/svn shows that SVN depends on /usr/bin/cygsqlite3-0.dll.
After I change libsqlite3 from 3.7.12 back to 3.7.3, the problem seems to go away. So this may be a SQLite library problem.
Using TortoiseSVN, ticking off Refresh shell overlays at clean up solved the problem for me.
For others reference, I just had this same error (svn: E200030: sqlite: disk I/O error) and found that one of my log files was taking up all my space (and could not write to the HDD because there was no free space).
Run (to make sure you have enough disk space)
df -h
(If you don't delete some large files (I just removed some backup and log files)
Then I just needed to run:
svn cleanup
This resolved the error for me.

Running Visual Studio in Parallels for mac - problem with debugging sites sitting in os x drive

I've installed parallels desktop on my MacBook to be able to run Visual Studio 2008 in a XP installation. Everything works great except when I decided to put my websites in my sites folder in the os x file system (Which by default automatically happens because the My Documents folder is mapped to the Mac's Documents folder, and I'd rather put my code there so that both OS's can easily access it.).
When trying to build or debug I get this error:
Failed to start monitoring changes to 'Z:\xxx...'
How do I get it so that I can get it to work under Parallels, from the shared drive?
Parallels uses network drives to simulate folders on OS X, and Windows can't monitor changes to network drives, so if you do this directly, it'll be broken.
If you want to keep them in sync though, use Live Mesh (http://www.mesh.com) and install it on both the host and guest. A little roundabout, but it'll make it so both copies are maintained (and Live Mesh is handy for other things too)
I recently flipped over to putting my source code onto my Mac volume, so I could use Time Machine to back it up and immediately got this same problem with my ASP.NET app. Other, procedural applications, built just fine, by the way.
I tried all sorts of things, including using Samba on the Mac side to share the directory, which led into the "too many BIOS commands" error described elsewhere. Unfortunately for me, the Registry hacks to fix that problem never worked for some reason.
I finally found another solution that avoids Samba and just uses the regular Parallels Shared Folders. It too is a Registry hack, but this one simply turns off file change monitoring for ASP.NET. It is a bit heavy-handed, but gets my builds to work again.
The reference for this change is here:
http://support.microsoft.com/kb/911272
The downside to this approach, I am finding, is that you need to be more deliberate about recompiling, or restarting the web server, as changes during development don't just magically appear anymore. I am still deciding whether that is a useful tradeoff.
UPDATE: After several days of this, development was just too difficult and, sadly, what I reverted to was keeping my source inside the Parallels virtual disk. To enable Time Machine backups and Spotlight searches, I used a lightweight MS utility called SyncToy to push stuff out of Parallels and out to my Mac drive several times a day. Despite the high hack factor, it is working well.
I know this isnt strictly a solution but VMware fusion is superior when it comes to shared drive space on a virtual machine. Its what i currently use and hasn't let me down thus far...
People always give me odd looks when they see visual studio on my mac :P
Try moving the project on to the VMs C drive. Its not an ideal situation, but you can access the VMs C drive from OS X.
I have a similar problem with a php site that uses an MS Access database (its a clients system). I have alias's that point to the php site on the VM so that I can still do all of my coding in OS X. To do this I created a network share on the VM and then connected to it from OS X. Once connected make the alias's. If the network drive is not open and you open a file in OS X it will try to reconnect. It means the VM will need to be running to get to the files, but this isn't normally a problem since the VM is hosting the site anyways.
.NET has funny issues trying to debug the objects on a network drive.
make sure that you have full trust on your local network between your Mac and XP install.
Check out: http://msdn.microsoft.com/en-us/library/aa302361.aspx
If at the end of that research, I"m afraid you will have to look into the option of keeping it on the VMDisk and moving it when you need it.
I see a similar problem on my machine connected to the windows domain. My documents is mapped to a network share and I can't debug|run|etc. I had to eventually move to my local disk for debugging.
I definately recommend Live Mesh as a way to keep directories in sync. Just keep the VM's directory in sync with the Mac's directory.
Or use SVN to hold copies in both machines and do commit/update as appropriate. That way you get versioning, history and if your project grows bigger, you can share with other devs.
I know dropbox also has history and sharing, but not check in/check out/conflicts and all the other advantages of a real source control.
Oh, if you have money you can also go for TFS. I would but it is just too expensive :)

Resources