I've written a .Net Web API which takes it's inputs, parses them and then stores an XML file on a network share linked to our server. I have also built a Windows service which scans the network share for new files to process our business logic.
This works nearly 100% of the time, but very occasionally (1 in 20,000 times) IIS6 holds a lock on the file it creates and won't clear until IIS is restarted. The locked files are always 0 bytes.
Currently I have a file which has been locked for nearly 20 hours! Here is the code that creates the file:
Try
'-- Make sure the file doesn't already exist
TempFileName = strFullFileName
i = 1
While IO.File.Exists(TempFileName)
TempFileName = strFullFileName.Replace(".xml", "_" & i & ".xml")
i += 1
End While
strFullFileName = TempFileName
'-- Deserialise the message into a file
drSerializer = New XmlSerializer(DetailsOfMsg.GetType)
FS = New FileStream(strFullFileName, FileMode.Create, FileAccess.ReadWrite, FileShare.None)
XW = XmlWriter.Create(FS)
drSerializer.Serialize(XW, DetailsOfMsg)
Finally
Try : XW.Flush() : Catch : End Try
Try : FS.Close() : Catch : End Try
Try : XW.Close() : Catch : End Try
FS = Nothing
XW = Nothing
End Try
Why is IIS still holding a lock?
Did you try to wrap the code within "Using" blocks? This ensures that types of FileStream and XmlWriter get disposed once the block's scope ends.
I think you need to separate this process. First create file on say folder X. Once created then move this file from folder X to shared location as there is watcher associated with this network share. Also, once file found then pick it and move to working folder and then start your business process on that file. 0 byte may be indicator of write and watch deadlock.
I can see that, you have created FileStream instance with FileShare = none, whereas your requirement says that, you need simultaneously read and write on shared location.
Correct code would be
FS = New FileStream(strFullFileName, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite)
FileShare - A constant that determines how the file will be shared by processes.
For more info - refer this - https://msdn.microsoft.com/en-us/library/5h0z48dh(v=vs.110).aspx
EDIT
From comment, I found that you need lock to be applied for Read operation, and the error you are getting ( not so frequently) might be because of write lock. To avoid this, you can use following.
FS = New FileStream(strFullFileName, FileMode.Create, FileAccess.ReadWrite, FileShare.Write)
FileShare.Write - Allows subsequent opening of the file for writing.
If this flag is not specified, any request to open the file for
writing (by this process or another process) will fail until the file
is closed. However, even if this flag is specified, additional
permissions might still be needed to access the file.
Related
I am working with Sqlite db in uwp, I am able to create db successfully without any issue in Local folder of my app.
But when I try to create in other places, I get error as
"SQLite.Net.SQLiteException: 'Could not open database file: E:\Users\PC3\Pictures\Abcd\MyDbFolder\mydb.sqlite (CannotOpen)"
I got the path string using StorageFolder.Path which I got through Folder picker, then I added it to FutureAccessList too. Though not works. Since the New db connection method expects the path, I am struggling for a long time, to use path in the Constructor.
//DbPathFromFAList = foldername.path (got from FutureAccess List) + mydbName.sqlite;
sqliteConn = new SQLiteConnection(new SQLitePlatformWinRT(), DbPathFromFAList);
As Rob said in this thread,
SQLite uses paths not streams and bypasses the file broker. It's database has to be in application data so the app has direct read/write permissions or install dir for read only.
You could not pass the path string parameter such as D: E: to SQLiteConnection method, even though you have used FilePicker to get the full file's permission.
In general, we use LocalFolder to store db file that could access directly.
DbFilePath = Path.Combine(ApplicationData.Current.LocalFolder.Path, "Sqlite.db");
sqliteConn = new SQLiteConnection(new SQLitePlatformWinRT(), DbFilePath);
And the specified path is located at:
C:\Users[Your User Name]\AppData\Local\Packages[Your Package Name]\LocalState
You can not set it to the D or E disk. If you have used external database, please copy the db file to the local folder. for more please refer this document.
The code in my project now:
local lsqlite3 = require "lsqlite3complete"
self.db_conn = lsqlite3.open("cost.db")
function showrow(udata,cols,values,names)
assert(udata=='test_udata')
for i=1,cols do
print('',names[i],values[i])
end
return 0
end
self.db_conn:exec('select * from cost',showrow,'test_udata')
It is no problem to select the cost records from the code above, but if I change like below and try to open it in memory:
self.db_conn = lsqlite3.open_memory("cost.db")
The code has no error but there is no records or tables inside when I do the query. How can I change my code so that I can open and put my database inside memory? Since I would like to access my data quickly in memory instead of keep connecting to a database.
A memory database is one that exists only in memory. That is, it doesn't get its data from a file. Because of that, open_memory doesn't take any parameters.
If you want to use a database that lives in a file, then that means accessing that file.
You should not need to "keep connecting to a database". You connect to it once at the beginning of the application and keep it open until your application terminates.
I needed a audio conversion library. After already pulling my hair..I have given up on the fact that there is no such audio library out there..every library out there has some or the other problem.
The only option left is ffmpeg which is the best but unfortunately you cannot use it in asp.net (not directly I mean). Every user on the website that will convert a file; will launch an exe?; I think I will hit the server memory max soon.
Bottom Line: I will try using ffmpeg.exe and see how many users it can support simultaneously.
I went to the ffmpeg website and in the windows download section I found 3 different version; static, shared and dev.
Does any one know which would be the best? All packed in one exe (static) or dll's separely and exe small, wrt using it in asp.net?
PS: any one has a good library out there..would be great if you can share.
Static builds provide one self-contained .exe file for each program (ffmpeg, ffprobe, ffplay).
Shared builds provide each library as a separate .dll file (avcodec, avdevice, avfilter, etc.), and .exe files that depend on those libraries for each program
Dev packages provide the headers and .lib/.dll.a files required to use the .dll files in other programs.
ffMpeg is the best library out there from what I have used but I wouldn't recommend trying to call it directly from asp.net.
What I have done, is accepted the upload, stored it on the server, or S3 in my case, then have a worker role (if using something like Azure) and a process that continuously looks and monitors for new files to convert.
If you needed a realtime like solution, you could update flags in your database and have an AJAX solution to poll the database to keep providing progress updates, then a link to download once the conversion is complete.
Personally my approach would be
Azure Web Roles
Azure Worker Role
ServiceBus
The WorkerRole starts up and is monitoring the ServiceBus Queue for messages.
The ASP.NET site uploads and stores the file in S3 or Azure
The ASP.NET site then records information in your DB if needed and sends a message to the ServiceBus queue.
The WorkerRole picks this up and converts.
AJAX will be needed on the ASP.NET site if you want a realtime monitoring solution. Otherwise you could send an email when complete if needed.
Using a queuing process also helps you with load as when you are under heavy load people just wait a little longer and it doesn't grind everything to a halt. Also you can scale out your worker roles as needed to balance loads, should it ever become too much for one server.
Here is how I run ffMpeg from C# (you will need to change the parameters for your requirements)
String params = string.Format("-i {0} -s 640x360 {1}", input.Path, "C:\\FilePath\\file.mp4");
RunProcess(params);
private string RunProcess(string Parameters)
{
//create a process info
ProcessStartInfo oInfo = new ProcessStartInfo(this._ffExe, Parameters);
oInfo.UseShellExecute = false;
oInfo.CreateNoWindow = true;
oInfo.RedirectStandardOutput = true;
oInfo.RedirectStandardError = true;
//Create the output and streamreader to get the output
string output = null; StreamReader srOutput = null;
//try the process
try
{
//run the process
Process proc = System.Diagnostics.Process.Start(oInfo);
proc.ErrorDataReceived += new DataReceivedEventHandler(proc_ErrorDataReceived);
proc.OutputDataReceived += new DataReceivedEventHandler(proc_OutputDataReceived);
proc.BeginOutputReadLine();
proc.BeginErrorReadLine();
proc.WaitForExit();
proc.Close();
proc.Dispose();
}
catch (Exception)
{
// Capture Error
}
finally
{
//now, if we succeeded, close out the streamreader
if (srOutput != null)
{
srOutput.Close();
srOutput.Dispose();
}
}
return output;
}
Is it possible to use a lock on one xml file that can potentially be written to from multiple aspx pages at the same time?
I'm asking because MSDN suggest that the lock statement should be used with a private static object instance as the expression, and since there are multiple pages involved i guess i can't use the same object on all the pages?
You can use the File.Open overload that takes a FileShare enum that is set to None. No other thread will be able to open the file until closed.
FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.None);
This piece of code will open the (existing) file specified in the path argument, for reading and without any sharing.
I am developing an application to read an excel spreadsheet, validate the data and then map it to a sql table. The process is to read the file via a streamreader, validate the data, manually make corrections to the excel spreadsheet, validate again -- repeat this process until all data validates.
If the excel spreadsheet is open, then when I attempt to read the data via a streamreader I get an error, "The process cannot access the file ... because it is being used by another process." Is there a way to remove the lock or otherwise read the data into a streamreader without having to open and close excel each time?
When you call File.Open to get the stream are you using the overload that allows you to specify FileAccess?
http://msdn.microsoft.com/en-us/library/y973b725.aspx
Note the parameters:
public static FileStream Open(
string path,
FileMode mode,
FileAccess access,
FileShare share
)
You can pass FileAccess.Read to the third param to indicate you only need read-only access. You should also set FileShare.Read to allow others to open the file read-only instead of locking it yourself. Note that if MS Excel opens the file with FileShare.None, you probably wont be able to access it.