Lucene.NET --> access denied to segments - asp.net

Question

I have a problem with Lucene.NET. During an index, I receive the error 'Access to the path segments is denied'. Or sometimes 'Access to the path deletable is denied'. I eventually gave 'Everyone' full security rights to the index directory, but the problem still existed.
I then found out that during the index run, lucene renames the segments file to 'segments.new', and then this error happens. I guess some process still tries to read from the old segments file after it has been renamed? I have no clue as to why this happens, or how to fix this. Strangely enough, my co-developers can run the index on their computer without a problem.
The error happens at happens in Lucene.Net.Index.IndexModifier.AddDocument(Document).
Any ideas would be much appreciated.

Answer 1

I suspect that your IndexModifier is in contention with a Searcher.
Here's how I use Lucene.Net in my bug tracking app, BugTracker.NET, which seems to be working ok.
I create the index at app startup.
I create a searcher and keep it around so that the index isn't reloaded with each search. All threads share the same searcher. When the searcher searches, it grabs a lock, searches, then releases the lock, so that another thread can search. Forces the searches into single file is doable in my app because Lucene.NET is quick and a bug tracking system isn't THAT busy.
Meanwhile, I have an IndexWriter that updates the index when there is a data change. It is just changing a little bit so it does its task quick too. When it needs to run, it grabs the same lock, destroys the searcher, updates the index, and the re-recreates the searcher. The new searcher stays around until the next update of the index. The searcher always is working with an up-to-date index.
You can get the BugTracker.NET source and look at the files my_lucene.cs and search_text.aspx. It's all in those two files, and there isn't that much code.

Answer 2

This problem is caused by an online virus scanner locking the segments(.new) file. I have had to write a custom Lucene Directory implementation to work around this.

Answer 3

I think i found a solution.. well at least it worked for me..
I was testing for the "segments.new" problem and below you have the code .. so as you can see in a loop i created thousands of lucene documents (6000).. At about 1360 document an error appears saying that he couldn´t rename blablabla.. The code is written in c#.. basically you just have to insert a try catch (inside the loop) for the error and when the error pops up you just try again subtracting the int loop nunmber(y) by one (y = y - 1) ..
//-----------------Problem -------------------------------------
for (int y = 0; y < 6000; y++)
{
Document doc = new Document();
doc.Add(new Field("URL", "C:/Users/blabla/(convert-csharp)/IMssg", Field.Store.YES, Field.Index.TOKENIZED));
writer.AddDocument(doc);
}
//--------------------Solution----------------------------------------
IndexWriter writer = new IndexWriter("C:/Users/blabla/(convert-csharp)/IMssg", new StandardAnalyzer(), false);
for (int y = 0; y < 6000; y++)
{
try
{
Document doc = new Document();
doc.Add(new Field("URL", "C:/Users/blabla/(convert-csharp)/IMssg", Field.Store.YES, Field.Index.TOKENIZED));
writer.AddDocument(doc);
}
catch (Exception t)
{
y = (y < 0) ? 0 : y - 1;
string gfff = t.Message.ToString();
}
}
writer.Close();
Im not a english guy so im sory if there´s any error in some word...
by now
regards immanouel

Answer 4

I second Imma's solution. I had this problem also. The fix for me was to put the try/catch around IndexWriter.AddDocument(doc):
int attemptNo = 0;
while (attemptNo < 2)
{
try
{
writer.AddDocument(doc);
break;
}
catch (Exception e)
{
String ErrMsg = String.Format("{0} ({1}): While adding Document {2}/{3}, caught {4}", DateTime.Now, attemptNo, doc.GetField("kanji").StringValue(), doc.GetField("kana").StringValue(), e.Message);
attemptNo++;
System.Threading.Thread.Sleep(30);
Application.Current.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal, (Action)delegate()
{
ViewModel.Log.Add(ErrMsg);
});
}
reference: http://issues.apache.org/jira/browse/LUCENE-665:
"The gist of the issue is: on Windows, you sometimes see intermittant
"Access Denied" errors in renaming segments.new to segments or
deletable.new to deletable, etc. Lucene typically writes files first
to X.new and then renames then to X."

Answer 5

I read about this. However, I do not have any virus scanners running. I also disabled Vista Search Index for the index directory, killed the search index process from the task manager, to make sure no other process is locking the file. Unfortunately, to no avail. Moreover, the problem seems more to be that the 'segments' file it tries to access, is gone (since lucene renamed it to segments.new). I'm not sure if they are the same problems...