Write to a log file asynchronously from asp.net task - asp.net

Question

Using VB.Net (Framework version 4.5.1)
I have a program that sets up a list of (System.Threading.Tasks.Task) tasks that are executed as follows (only relevant code is shown):
po = New System.Threading.Tasks.ParallelOptions()
po.MaxDegreeOfParallelism = 5
Parallel.ForEach( task_list, po, AddressOf do_work )
The program works fine, but I want to add a log file rather than using just Console.WriteLine()
In the do_work() Sub, I want to write to a log file, for example:
Sub do_work( param As String )
Dim thread_id as String = "Thread ID " & System.Threading.Thread.CurrentThread.ManagedThreadId.ToString() & ": "
joblog_append( thread_id & "Beg" )
joblog_append( thread_id & "param = " & param )
joblog_append( thread_id & "End" )
End Sub
Ideally, I would like to create three functions such as:
joblog_open() to open a log file at the start o the program.
joblog_append() to append to the log file; callable from within any task
joblog_close() to close the log file
What is the correct way to implement such logging when the joblog_append() will be called from multiple tasks being executed on separate threads?
All attempts I have tried so far seem to be hit and miss; sometimes the data is written to the output file, sometimes it is not.
Any advice (or better yet, a code example) would be most appreciated.
Thanks in advance.

Answer 1

I think that the issue you have is due to the fact multiple threads are accessing the file at the same time.
A better approach would be to make sure only one thread access the file at a time. You could use a lock (see this SO) or append the messages to be written to a concurrentQueue of string, then process that queue on another thread.
The joblog_append calls _Logger.Log, which in turns enqueue the message and start a new thread to process the queue.
private _Logger as new Logger
Private Sub joblog_append(Message As String)
_Logger.Log(Message)
End Sub
The logger class performs the following.
Append message to to a concurrent queue
Create and start a task (if no one already running) to write queue content to the file.
Set the task to nothing when completed
In the event the task is already created, the message is enqueued and the While condition in the task itself should take care of any messages added while it's running.
'Missing: Idisposable, Fileaccess.Shared,
'TODO: remove debugger.break
Public class Logger
Public Property Messages As new ConcurrentQueue(Of string)
private _WorkerTask as Task
private event WorkerTaskCompleted
private _Stream as FileStream
private _Writer as StreamWriter
Public sub New()
_Stream = io.file.OpenWrite("Mylog.txt")
_Writer = New StreamWriter(_Stream)
End sub
Public sub Log(Message as string)
Messages.Enqueue(Message)
if _WorkerTask Is Nothing
_WorkerTask = New Task(sub()
While Messages.Any
Dim CurrentMessage as string = ""
if Messages.TryDequeue(CurrentMessage)
_Writer.WriteLine(CurrentMessage)
else
debugger.Break
End If
End While
_Writer.Flush
_Stream.Flush
RaiseEvent WorkerTaskCompleted
End Sub)
_WorkerTask.Start
End If
End sub
Private Sub Logger_WorkerTaskCompleted() Handles Me.WorkerTaskCompleted
_WorkerTask = Nothing
End Sub
End Class
Please note. This is my approach to this problem but I do not have anything similar implemented and tested. Therefore, you will have to make your tests to confirm it is working properly.