Issue running ASPX page using Scheduled Task - asp.net

Question

I have a scheduled task set up to run Scan.aspx every 3 minutes in IE7. Scan.aspx reads data from 10 files in sequence. These files are constantly being updated. The values from the file are inserted into a database.
Sporadically, the value being read is truncated or distorted. For example, if the value in the file was "Hello World", random entries such as "Hello W", "Hel", etc. will be in the database. The timestamps on these entries appear completely random. Sometimes at 1:00 am, sometimes at 3:30 am. And some nights, this doesn't occur at all.
I'm unable to reproduce this issue when I debug the code. So I know under "normal" circumstances, the code executes correctly.
UPDATE:
Here is the aspx codebehind (in Page_Load) to read a text file (this is called for each of the 10 text files):
Dim filename As String = location
If File.Exists(filename) Then
Using MyParser As New FileIO.TextFieldParser(filename)
MyParser.TextFieldType = FileIO.FieldType.Delimited
MyParser.SetDelimiters("~")
Dim currentrow As String()
Dim valueA, valueB As String
While Not MyParser.EndOfData
Try
currentrow = MyParser.ReadFields()
valueA= currentrow(0).ToUpper
valueB = currentrow(1).ToUpper
//insert values as record into DB if does not exist already
Catch ex As Exception
End Try
End While
End Using
End If
Any ideas why this might cause issues when running multiple times throughout the day (via scheduled task)?

Answer 1

First implement a Logger such as Log4Net in your ASP.NET solution and Log method entry and exit points in your Scan.aspx as well as your method for updating the DB. There is a chance this may provide some hint of what is going on. You should also check the System Event Log to see if any other event is associated with your failed DB entries.
ASP.NET is not the best thing for this scenario especially when paired with a Windows scheduled task; this is not a robust design. A more robust system would run on a timer inside a Windows-Service-Application. Your code for reading the files and updating to the DB could be ported across. If you have access to the server and can install a Windows Service, make sure you also add Logging to the Windows Service too!
Make sure you read the How to Debug below
Windows Service Applications intro on MSDN: has further links to:
How to: Create Windows Services
How to: Install and Uninstall Services
How to: Start Services
How to: Debug Windows Service Applications]
Walkthrough: Creating a Windows Service
Application in the Component Designer
How to: Add Installers to Your Service Application
Regarding your follow up comment about the apparent random entries that sometimes occur at 1am and 3.30am: you should:
Investigate the IIS Log for the site when these occur and find out what hit(visited) the page at that time.
Check if there is an indexing service on the server which is visiting your aspx page.
Check if Anti-Virus software is installed and ascertain if this is visiting your aspx page or impacting the Asp.Net cache; this can cause compilation issues such as file-locks on the aspnet page in the aspnet cache; (a scenario for aspnet websites as opposed to aspnet web applications) which could give weird behavior.
Find out if the truncated entries coincide with the time that the files are updated: cross reference your db entries timestamp or logger timestamp with the time the files are updated.
Update your logger to log the entire contents of the file being read to verify you've not got a 'junk-in > junk-out' scenario. Be careful with diskspace on the server by running this for one night.
Find out when the App-Pool that your web app runs under is recycled and cross reference this with the time of your truncated entries; you can do this with web.config only via ASP.NET Health Monitoring.
Your code is written with a 'try catch' that will bury errors. If you are not going to do something useful with your caught error then do not catch it. Handle your edge cases in code, not a try catch.
See this try-catch question on this site.