I recently started this question in another thread (to which Reed Copsey
graciously responded) but I don't feel I framed the question well.
At the core of my question, I would like an illustration of how to gain
access to data AS it is being get/set.
I have Page.aspx.cs and, in the codebehind, I have a loop:
List<ServerVariable> files = new List<ServerVariable>();
for (i = 0; i <= Request.Files.Count - 1; i++)
{
m_objFile = Request.Files[i];
m_strFileName = m_objFile.FileName;
m_strFileName = Path.GetFileName(m_strFileName);
files.Add(new ServerVariable(i.ToString(),
this.m_strFileName, "0"));
}
//CODE TO COPY A FILE FOR UPLOAD TO THE
//WEB SERVER
//WHEN THE UPLOAD IS DONE, SET THE ITEM TO
//COMPLETED
int index = files.FindIndex(p => p.Completed == "0");
files[index] = new ServerVariable(i.ToString(),
this.m_strFileName, "1");
The "ServerVariable" type gets and sets ID, File, and Completed.
Now, I need to show the user the file upload "progress" (in effect,
the time between when the loop adds the ServerVariable item to the
list to when the Completed status changes from 0 to 1.
Now, I have a web service method "GetStatus()" that I would like to
use to return the files list (created above) as a JSON string (via
JQuery). Files with a completed status of 0 are still in progress,
files with a 1 are done.
MY QUESTION IS - what does the code inside GetStatus() look like? How
do I query List **as* it is being populated and
return the results real-time? I have been advised that I need to lock
the working process (setting the ServerVariable data) while I query
the values returned in GetStatus() and then unlock that same process?
If I have explained myself well, I'd appreciate a code illustration of
the logic in GetStatus().
Thanks for reading.
Have a look at this link about multi threading locks.
You need to lock the object in both read and write.
Related
I have written code to extract tables and name value pairs from pdf using Amazon Textract. I followed this example:
https://docs.aws.amazon.com/textract/latest/dg/async-analyzing-with-sqs.html
which was in sdk for java version 1.1.
I have refactored it for version 2.
This is an async process that only applies to multi page documents. When i get back the results it is pretty accurate for first page. But the consecutive pages are mostly empty rows. The documents i parse are scanned so the quality is not great. However if i take a jpg of individual pages and use the one page operation, i.e. AnalyzeDocumentRequest, each page comes out good. Also Amazon Textract tryit service renders the pages correctly.
So the error must be in my code but can't see where.
As you see it all happens in here :
GetDocumentAnalysisRequest documentAnalysisRequest = GetDocumentAnalysisRequest.builder().jobId(jobId)
.maxResults(maxResults).nextToken(paginationToken).build();
response = textractClient.getDocumentAnalysis(documentAnalysisRequest);
and i can't really do any intervention.
The most likely place I could make a mistake would be in the util file that gathers the page and table blocks i.e. here:
PageModel pageModel = tableUtil.getTableResults(blocks);
But that works perfectly for the first page, and i could also see in the response object above, that the number of blocks returned are much less.
Here is the full code:
private DocumentModel getDocumentAnalysisResults(String jobId) throws Exception {
int maxResults = 1000;
String paginationToken = null;
GetDocumentAnalysisResponse response = null;
Boolean finished = false;
int pageCount = 0;
DocumentModel documentModel = new DocumentModel();
// loops until pagination token is null
while (finished == false) {
GetDocumentAnalysisRequest documentAnalysisRequest = GetDocumentAnalysisRequest.builder().jobId(jobId)
.maxResults(maxResults).nextToken(paginationToken).build();
response = textractClient.getDocumentAnalysis(documentAnalysisRequest);
// Show blocks, confidence and detection times
List<Block> blocks = response.blocks();
PageModel pageModel = tableUtil.getTableResults(blocks);
pageModel.setPageNumber(pageCount++);
Map<String,String> keyValues = formUtil.getFormResults(blocks);
pageModel.setKeyValues(keyValues);
documentModel.getPages().add(pageModel);
paginationToken = response.nextToken();
if (paginationToken == null)
finished = true;
}
return documentModel;
}
Has anyone else encountered this issue?
Many thanks
if the response has NextToken, then you need to recall textract and pass in the NextToken to get the next batch of Blocks.
I am not sure how to do this in Java but here is the python example from AWS repo
https://github.com/aws-samples/amazon-textract-serverless-large-scale-document-processing/blob/master/src/jobresultsproc.py
For my solution, I did a simple if response['NextToken'] then recall method and concat the response['Blocks'] to my current list.
I have been working on a WCF web service, which is used by a mobile app that would send some data to it and save to DB.
One of the test case is that we try to append 2 (or more) records in the app, and the service is called to do a batch insert / update action.
Everything goes fine when I test using localhost, but when we test it using production server, only
the first record is saved, while the other record triggers the error message
Unable to determine a valid ordering for dependent operations...store-generated values.
I have no idea what is the cause and how to solve it. I have done some research and I am quite sure that the related model/DB table has NO circular dependency or self dependency.
Below is a snippet of the web service:
public void submit(List<SubmissionParameter> param){
using (var context = ObjectContextManager.AuditEnabledInstance){
foreach (var item in param){
ReadingSubmission readingSubmission = context.ReadingSubmissions.Where(p => p.ReadingSubmissionUniqueIdentifier == item.Readingsubmissionuniqueidentifier).SingleOrDefault();
if (readingSubmission == null){
readingSubmission = new ReadingSubmission();
context.ReadingSubmissions.AddObject(readingSubmission);
}
readingSubmission.ReadingSubmissionUniqueIdentifier = item.Readingsubmissionuniqueidentifier;
readingSubmission.SystemID = item.Systemid;
readingSubmission.UserID = item.Userid;
foreach (var record in item.Readings){
SystemReading systemReading = context.SystemReadings.Where(p => p.SystemReadingUniqueIdentifier == record.Systemreadinguniqueidentifier).SingleOrDefault();
if (systemReading == null){
systemReading = new SystemReading();
readingSubmission.SystemReadings.Add(systemReading);
}
systemReading.SystemReadingUniqueIdentifier = record.Systemreadinguniqueidentifier;
systemReading.MeasurementID = record.Measurementid;
}
context.SaveChanges();
}
}
}
ReadingSubmission and SystemReading is a 1 to many relation
SubmissionParameter is just a transmission object as the mobile client will send the JSON object to this web service.
I use Telerik Fiddler to post the JSON into this web service for testing, so I am quite sure the problem is not at the mobile client side.
Any help is appreciated! Thanks!
Finally I solve the problem though I am not quite sure why it works.
I move the context.SaveChanges() out of the foreach loop then it all works again for
both localhost and production
Hope it can help someone to save some time
I have an application which plays video using DirectShow.Net and also records video. When I try to record to a filename that has just been used for playback by DirectShow it fails as the file is still in use. Sometimes it will work but take anywhere from 5 - 60 seconds until the file is unlocked. Before the recording is attempted the playback graph has definitely been destroyed. The code for creating and destroying the graph is below. If I stop and start my application following playback I can record to the same filename and there are no file locking issues.
Can anyone advise how I can correctly release the source filter so the file is not locked?
Creating the graph
try
{
graphBuilder = (IGraphBuilder)new FilterGraph();
#if DEBUG
// "Connect to remote graph" in GraphEdit
rotEntry = new DsROTEntry(graphBuilder);
#endif
hr = graphBuilder.AddSourceFilter(filename, filename, out baseFilter);
DsError.ThrowExceptionForHR(hr);
vmr9 = (IBaseFilter)new VideoMixingRenderer9();
ConfigureVMR9InWindowlessMode();
hr = graphBuilder.AddFilter(vmr9, "Video Mixing Renderer 9");
DsError.ThrowExceptionForHR(hr);
FilterGraphTools.ConnectFilters(graphBuilder, baseFilter, "Output", vmr9, "VMR Input0", true);
}
Destroying the graph
if (vmr9 != null)
{
Marshal.ReleaseComObject(vmr9);
vmr9 = null;
windowlessCtrl = null;
}
if (graphBuilder != null)
{
// Remove and release all filters
FilterGraphTools.RemoveAllFilters(graphBuilder);
Marshal.ReleaseComObject(graphBuilder);
graphBuilder = null;
baseFilter = null;
}
#if DEBUG
if (rotEntry != null)
{
rotEntry.Dispose();
rotEntry = null;
}
#endif
Eventually the graph is a set of connected COM objects, and successful graph termination depends on correct release, without any leaked references. References that you might have left unreleased are leading to objects kept alive and possibly locking certain resources.
The best you can do is explicit termination/removal of individual objects:
Stop the graph
Remove all fitlers explicitly using IFilterGraph2.RemoveFilter
Use filter dependent methods calls to possibly terminate individual filters, such as by putting empty path to file source/sink filters
If even leak takes place, the graph should no longer reference resources. Note that you sometimes can also reuse filters if you are re-creating the graph.
My asp.net(c#) method looks as follows:
static public bool GetVideoLength(string fileName, out double length)
{
DirectShowLib.FilterGraph graphFilter = new DirectShowLib.FilterGraph();
DirectShowLib.IGraphBuilder graphBuilder;
DirectShowLib.IMediaPosition mediaPos;
length = 0.0;
try
{
graphBuilder = (DirectShowLib.IGraphBuilder)graphFilter;
graphBuilder.RenderFile(fileName, null);
mediaPos = (DirectShowLib.IMediaPosition)graphBuilder;
mediaPos.get_Duration(out length);
return true;
}
catch
{
return false;
}
finally
{
mediaPos = null;
graphBuilder = null;
graphFilter = null;
}
}
I got the duration with the above method. But my problem is i can't delete the physical file
after my operation. I used
File.Delete(FilePath);
While performing this action i got an exception as follows:
"The process cannot access the file because it is being used by another process."
My Os is windows 7(IIS 7)
Any one please help me to sort this out?
I've got no experience in coding directshow apps in C#, but plenty of experience in C++.
DirectShow is based on a technology called COM - which uses reference counting to tell it when an object is in use.
It would use a COM object to represent the IGraphBuilder for example.
In C++, we would have to deconstruct the graph, by removing all its filters, then release the graph.
I understand that C# has its own garbage collection etc., but unless you explicitly release the objects you use, they'll remain in memory.
It seems from the code you've quoted, that the graph is still opened, even though playback may have finished. In that case, it'll hold a reference to the file which you've played back, which would explain why you can't delete it - e.g. there's a read lock on the file.
Hope this points you in the right direction!
I have a lot of XSL files in my ASP.NET web app. A lot. I generate a bunch of AJAX HTML responses using this kind of generic transform method:
public void Transform(XmlDocument xml, string xslPath)
{
...
XslTransform myXslTrans = new XslTransform();
myXslTrans.Load(xslPath);
myXslTrans.Transform(xml,null, HttpContext.Current.Response.Output);
}
I'd like to move the XSL definitions into SQL Server, using a column of type xml.
I would store an entire XSL file in a single row in SQL, and each XSL is self-contained (no imports). I would read out the XSL definition from SQL into my XslTransform object.
Something like this:
public void Transform(XmlDocument xml, string xslKey)
{
...
SqlCommand cmd = new SqlCommand("GetXslDefinition");
cmd.AddParameter("#xslKey", SqlDbType.VarChar).Value = xslKey;
// where the result set has a single column of XSL: "<xslt:stylesheet>..."
...
SqlDataReader dr = cmd.ExecuteReader();
if(dr.Read()) {
SqlXml xsl = dr.GetSqlXml(0);
XslTransform myXslTrans = new XslTransform();
myXslTrans.Load(xsl.CreateReader());
myXslTrans.Transform(xml,null, HttpContext.Current.Response.Output);
}
}
It seems like a straightforward way to:
add metadata to each XSL, like lastUsed, useCount, etc.
bulk update/search capabilities
prevent lots of disk access
avoid referencing relative paths and organizing files
allow XSL changes without redeploying (I could even write an admin page that selects/updates the XSL in the database)
Has anyone tried this before? Are there any caveats?
EDIT
Caveats that responders have listed:
disk access isn't guaranteed to diminish
this will break xsl:includes
The two big issues I can see are:
We use a lot of includes to ensure that we only do things once, storing the XSLT in the database would stop us from doing that.
It makes updating XSLs more interesting - we've been quite happy to dump new .xsl files into deployed sites without doing a full update of the site. For that matter we've got bits of code that look for client specific xsl in a folder and those bits of code can reach back up to common code (templates) in the root - so I'm not sure about the redeploy thing at all, but this will depend very much on the particular use case, yours is certainly different to ours.
In terms of disk access, hmm... the db still has to go access the disk to pull the data and if you're talking about caching then the db isn't a requirement for enabling caching.
Have to agree about the update/search options - you can do stuff with Powershell but that needs to be run on the server and that's not always a good idea.
Technically I can see no reason why not (excepting the wish to do includes as above) but practically it seems to be fairly balanced with good arguments either way.
I store XSLTs in a database in my application dbscript. (However I keep them in an NVARCHAR column, since it also runs on SQL Server 2000)
Since users are able to edit their XSLTs, I needed to write a custom validator which loads the text of TextBox in a .Net XslCompiledTransform object like this:
args.IsValid = true;
if (args.Value.Trim() == "")
return;
try
{
System.IO.TextReader rd = new System.IO.StringReader(args.Value);
System.Xml.XmlReader xrd = System.Xml.XmlReader.Create(rd);
System.Xml.Xsl.XslCompiledTransform xslt = new System.Xml.Xsl.XslCompiledTransform();
System.Xml.Xsl.XsltSettings xslts = new System.Xml.Xsl.XsltSettings(false, false);
xslt.Load(xrd, xslts, new System.Xml.XmlUrlResolver());
xrd.Close();
}
catch (Exception ex)
{
this.ErrorMessage = (string.IsNullOrEmpty(sErrorMessage) ? "" : (sErrorMessage + "<br/>") +
ex.Message);
if (ex.InnerException != null)
{
ex = ex.InnerException;
this.ErrorMessage += "<br />" + ex.Message;
}
args.IsValid = false;
}
As for your points:
file I/O will be replaced by database-generated disk I/O, so no gains there
deployment changes to providing an INSERT/UPDATE script containing the new data