every time I save an object, I get this warning in the instance log:
[..]deployment/eggs/ZODB3-3.10.5-py2.6-linux-i686.egg/ZODB/Connection.py:660: UserWarning: The <class 'persistent.mapping.PersistentMapping'>
object you're saving is large. (29343532 bytes.)
and indeed, saving takes long. Putting a pdb into the place where this is printed triggers during the transaction commit and shows me indeed that a PersistentMapping is written of type:
{1: path_to_a_content_object,
2: path_to_other_content_object,
...
129000: path_to_yetanother_content_object }
It seems I have one entry for every content object in my site. As this happens during commit, I cannot see where that mapping is stored.
Does anybody have a pointer what that could be?
Thanks a lot!
Alex
Take a look at http://plone.org/documentation/kb/debug-zodb-bloat
Related
I have an import interface (not coded by me) that imports XML data and creates LedgerJournalTable (1) and LedgerJournalTrans (1..n) records.
When handling LJT dimensions, the code first checks that the dimension exists in AX, then inserts the data in the dimension[x] field. However, in the case that the dimension doesn't exist, a warning is shown to the user after the import run ends, but the data is still inserted as is.
And when the user goes to the LJT line after the import is complete, the erronous value is shown in the dimension field. When the lookup/drop-down of this dimension is clicked, the lookup does not open and AX client hangs. Ctrl+break will recover it, but the lookup never opens. You can delete the value, save, and the problem will still persist. You can manually enter an existing value and save, and the problem will still persist.
Problem extends to the table browser also.
Any idea why this is happening and how can it be fixed, other than not saving the erronous value in the first place (I have no idea why this is done this way in the first place)?
Thanks in advance.
Let me know if I'm reading this correctly.
User runs some process to import LJ table/trans records from XML.
If a bad dimension is inside XML, it shoves the data into the LJ trans dimension[x] field even though it's invalid, and presents a warning to user.
User views the journal and sees the bad data and attempts to use the lookup to correct it, but the lookup hangs/crashes.
Seems to me the issue may be that you've been shoving a bunch of bad data into AX and the lookup is trying to use table/edt relations that are invalid.
If I'm right, you need to go to SQL directly and query the ledger trans table and look for any bad dimension data and correct/remove it.
I suspect existing bad data is causing the lookup to fail and not merely whatever bad data you imported and are looking at.
Perhaps what caused the problem is, a user imported bad data, received a warning, ignored warning, clicked "post" as-is (with bad data) and now it's in AX? And now when you do a 2nd import, and try to use the lookup, it's crashing on that bad-data-relation.
Edited: So, while there was an corruption in the DB, the actual culprit was found: the standard AX code creating temp data for the dimension lookup - there was a mod code in Dimensions.insert() that wrote an XML file every dimensions were updated or inserted. This took so long in this case that it hang up the client. I put the code inside an if clause like so:
if(!this.isTemp())
{
// offending code
}
Problem solved.
I have to extend report's query to add a new field.
I've created extension of a query, joined required datasources and can see new field in the list of fields.
For some reason the report in the AOT is displaying with red cross sign:
In properties i can see error in metadata: "There was an error reading metadata. Make sure the metadata xml file(s) are accessible, are well formed and are not corrupted with duplicate or missing xml elements.
The exception message is: Element named: 'Copy1' of type 'ModelElement' already exists among elements: 'Copy1'.
Parameter name: item
Additional information:
AOT/Reports/Reports/WHSInvent"
There is an .xml of that object in packages local directory, there are no any duplicate names in any node of that report.
Any ideas how it can be fixed?
I've run into this before and there are a two things that come to mind.
Often times it's due to an incorrect merge where changes are merged and metadata is accidentally duplicated (in your case it's possible there are two xml nodes with the same name/id in the .rdl file)
If this report is checked in with corrupt metadata, you need to manually modify the RDL file, which is not great, but hopefully the error contains enough hints. Open the report rdl file in your favourite editor (report likely located in a similar path as this: K:\AosService\PackagesLocalDirectory\YOURMODEL\Reports) and look for an xml node with an attribute Name="Copy1". With luck, you have two duplicate nodes next to each other due to the merge. Remove the offending duplicate node, save, and refresh the AOT in Visual Studio.
If the error is in your local changes only (xml file is corrupted for whatever reason) and you are sure that your source control contains the correct version and you simply wish to overwrite the local contents with the source controlled version, follow these steps. Note: this will overwrite local changes.
First, undo pending changes.
Then force a get latest:
I'm trying to move object (i.e. image file) from one folder to another one and then rename it - overwrite with existing object (i.e. updating file).
I'm doing it in Zope (I use Plone 4.2) in the following way:
from zope.component.hooks import getSite
from zope.traversing.api import traverse
site = getSite()
src = traverse(site, "preview/")
dst = traverse(site, "images/")
source = src.manage_cutObjects(ids=[previewName])
dst.manage_pasteObjects(source)
dst.manage_delObjects(ids=[existingName])
That piece of code does what it should do, it moves the object from one folder to another, then, as manage_renameObject procedure doesn't allow overwritting I delete the existing object which will be replaced. But if I add this line of code to achieve the final goal:
dst.manage_renameObject(previewName, existingName)
the exception is thrown, that ID provided by existingName is invalid (because the object with such ID already exists and no matter that I have deleted it before).
It looks like I need some commit or update to finalize object moving (or wait etc) but I can't find anything about that.
UPDATE
I forget to add: all changes (object moving, object deletion) before exception in manage_renameObject was thrown weren't applied. Now, with transaction.commit() (as proposed by lewicki) the changes are applied but exception still occurs. Procedure transaction.savepoint() didn't help much, the exception was still thrown and changes weren't applied.
RESOLVED
I was confused initially by the error message:
<CENTER>
('Empty or invalid id specified', u'27')
</CENTER>
When I tried to reproduce the issue in the ZMI I got:
<CENTER>
The id "27" is invalid - it is already in use.
</CENTER>
and I realized that I needed to look into the code.
I dig into the installed Plone code and then located what I looked for in eggs/Zope2-2.13.18-py2.7.egg/OFS/ObjectManager.py module, in checkValidId() procedure.. There was the root of all my problems:
if not id or not isinstance(id, str):
if isinstance(id, unicode):
id = escape(id)
raise BadRequest, ('Empty or invalid id specified', id)
so, I strict object names (IDs) to str type and it all became working...
Neither commit() nor savepoint() even was needed (but initially when I was figuring out the issue and observing in ZMI that my files was neither moved nor deleted that was confused me a lot).
Thank you for your assistance!
It's dst.manage_renameObject(oldid, newid). This should work
Documentation
Yes, you're right - commit may be requred. I had a similar problem and it was solved that way:
from zope.component.hooks import getSite
from zope.traversing.api import traverse
import transaction
site = getSite()
src = traverse(site, "preview/")
dst = traverse(site, "images/")
source = src.manage_cutObjects(ids=[newName])
dst.manage_pasteObjects(source)
dst.manage_delObjects(ids=[existingName])
transaction.commit()
Now I think that
transaction.savepoint()
should be enough but you have to check on your own.
I'm doing the Wicked Code implementation of SqlSiteMapProvider, except in VB.NET.
There's a few things with the code that are causing issues, and I don't understand how it is supposed to work the way it's written in the article. I've provided the code straight from the article provided below. I've pasted the code here for ease of viewing
First issue - the depedency is instantiated BEFORE (lines 134-137) the tree is created (151-160) - so as soon as you add the depedency to the http.cache (165-167), the OnSiteMapChanged event (242) fires immmediately - making the entire proccess run again - and this loops many times until finally something makes it stop. (i stepped through it and counted the code looping at least 20 times before I gave up on trying to guess when it hit last)
OK, so to fix this I just moved the code to the create the dependency to AFTER the tree is built, right before inserting to http.cache (so HasChanged property is false when adding to http.cache, and you don't get stuck in this psuedo-ifinite-loop).
I still have a problem though - every time a page loads, the BuildSiteMap() hits and line 121 checks if _root is not null - it seems it is never null after it is first built... this is good because I don't want to hit the DB each time. Now, I insert a record into the table... the OnSiteMapChanged event never fires. As I browse pages on the app, the sitemap does not reflect the newly inserted record - stepping through the code, I see that the check at line 121 is still causing the function to short circuit... The sitemap will only refresh if i re-start Visual Studio, which causes the private _root field to become null again, and re-builds the sitemap, reflecting the new changes.. (refreshing the browser or starting new browser instances does not work)...
EDIT: THE ISSUE STEMMED FROM A SILLY 'SET NOCOUNT ON' LINE IN THE TOP OF MY STORED PROC. APPARENTLY THIS BREAKS THE QUERY NOTIFICATION. It seems that this statement is seen as a result set and that the second, actual query statement invalidates the result set resulting in a notification. This was very hard to find and nowhere in the MSDN documentation until I added the comment. Hope this saves someone else the miser I went through!
THE ISSUE STEMMED FROM A SILLY 'SET NOCOUNT ON' LINE IN THE TOP OF MY STORED PROC. APPARENTLY THIS BREAKS THE QUERY NOTIFICATION. It seems that this statement is seen as a result set and that the second, actual query statement invalidates the result set resulting in a notification. This was very hard to find and nowhere in the MSDN documentation until I added the comment. Hope this saves someone else the miser I went through!
I have a parts application that is built on Flex 3 and I having a problem with the parts list. The parts list uses two objects:
Part
ArrayOfPart
I cache the parts list locally in a SharedObject. This has been working fine for a couple months. I added a new Boolean property to the Part class and re-imported web services (which shouldn't have affected anything) and now, I can save the ArrayOfPart SharedObject fine, but when I try to retrieve the SharedObject, I get "ArgumentError: Error #1063: Argument count mismatch on ArrayOfPart. Expected 1, got 0. AND then it DELETES my SOL file completely.
(I used RegisterClass on Part and ArrayOfPart, so they both are serializable to SharedObjects)
Here's the steps I followed to get the error:
Save the shared object:
so = SharedObject.getLocal("PartsList");
so.data.AllParts = AllParts;
so.flush();
Verify the SharedObject:
The SharedObject PartsList.sol exists where it should
I opened the SharedObject file and the ArrayOfPart data looks valid
Restart the application and it retrieves the parts list from the SharedObject. This is the line that throws the Error #1063 and causes the sol file to be deleted:
so = SharedObject.getLocal("PartsList");
It looks like the data may not be well-formed when it's saved in the SharedObject? Is this possible?
I solved my own problem.
The ArrayOfPart had a constructor with a parameter. When I removed the constructor, the error went away.
By the way, this was Flash 9.
This looks like a tough one.
I don't think it is a formation or serialization issue. Maybe something else in your code is writing to the shared object?
Is there any way you can share your source? This would help as I personally don't have enough info to diagnose what is going on w/o seeing the rest of the code :(
Also....
Is there anything else that is writing to or changing this shared object?
Are all my objects created in AS3 or are some in MXML (I have noticed that the Flex compiler does not always do a good job figuring out MXML).
What version of Flash do I have?
If its Flash 10 does the same problem occur in Flash 9?
Does this happen in all browsers and on all platforms?
Just a hunch, but, since I installed Flash Player 10, I am seeing lots of errors with Shared Objects when browsing. Could it be related to newly introduced sandbox/security restrictions?