Custom staging process that contains replication groups:
predefined (tables Catalogs, Products, Promotions...)
custom (one table A1MKWebPrice)
was working fine for months on daily basis.
Since the increase of data that is is imported in custom table A1MKWebPrice, replication process ends with status:
86% - Error: The refresh of caches failed!
ORM monitoring:
WebPriceBOCache - ObjectCacheGroup (Business Layer) defined in component framework:
I have cleared cache numerous ways in WebPriceBORepository after import of price:
LRUHashMap<String, Collection<WebPriceBO>> cache.clear();
webPricePOFactory.getEngine().getCacheManager().clearCache();
That clears Loaded (103.881 to 0) persistent objects A1MKWebPricePO but Count stays (103.884).
If i understand correctly cached persistent objects are in the JVM memory of an application server and they can not be controlled (deleted).
Should i somehow clear cache of persistent objects or increase JVM memory?
As I'm always a bit hesitant to increase memory size, cpu power or anything, as this is probably only a temporary solution: could you maybe check what Processor is being used and if that is the correct processor? For example: you might be using FullDomainSpecificStagingProcessor but maybe DeleteAppendDomainSpecificStagingProcessor is sufficient and would, imho, be more efficient.
Also please check your clearCache method, are you clearing cache for all PersistentObjectPOKeys? Isn't that nearly everything?
I honestly think you should go to Intershop Support with that question.
Related
I have this in my log:
2016-01-07 12:22:38,720 WARN [alfresco.cache.immutableEntityTransactionalCache] [http-apr-8080-exec-5] Transactional update cache 'org.alfresco.cache.immutableEntityTransactionalCache' is full (10000).
and I do not want to just increase this parameter without knowing what is really going on and having better insights of alfresco caches best practices!
FYI:
The warning appears when I list the element from document library root folder in a site. Note that the site does have ~300 docs/folder at that level, several of which are involved in current workflows and I am getting all of them in one single call (Client-side paging)
I am using an Alfresco CE 4.2.c instance with around 8k nodes
I ve seen this in my logs whenever you do a "big" transaction. By that I mean making a change to 100+ files in a batch.
Quoting Axel Faust:
The performance degredation is the reason that log message is a warning. When the transactional cache size is reached, the cache handling can no longer handle the transaction commit properly and before any stale / incorrect data is put into the shared cache, it will actually empty out the entire shared cache. The next transaction(s) will suffer bad performance due to cache misses...
Cache influence on Xmx depends on what the cache does unfortunately. The property value cache should have little impact since it stores granular values, but the node property cache would have quite a different impact as it stores the entire property map. I only have hard experience data from node cache changes and for that we calculated additional need of 3 GiB for an increase to four-times the standard cache size
It is very common to get these warnings.
I do not think that it is a good idea to change the default settings.
Probably you can try to change your code, if possible.
As described in this link to the alfresco forum by one of the Alfresco engineer, the value suggested by Alfresco are "sane". They are designed to work well in standard cases.
You can decide to change them, but you have to be careful because you can get lower performances than what you would get doing nothing.
I would suggest to investigate why your use of this webscript is causing the cache overflow and check if you can do something about it. The fact that you are retrieving 300 documents/folders in the same time, it is likely to be the cause.
In the following article you can find how to troubleshoot and solve issues with the cache.
Alfresco cache tuning
As described in that article, I would suggest to increase the log level for ehcache:
org.alfresco.repo.cache.EhCacheTracerJob=DEBUG
Or selectively adding the name of the cache that you want to monitor.
Looking at my ZEO workers I get to see quite a lot of:
2013-10-18T11:59:54 INFO ZPublisher.Conflict ConflictError at
/VirtualHostBase/http/www.domain.com:80/Plone/VirtualHostRoot/:
database conflict error (oid 0x533cd5, class
persistent.mapping.PersistentMapping) (78 conflicts (0 unresolved)
since startup at Mon Oct 14 04:09:45 2013)
As they are logged as INFO should I assume that is not harmful at all?
And I guess that if there is a conflict is because there are too much writes on the ZODB?
The conflicts are indeed caused because two requests are trying to change a PersistentMapping at the same time. One of these is then forced to retry the commit.
Use these entries to pinpoint bottlenecks in your application; perhaps replace the specific mapping with a BTree.OOBTree which minimizes conflicts by spreading key-value pairs out over separate persistent buckets.
Without traffic data and what that specific PersistentMapping holds or what your application does with it, it is impossible to say if 78 conflicts in 4 days is a lot or a little, and if it is worth your while switching to a different container.
Conflict errors are not -- in themselves -- harmful. The ZEO server will retry several times to resolve the error. But they are a sign of write-contention in the database, and a lot of them will indicate that you have a bottleneck in your current configuration. Your users soon will be complaining of poor performance.
You should probably begin analysis to determine if you've some add-on package that's doing excessive or very inefficient writes to the database. The worst case, for example, would be some code that's trying to write to the database on every page load like a traffic logger. The ZODB is optimized for reading, not writing, and those operations should be redesigned to put their data stores somewhere other than the ZODB.
If it's just content writes that are the problem, look to reduce catalog indexes and metadata. If at all possible, replace old Archetypes-style content with Dexterity content types. Dexterity is far more efficient in content creation.
I have a huge (>10GB) sqlite database that is shared among many (up to CPU core count) processes (same executable). This is a specialized application so RAM is not an issue and I want to cache as much of the database in memory. I have found about PRAGMA cache_size; and I am successfully using it but this blows the RAM usage out of proportion as each of many processes has its own private cache.
Now, I found SQLite Shared-Cache Mode but I can't see if this applies to different processes or just threads in one process. I have run some tests which confirm the latter but I am not sure if I am doing something wrong or whether something else needs to be done to make this work.
That page explains that "the same cache can be shared across an entire process".
In theory, you could try to configure your OS so that the entire database is held in the file cache.
If the amount of data in individual queries is small, it might be worthwhile to use a client/server database so that the caching needs to be done only in the server process.
I've tried to read up on Caching in ASP.NET and still have a few questions.
When using a Sql Cache Dependency ... I know that you can specify which tables will be monitored but if a change happens to any one of those tables does it reset the entire cache? I understand that I don't want to cache tables that will have frequent changes but we could end up with a good handful of cached tables and even if each table only gets a few updates a day, that could turn into 50ish resets of the cache daily (8 hour window).
I would be creating and maintaining this cache via a GAC DLL. A large number of different applications would be accessing that GAC at any one time. Does each application maintain its own copy of the cache or is it just stored in one global location (or possibly per app pool)?
Is there a physical location on the server where I can see how much space the Cache is currently consuming? This would be extremely pertinent if each application maintains its own Cache as that could end up taking large amounts of disk space.
Is there some way to physically force the cache to rebuild itself? I could see my boss assuming that the cache was at fault for a particular issue and I'd need to be able to rule that out at the rootest level. No "changing a record and saying that SHOULD rebuild the cache" but rather "doing [Action X] and KNOWING that whatever was in the cache is now gone"
Thanks in advance for your answers and time.
SqlCacheDependency only monitors tables in the old-style SQL 2000 approach, which relies on triggers and polling. The SQL 2005+ method monitors changes at the row level, and uses Service Broker. At the level of the Cache object, changes will invalidate just the Cache entries associated with the given SqlCacheDependency (not the entire cache).
Each application has a separate copy of the Cache. If you have many apps sharing the same data, you might consider creating a separate "caching server," and have your apps get their data from there, using WCF -- basically add another tier to your app.
You can look at a couple of cache-related performance counters, but if your concern is disk space, then there's nothing to worry about, since the ASP.NET cache is stored entirely in RAM. In addition, if RAM gets too full, one feature of the cache is that it will let go of old/infrequently referenced objects to make room for new objects.
The easiest way to force the cache to be dropped is to simply recycle your application or AppPool (which happens once a day or so by default anyway). If you want something more targeted, you would need to write some code to forcibly remove certain items from the cache, either using Cache.Remove() or using linked dependencies.
from top of my head:
Only that table's content will be invalidated.
Each web application has it's own cache.
Cache is stored in memory. and see this question How to determine total size of ASP.Net cache? regarding cache size
http://bit.ly/vsqNDl this may help
I am using ASP.NET's data caching API. For example:
HttpRuntime.Cache.Insert(my_data, my_key);
Is there any way to configure cache so its contents are preserved when the App Domain recycles?
I load many object into cache, but there is a substantial delay re-loading these every time the app domain restarts. Assume for this question that I can't prevent the appdomain restart due to a server configuration.
Is there any way to configure cache so
its contents are preserved when the
App Domain recycles?
No. The Cache object holds references in RAM. Period.
Alternatives:
Out-of-process Session state (although that's per-user)
Distributed cache
Use SQL Server as a cache (where it keeps data in memory, rather than on disk)
Write the objects to disk at the web tier
I generally prefer #3 myself, although there are scenarios where the others are appropriate.
Recycling the appdomain dumps the cache. If you want to get around this you'd need to use a distributed cache. Here's an example.
For your most expensive data you can cache the objects with a distributed cache such as Memcached or velocity. Depending on the size of the object and the length of runtime you could also serialize it to disk or to your database provided that the access speed to these resources is less than the time to create the object.
Now since the in-proc cache is super fast compared to any other solution since it just holds a reference to the memory object you will want to use it where possible. You can keep a copy of your cached object on disk until it is lost and then re-create it from that and place it in memory. Where this is tricky is when you have to expire the data so only use the cache/disk/sql combo where you won't need to expire/invalidate the data otherwise you will need to ensure that you clear both. This will also get around not being able to implement distributed caching on a shared server for example.