I signed up to GitHub and forked Symfony2. I tried to run the Symfony2 tests according to the instructions. I type > phpunit but every time, after some tests, I suddenly get out of memory message:
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 40 bytes) in ...\Symfony\Component\HttpFoundation\HeaderBag.php on line 37
Then, the tests result is something like that:
503 Service Unavailable503 Service Unavailable503 Service Unavailable503 Service
Unavailable503 Service Unavailable503 Service Unavailable503 Service Unavailable
What may be causing it?
I ran into the same problem using the 2.0 branch. The full set of tests require approx. 163Mb of memory and like first answer said php defaults to 128M. You have two options.
Change the default max memory allocation to something greater than 163Mb.
memory_limit = 200M in php.ini
or
copy phpunit.xml.dist to phpunit.xml
alter the attribute bootstrap to read bootstrap="tests/local-bootstrap.php"
add an entry into your .gitignore for tests/local-bootstrap.php
create local-bootstrap.php with something like:
local-bootstrap.php
<?php
ini_set('memory_limit', '200M'); // might need to adjust the value
require 'bootstrap.php' // no need to duplicate the bootstrap
This way allows you to add local changes without having to worry about git tacking any modifications to files that are local to your machine.
That is because the script exceeded the maximum amount of memory defined in php.ini
By default is 128M (134217728 bytes)
Here is explained http://php.net/memory-limit
Related
I had 2 similar questions before, however after more debugging I came to the conclusion the problem was (probably) not within my own code.
In my code I am trying to unzip a gzipped file, for this I wrote a small method;
<?php
namespace App\Helpers;
class Gzip
{
public static function unzip($filePath)
{
$outFilePath = str_replace('.gz', '', $filePath);
// Open our files (in binary mode)
$file = gzopen($filePath, 'rb');
$outFile = fopen($outFilePath, 'wb');
// Keep repeating until the end of the input file
while (!gzeof($file)) {
// Read buffer-size bytes
// Both fwrite and gzread and binary-safe
fwrite($outFile, gzread($file, 4096));
}
// Files are done, close files
fclose($outFile);
gzclose($file);
}
}
This should result in the unzipped file;
Gzip::unzip('path/to/file.csv.gz');
This is where it gets tricky, sometimes it will unzip the file and sometimes it will throw this exception; (keep in mind that this has nothing to do with the StreamHandler itself, this is a pure input/output error problem)
I can refresh the page as many times as I want but nothing will change, if I would try the gunzip command on the command line it will fail with sort off the same error;
Which file I am unzipping does not matter, it randomly happens to a random file.
Now it also won't matter if I run the gunzip command multiple times, but like I said these exceptions / errors happen randomly so they also randomly "fix" them self.
The application is written in Laravel 8.0, PHP7.4 running on a Homestead environment (Ubuntu 18.04.5 LTS) my base laptop runs on Windows 10.
To me it's super weird that this exception / error happens randomly and also randomly out of nowhere "fixes" itself, so my question is: how does this happen, why does this happen and ultimately how can I fix it.
errno=5 Input/output error is a failure to read/write the Linux file system.
A real server, you need to check the disk with fsck, etc...
Homestead running on Windows, I think we should look for the windows 10 homestead errno -5 issue.
winnfsd - https://github.com/winnfsd/vagrant-winnfsd/issues/96#issuecomment-336105685
If your Vagrant on Windows is using VirtualBox, HyperV can course this.
Try to disable HyperV for VirtualBox on the powershell, and reboot Windows
Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-Hypervisor
regards
The problem relied in me using Homestead (a Vagrant box) with NFS turned on, Vagrant + NFS + Windows = problems. There are many possible solutions to the problem, most exceptions regarding a errno5 come down to NFS + Vagrant.
The solution for me was to stop using NFS, for now this will be the accepted answer as this fixes my problem. However if someone manages to find a actual solution to this error I will accept that.
I solved it and I keep using NFS.
This situation happens to me usually when I'm dropping a database schema or creating a new one and fill it with a lot of data on my virtual box. I'm talking about volumes like around 50 Megabytes. I guess this is enough for virtual box to start re-scaling the virtual hard disc and it makes Ubuntu crazy and Kernel panicking.
So the solution is to reboot vagrant as many times as it takes for it to fix the issue.
That is what usually work for me:
make vagrant halt - it would go with errors
then vagrant up - it probably would not work
them vagrant halt - it probably would go with errors again
then vagrant up --provision - it would take time and probably also give errors
then vagrant halt - it should work this time
then vagrant up --provision - because "why not provisioning it again" and it is usually enough.
In 9 out of 10 cases it is enough. When it is not enough then I just create a new homestead.
I keep encountering the below error message in the sha256_migration.log. It doesn't restart after failure, however if I restart the artifactory service it begins the SHA256 migration from where it left off until it fails again.
2018-11-13 10:24:35,060 [art-exec-3] [ERROR] (o.a.s.j.m.s.Sha256MigrationJob:78) - Caught unexpected exception during SHA256 Migration job, operation will break.
org.springframework.core.task.TaskRejectedException: Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor#70a2137a] did not accept task: org.artifactory.schedule.aop.AsyncAdvice$$Lambda$654/1640835804#7dbf55d3
at org.springframework.core.task.support.TaskExecutorAdapter.submit(TaskExecutorAdapter.java:93)
at org.springframework.scheduling.concurrent.ConcurrentTaskExecutor.submit(ConcurrentTaskExecutor.java:143)
at org.artifactory.schedule.aop.AsyncAdvice.submitWorkQueueTask(AsyncAdvice.java:235)
at org.artifactory.schedule.aop.AsyncAdvice.submit(AsyncAdvice.java:217)
at org.artifactory.schedule.aop.AsyncAdvice.executeInvocation(AsyncAdvice.java:146)
at org.artifactory.schedule.aop.AsyncAdvice.invoke(AsyncAdvice.java:124)
at org.artifactory.schedule.aop.AsyncAdvice.invoke(AsyncAdvice.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy144.updateSha2(Unknown Source)
at org.artifactory.storage.jobs.migration.sha256.Sha256MigrationJob.migrationLogic(Sha256MigrationJob.java:134)
at org.artifactory.storage.jobs.migration.MigrationJobBase.migrationLoop(MigrationJobBase.java:106)
at org.artifactory.storage.jobs.migration.MigrationJobBase.runMigration(MigrationJobBase.java:83)
at org.artifactory.storage.jobs.migration.MigrationJobBase.onExecute(MigrationJobBase.java:73)
at org.artifactory.schedule.quartz.QuartzCommand.execute(QuartzCommand.java:48)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.artifactory.concurrent.ArtifactoryRunnable.run(ArtifactoryRunnable.java:30)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.artifactory.concurrent.ArtifactoryRunnable#4afb003 rejected from java.util.concurrent.ThreadPoolExecutor#33daf5aa[Running, pool size = 64, active threads = 64, queued tasks = 10000, completed tasks = 120723]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at org.artifactory.schedule.ArtifactoryConcurrentExecutor.execute(ArtifactoryConcurrentExecutor.java:69)
at org.springframework.core.task.support.TaskExecutorAdapter.submit(TaskExecutorAdapter.java:88)
... 19 common frames omitted
My artifactory.system.properties regarding sha256migration
##SHA2 Migration block
artifactory.sha2.migration.job.enabled=true
artifactory.sha2.migration.job.queue.workers=100
My setup:
Cloned production instances of Artifactory infrastructure into test instance (save for IP address and DNS records).
Ensure that DB and filestore configurations (db.properties and binarystore.xml) were updated accordingly on the cloned instances.
Things I've tried without luck:
I ran the Artifactory GC couple of times.
Increase the CPU count to 16
Increase the RAM to 16G
Ensure that I am running the latest Oracle Java 8u192
What I know:
It keep running fine for a while until it crashes.
When I restart artifactory service, the migration resumes and the total artifacts to migrate is lower.
I cannot keep restarting Artifactory in production to finish the sha256migrationjob, I have over 500k artifacts.
My question:
Any way method to restart the SHA256MigrationJob without restarting Artifactory?
Is there a way to find the artifact that it has trouble migrating to SHA256?
In the stack trace above, I feel the issue is at com.sun.proxy.$Proxy144.updateSha2(Unknown Source).
-- Workaround --
I ended up creating a new VM and installed a clean copy Artifactory 6.5.3 (with latest Oracle Java 8 Server-JRE). In the above issue, I was doing an in-place upgrade, just in a new folder.
I moved the necessary files in the etc to the new VM; such as master.key, binarystore.xml, db.properties and etc. I then executed the bin/installService.sh [user] [group], this creates the /etc/opt/jfrog/artifactory configuration symlink/folder move. My filestore and artifactory database are running on different VMs, thus only Artifactory and it's file configurations needed to be ported.
The new Artifactory 6.5.3 version started up without issues.
The sha256migrationjob is actually running without any problems. Last upgrade run I did, it worked fine without the job dying.
Notes: I did also sanely adjust the configuration values on the queue workers. https://www.jfrog.com/confluence/display/RTF/Checksum-Based+Storage#Checksum-BasedStorage-ConfiguringtheMigrationProcess
tl;dr decrease artifactory.sha2.migration.job.queue.workers to somewhere around 2 - 2 * number of cores.
What you are experiencing is an exhaustion of the ThreadPoolExecutor.
The size of the thread pool by default is 4 * [number of cores].
In your case 64 (16 cores * 4)
However the limit on the number of threads you have configured for the sha256 migration job is 100.
On an artifactory instance without any load, this will not cause any failures, because there is a queue backing up the thread pull (in your case the size is 10000).
In your case the thread pull and the queue are full.
If I understand correctly you have overcame the issue. But for other people bumping into this thread, I would recommend decreasing the value of artifactory.sha2.migration.job.queue.workers to no more than half of the thread pull number of threads.
So in your case 32: (16 * 4)/2
For some reason, I am suddenly getting the following Fatal error when clearing my cache in my Symfony2 project:
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 130968 bytes) in /Applications/XAMPP/xamppfiles/htdocs/instorecrm/vendor/twig/twig/lib/Twig/Environment.php on line 286
When I refer to the line 286 in the Environment.php file, it is this function:
public function getCacheFilename($name)
{
#trigger_error(sprintf('The %s method is deprecated and will be removed in Twig 2.0.', __METHOD__), E_USER_DEPRECATED);
$key = $this->cache->generateKey($name, $this->getTemplateClass($name));
return !$key ? false : $key;
}
I cannot think of anthing I've done that would cause this, the only change I have made is to the parameters.yml file to stipulate Gmail as my mail host. It does not seem to affect the working of the site (at least I have not found it to as yet) but it worries me that something might be broken.
I am using localhost, if that helps any.
Any help appreciated,
Thank you
Michael
Try cleaning the cache manually by deleting the app/cache folder.
I have a corrupted Lucene index. If I run "CheckIndex -fix" the problem is resolved, but as soon as I restart tomcat it becomes corrupted again.
The index directory is shared between two application servers running Liferay-Tomcat. I am fixing the index on 1 server and restarting that whilst the other is running. This is a production environment so I cannot bring them both down.
Any suggestions please?
Before fix, CheckIndex says:
Opening index # /usr/local/tomcat/liferay/lucene/0
Segments file=segments_5yk numSegments=1 version=FORMAT_SINGLE_NORM_FILE [Lucene 2.2]
1 of 1: name=_2vg docCount=31
compound=false
hasProx=true
numFiles=8
size (MB)=0.016
no deletions
test: open reader.........FAILED
WARNING: fixIndex() would remove reference to this segment; full exception:
java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:335)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:119)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:652)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:605)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:491)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:903)
WARNING: 1 broken segments (containing 31 documents) detected
WARNING: would write new segments file, and 31 documents would be lost, if -fix were specified
If you access your search index with more than one application server, I would suggest integrating a Solr Server. So you don't have the problem that 2 app servers are trying to write on the same file. This could be error prone as you already found out.
To get Solr up and running you have to follow those steps:
Install a Solr Server on any machine you like. A machine running only Solr would be quite preferable.
Install the Solr search portlet in Liferay
Adjust the config files according to the setup document of Sol Search portlet.
Here are some additional links:
http://www.liferay.com/de/marketplace/-/mp/application/15193648
http://www.liferay.com/de/community/wiki/-/wiki/Main/Pluggable+Enterprise+Search+with+Solr
I have this same error as others when running php ~/composer.phar update:
The following exception is caused by a lack of memory and not having swap configured
Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details
Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php:974
Stack trace:
0 [internal function]: Composer\Util\ErrorHandler::handle(2, 'proc_open(): fo...', 'phar:///home/te...', 974, Array)
1 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(974): proc_open('stty -a | grep ...', Array, NULL, NULL, NULL, Array)
2 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(784): Symfony\Component\Console\Application->getSttyColumns()
3 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(745): Symfony\Component\Console\Application->getTerminalDimensions()
4 phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php(675): Symfony\Component\Console\Application->getTerminalWidth()
5 phar:///home/tea/composer in phar:///home/tea/composer.phar/vendor/symfony/console/Symfony/Component/Console/Application.php on line 974
...but with a large instance: 4gb RAM and 4gb swap. The free RAM is never exhausted, let alone the available/cached RAM, and the swap isn't touched!
total used free shared buff/cache available
Mem: 3788 885 1908 9 993 2692
Swap: 3967 0 3967
It's the first time running composer update on this new machine, CentOS/CloudLinux 7.1 (with cPanel).
In desperation, I've tried
# php -dmemory_limit=1G ../composer.phar update --no-scripts --prefer-dist
and I've tried removing the composer.lock and vendor files and even tried adding a temporary swap file but it really doesn't seem to be a memory problem - could the error be misguided?
proc_open is not disabled and I also tried with shell fork bomb protection disabled but no jive.
Would love a heads up.
N.B. I'm aware of the advice to commit the composer.lock file and do a composer install but this instance is being used for dev (as was the previous CentOS/CloudLinux 6.x machine with smaller resource specs) so we need to use the same methods we were using previously.
OK so it was CloudLinux limiting the memory for the user to 1024mb, because it works when the limit is doubled to 2048mb.
That's the same setting on our previous server (CentOS/CloudLinux 6.x) but it looks like each version of CentOS is much more memory hungry than the rest.
Whats weird is that running composer with --profile shows the most it uses is 482mb. Even if it doubles when forking (as has been suggested) that's still below the 1024mb limit.
I ran into the same problem. My system had 1.5GB RAM free and it was not enough...Composer was eating too much memory very fast.
My only solution was to clear cache and update to latest version (1.4.2):
composer clear-cache
sudo composer selfupdate
That happens when you have low memory resources and you don't have swap enabled, i had the same problem and fixed using this few commands bellow, or you can create a swap partition or file, just make sure that you activate swap.
$ /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
$ /sbin/mkswap /var/swap.1
$ /sbin/swapon /var/swap.1
I hope it works for you...