Circular References in PHPExcel - infinite loop or wrong result - phpexcel

Question

I am using PHPExcel 1.8.0
I have read the posts regarding circular references, such as this one, but I am still running into issues.
If a spreadsheet contains one circular ref. formula, PHPExcel's calculations do not match MS Excel.
If a spreadsheet contains more than one circular ref, then PHPExcel goes into an infinite loop.
Here are the details of what I've done so far.
Assume a spreadsheet where A1 = B1 and B1 = A1+1, and Excel is set to 100 iterations. Here is my code:
// create reader
$objReader = PHPExcel_IOFactory::createReader('Excel2007');
$objReader->setReadDataOnly(true);
// load workbook
$objPHPExcel = $objReader->load($this->_path);
// set iterative calculations max count
PHPExcel_Calculation::getInstance($objPHPExcel)->cyclicFormulaCount = 100;
// calculate
$objWorksheet = $objPHPExcel->getSheetByName('Testing');
$data = $objWorksheet->rangeToArray('A1:B1');
echo '<pre>';
print_r ($data);
echo '</pre>';
// release resources
$objPHPExcel->disconnectWorksheets();
unset($objPHPExcel);
MSExcel results in A1 = 99, B1 = 100. My code produces this:
Array
(
[0] => Array
(
[0] => #VALUE!
[1] => #VALUE!
)
)
Further to that, if I add A2 = B2 and B2 = A2+1, and attempt to calculate (A1:B2), it goes into an infinite loop and eventually crashes:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 24 bytes) in C:\xampp\htdocs\cgc\bulldog\application\third_party\PHPExcel\Calculation.php on line 2837
Here is what I've done so far. In _calculateFormulaValue in Calculation.php:
Line 2383: $cellValue = ''; - this is the cause of the #Value! error. I changed that to $cellValue = 0;
Line 2400:
} elseif ($this->_cyclicFormulaCell == '') {
$this->_cyclicFormulaCell = $wsTitle.'!'.$cellID;
This is the cause of the infinite loop. $this->_cyclicFormulaCell does not get re-set to '' after the formula in row 1 is done, so this condition does not work for the formula in row 2.
I fixed this as follows, starting from Line 2389:
if (($wsTitle{0} !== "\x00") && ($this->_cyclicReferenceStack->onStack($wsTitle.'!'.$cellID))) {
if ($this->cyclicFormulaCount <= 0) {
return $this->_raiseFormulaError('Cyclic Reference in Formula');
} elseif (($this->_cyclicFormulaCount >= $this->cyclicFormulaCount) &&
($this->_cyclicFormulaCell == $wsTitle.'!'.$cellID)) {
// Olga - reset for next formula
$this->_cyclicFormulaCell = '';
return $cellValue;
} elseif ($this->_cyclicFormulaCell == $wsTitle.'!'.$cellID) {
++$this->_cyclicFormulaCount;
if ($this->_cyclicFormulaCount >= $this->cyclicFormulaCount) {
// Olga - reset for next formula
$this->_cyclicFormulaCell = '';
return $cellValue;
}
} elseif ($this->_cyclicFormulaCell == '') {
$this->_cyclicFormulaCell = $wsTitle.'!'.$cellID;
if ($this->_cyclicFormulaCount >= $this->cyclicFormulaCount) {
// Olga - reset for next formula
$this->_cyclicFormulaCell = '';
return $cellValue;
}
}
After these fixes, if I run $data = $objWorksheet->rangeToArray('A1:B2');, I get the following result:
Array
(
[0] => Array
(
[0] => 100 // should be 99
[1] => 100
)
[1] => Array
(
[0] => 100 // should be 99
[1] => 100
)
)
As you can see, the results from PHPExcel are not consistent with MS Excel. Why is this happening and how can I get around this?

Answer 1

Ok, I managed to debug this. My spreadsheet was very complicated, lots of circular refs. The most challenging is a scenario where A depends on B, B depends on both A and C, and C depends on B.
I also added a maxChange parameter, so the thing works like Excel. Otherwise it takes too long on my spreadsheet.
Anyway, here is a usage example:
$objPHPExcel = PHPExcel_IOFactory::load($path);
$objCalc = PHPExcel_Calculation::getInstance($objPHPExcel);
$objCalc->cyclicFormulaCount = 100;
$objCalc->maxChange = 0.001;
The two files that were modified are: Calculation.php and CalcEngine/CyclicReferenceStack.php
Here is the code (sorry Mark, I can't afford more time to submit it to git properly).
Calculation.php
add these to class properties:
private $_precedentsStack = array();
public $maxChange = 0;
replace _calculateFormulaValue() function with this:
public function _calculateFormulaValue($formula, $cellID=null, PHPExcel_Cell $pCell = null) {
$this->_debugLog->writeDebugLog("BREAKPOINT: _calculateFormulaValue for $cellID");
// Basic validation that this is indeed a formula
// We simply return the cell value if not
$formula = trim($formula);
if ($formula{0} != '=') return self::_wrapResult($formula);
$formula = ltrim(substr($formula,1));
if (!isset($formula{0})) return self::_wrapResult($formula);
// initialize values
$pCellParent = ($pCell !== NULL) ? $pCell->getWorksheet() : NULL;
$wsTitle = ($pCellParent !== NULL) ? $pCellParent->getTitle() : "\x00Wrk";
$key = $wsTitle.'!'.$cellID;
$data = array(
'i' => 0, // incremented when the entire stack has been calculated
'j' => 0, // flags the formula as having been calculated; can only be 0 or 1
'cellValue' => $pCell->getOldCalculatedValue(), // default value to start with
'precedents' => array(),
'holdValue' => FALSE // set to TRUE when change in value is less then maxChange
);
// add this as precedent
$this->_precedentsStack[] = $key;
// if already been calculated, return cached value
if (($cellID !== NULL) && ( $this->getValueFromCache($wsTitle, $cellID, $cellValue))) {
return $cellValue;
}
$this->_cyclicReferenceStack->getValueByKey($key, $data);
extract($data);
$this->_debugLog->writeDebugLog("iteration # $i");
// if already calculated in this iteration, return the temp cached value
if ($i >= $this->cyclicFormulaCount || $j == 1) {
return $cellValue;
}
// on stack, but has not yet been calculated => return default value
if (($wsTitle{0} !== "\x00") && ($this->_cyclicReferenceStack->onStack($key))) {
if ($this->cyclicFormulaCount <= 0) {
return $this->_raiseFormulaError('Cyclic Reference in Formula');
}
return $cellValue;
}
// calculate value recursively
$this->_cyclicReferenceStack->push($key);
$cellValue = $this->_processTokenStack($this->_parseFormula($formula, $pCell), $cellID, $pCell);
$this->_cyclicReferenceStack->pop();
// everything in precedent stack after the current cell is a precedent
// and every precedent's precedent is a precedent (aka a mouthfull)
while ( $this->_precedentsStack[ count($this->_precedentsStack) - 1 ] != $key ){
$data['precedents'][] = array_pop($this->_precedentsStack);
}
$data['precedents'] = array_unique($data['precedents']);
// check for max change
$oldValue = $this->_extractResult($data['cellValue']);
$newValue = $this->_extractResult($cellValue);
$data['cellValue'] = $cellValue;
$data['holdValue'] = (abs($oldValue - $newValue) < $this->maxChange);
// flag as calculated and save to temp storage
$data['j'] = 1;
$this->_cyclicReferenceStack->setValueByKey($key, $data);
// if this cell is a precedent, trigger a re-calculate
$tempCache = $this->_cyclicReferenceStack->showValues();
foreach ($tempCache as $tempKey => $tempData) {
if ( $tempData['holdValue'] == TRUE && ( in_array($key, $tempData['precedents'])) ) {
$tempData['holdValue'] = FALSE;
}
$this->_cyclicReferenceStack->setValueByKey($tempKey, $tempData);
}
// at the end of the stack, increment the counter and flag formulas for re-calculation
if (count($this->_cyclicReferenceStack->showStack()) == 0) {
$i++;
$this->_precedentsStack = array();
$tempCache = $this->_cyclicReferenceStack->showValues();
foreach ($tempCache as $tempKey => $tempData) {
$tempData['i'] = $i;
if ( ! $tempData['holdValue'] ) $tempData['j'] = 0;
$this->_cyclicReferenceStack->setValueByKey($tempKey, $tempData);
}
$this->_debugLog->writeDebugLog("iteration # $i-1 finished");
}
if ($i < $this->cyclicFormulaCount) {
$cellValue = $this->_calculateFormulaValue($pCell->getValue(), $cellID, $pCell);
} elseif ($cellID !== NULL) {
// all done: move value from temp storage to cache
$this->saveValueToCache($wsTitle, $cellID, $cellValue);
$this->_cyclicReferenceStack->removeValueByKey($key);
}
// Return the calculated value
return $cellValue;
} // function _calculateFormulaValue()
add this helper function:
private function _extractResult($result) {
if (is_array($result)) {
while (is_array($result)) {
$result = array_pop($result);
}
}
return $result;
}
CyclicReferenceStack.php
add a property:
private $_values = array();
add a bunch of functions:
public function setValueByKey($key, $value) {
$this->_values[$key] = $value;
}
public function getValueByKey($key, &$value) {
if (isset($this->_values[$key])) {
$value = $this->_values[$key];
return true;
}
return false;
}
public function removeValueByKey($key) {
if (isset($this->_values[$key])) {
unset($this->_values[$key]);
}
}
public function showValues() {
return $this->_values;
}