I have an exponential value for e.g. 3.22122E+23
In Marklogic when I try- xs:decimal(3.22122E+23)
I get this error:
[1.0-ml] XDMP-CAST: (err:FORG0001) xs:decimal(xs:double("3.22122E23")) -- Invalid cast: xs:double("3.22122E23") cast as xs:decimal
A lower value for e.g. xs:decimal(3.22122E+18) gives me the correct result i.e. 3221220000000000000.
I see that this is because of decimal overflow and cannot be represented as a decimal data type but is there any way in Marklogic to handle and calculate such huge values?
Same question applies for the negative values(3.22122E-23) where I can handle and display data above 20 decimal places.
It would be helpful to clarify what kind of logic or calculations you are trying to accomplish and why exactly you need to convert the value to decimal. For example, to "display" the double value, you can use the standard format-number function without any conversion to decimal:
let $x := xs:double(3.22122E+23)
return format-number($x,"#,##0.00")
yields:
322,122,000,000,000,000,000,000.00
See https://docs.marklogic.com/fn:format-number for details regarding fn:format-number() usage.
See https://help.marklogic.com/Knowledgebase/Article/View/487/0/marklogic-server-and-the-decimal-type-implementation for details of the limitations of the xs:decimal type.
Related
I am new to Pentaho, so please be gentle.
I am, perhaps naively, wanting to use a Formula to convert a six-character string in the form YYYYMM to the date representing the final day of that month.
I imagine doing this step by step using successive lines of the Formula: checking that the string is of the correct length and, if so:
extracting the year and converting it to integer (with error checking)
extracting the month and converting it to integer (also with error checking)
converting ([year], [month], 1) to a date (the first of the month)
adding a month
subtracting a day
Some of those steps may be combined but, overall, it relies on a succession of steps to achieve a final result.
Formula does not seem to recognise the values achieved along the way though, at least not by enclosing them in square brackets as you do with fields from previous objects in the mapping.
I suppose I could have a series of Formula objects one after the other in the mapping but that seems untidy and inefficient. If a single Formula object cannot have a series of values defined on successive lines, what is the point of even having lines? How do I use a value I have defined on a previous line?
The formula step isn’t the best way to achieve that. The resulting formula will be hard to read and quite cumbersome.
It’s better (and faster) to use a calculator step. A javascript step can also be used, and it will be easier to read, but slower (though that probably won't be a major issue).
So, one way forward is to implement this on a calculator step:
Create a copy of your string field as a Date
Create 2 constant fields: 1 and -1
Add 1 month to the date field
Subtract 1 day to the result
Create a copy of the result as a string.
See screenshot:
I am making an editor for a field with numbers. I tried a text field, but since it's a Number datatype coming in, it didn't go smoothly -- despite recasting strings as numbers etc.. it kept giving me NaN as the value. So I decided it would be best to go with a numeric stepper.
When I initially loaded it up it would drop all my decimals and only display my numbers as integers. I changed the stepIncrement to 0.1 and now it does show the decimals (a weird requirement imo).. but when I step up it occasionally gives me a value like '17.700000000000003' when I would expect 17.7. All of the numbers in my data have a single decimal place. I know I can write a dataformatter, but it seems like it shouldn't be necessary in this situation.
Is there another way I could deal with this?
You've stumbled upon the compromise of trying to represent decimal numbers in floating point binary formats like IEEE 754. Not all decimal numbers can be exactly represented. You can read up on this issue in great detail here:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
You can use Number.toFixed(fractionDigits:uint) to display to an arbitrary number of decimal places.
You can use the valueFormatFunction which takes the numeric value and formats it to a string. You will need to set explicit widths on your numeric steppers to make they fit though.
in your MXML
<s:NumericStepper valueFormatFunction="stepperFormatter"/>
in your script
protected function stepperFormatter(newValue:Number):String
{
return Math.ceil(newValue).toString()
}
I have a SQLite3 table with a column having format DECIMAL(7,2), but whenever I select rows with values not having a non-zero 2nd decimal place (eg. 3.00 or 3.10), the result always has trailing zero(s) missing (eg. 3 or 3.1). Is there any way that I can apply a formatting function in the SELECT statement so that I get the required 2dp? I have tried ROUND(), but this has no effect. Otherwise I have to keep converting the resulting column values into the required format for display (using Python in my case) every time I do a SELECT statement, which is a real pain.
I don't even mind if the result is string instead of numeric, as long as it has the right number of decimal places.
Any help would be appreciated.
Alan
SQLite internally uses IEEE binary floating point arithmetic, which truly does not lend itself well to maintaining a particular number of decimals. To get that type of decimal handling would require one of:
Fixed point math, or
IEEE decimal floating point (rather uncommon), or
Handling everything as strings.
Formatting the values (converting from floating point to string) after extraction is the simplest way to implement things. You could even hide that inside some sort of wrapper so that the rest of the code doesn't have to deal with the consequences. But if you're going to do arithmetic on the value afterwards then you're better off not formatting and instead working with the value as returned by the query, because the format and reconvert back to binary floating point (which Python uses, just like the vast majority of other modern languages) loses lots of information in the reduced precision.
I need to do calculations (division or multiplication) with very large numbers. Currently I am using Double and getting the value round off problems. I can do the same calculations accurately on C# using Decimal type. I am looking for a method to do accurate calculations in VB6.0 and I couldn't find a Decimal type in VB6.0.
What is the data type used for doing arithmetic calculations with large values and without getting floating point round off problems?
Depending on your data type, you can always use Currency, which is like Decimal(19,4), or 15 digits to the left of the decimal point, and 4 to the right.
In VB6, try using the variant datatype, and casting your numbers to that using CDec, as in:
Dim myDec As Variant
myDec = CDec(1.234)
See if that works.
I have an ASP page where I have 2 variables, strActualRate and strProposed.
The values are:
strActualRate = 33.30
strProposed = 33.3
So when I write the following line to compare:
if strActualRate <> strProposed then
Response.Writr "Both are not equal!"
end if
I am getting the output "Both are not equal", even though both are the same.
I am sure that I need to use some mathematical conversion function to compare.
Can anyone tell me how to solve this ?
Thanks in advance!
If I understand correctly, you think the two values are equal but because VBScript is comparing strings rather than numbers the two are coming back as not equal.
You're correct in the conversion idea, and here's the code:
if CDbl(strActualRate) <> CDbl(strProposed) then
Response.Write "Both are not equal!"
end if
That will convert your string values to numbers to do the comparison.
Your question doesn't really add up, so I'm not really sure what the problem is. I will try to clear up some things about data types and comparison.
You are using the prefix "str" for your variables which suggests that you intend to store string values in them, however you are instead storing numeric values in them. Either you are confused about how hungarian notation is used to keep track of the data type, or the code that you posted does not look like the code that you are actually using.
The numeric value 33.30 is exactly the same as the value 33.3. If you instead would have used the string values "33.30" and "33.3", they would be two strings that are not equal.
If your code is corrected (Response.Write instead of Response.Writr) so that it runs, it will not produce any output at all. As the values are equal, the condifion in the if statement evaluates to false.
If you do in fact assign string values to the variables, the code would output "Both are not equal!". This is just as expected as the strings are not equal. If you have strings and want to compare them as numerical values, you have to comvert them:
If CDbl(strActualRate) <> CDbl(strProposed) Then
Response.Write "Both are not equal!"
End If
Try casting the values to a double in the comparison statement with CDbl()
Are you intending to perform the comparison as strings, floating point numbers or some other method? If you are comparing them as strings, then clearly they are not equal, as one of them has an extra zero on the end. If you are comparing them as floating point numbers, then you generally want to use a comparison that involves taking the difference and checking that it is smaller than some small value. This is because floating point number calculations involve some degree of inaccuracy and comparisons between them can fail because of the underlying representation.