How can I convert a number with exponent value (such as 4.775900000000000e+004 ) to return a whole number, such as 47,759 using .net2.0 framework?
FYI, this is a weight category, not currency.
I guess it is a string, you can do int.Parse(s, NumberStyles.AllowExponent);. There might be problems with fractional parts, so use double.Parse, float.Parse or decimal.Parse when appropiate.
Related
I have an exponential value for e.g. 3.22122E+23
In Marklogic when I try- xs:decimal(3.22122E+23)
I get this error:
[1.0-ml] XDMP-CAST: (err:FORG0001) xs:decimal(xs:double("3.22122E23")) -- Invalid cast: xs:double("3.22122E23") cast as xs:decimal
A lower value for e.g. xs:decimal(3.22122E+18) gives me the correct result i.e. 3221220000000000000.
I see that this is because of decimal overflow and cannot be represented as a decimal data type but is there any way in Marklogic to handle and calculate such huge values?
Same question applies for the negative values(3.22122E-23) where I can handle and display data above 20 decimal places.
It would be helpful to clarify what kind of logic or calculations you are trying to accomplish and why exactly you need to convert the value to decimal. For example, to "display" the double value, you can use the standard format-number function without any conversion to decimal:
let $x := xs:double(3.22122E+23)
return format-number($x,"#,##0.00")
yields:
322,122,000,000,000,000,000,000.00
See https://docs.marklogic.com/fn:format-number for details regarding fn:format-number() usage.
See https://help.marklogic.com/Knowledgebase/Article/View/487/0/marklogic-server-and-the-decimal-type-implementation for details of the limitations of the xs:decimal type.
I am making an editor for a field with numbers. I tried a text field, but since it's a Number datatype coming in, it didn't go smoothly -- despite recasting strings as numbers etc.. it kept giving me NaN as the value. So I decided it would be best to go with a numeric stepper.
When I initially loaded it up it would drop all my decimals and only display my numbers as integers. I changed the stepIncrement to 0.1 and now it does show the decimals (a weird requirement imo).. but when I step up it occasionally gives me a value like '17.700000000000003' when I would expect 17.7. All of the numbers in my data have a single decimal place. I know I can write a dataformatter, but it seems like it shouldn't be necessary in this situation.
Is there another way I could deal with this?
You've stumbled upon the compromise of trying to represent decimal numbers in floating point binary formats like IEEE 754. Not all decimal numbers can be exactly represented. You can read up on this issue in great detail here:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
You can use Number.toFixed(fractionDigits:uint) to display to an arbitrary number of decimal places.
You can use the valueFormatFunction which takes the numeric value and formats it to a string. You will need to set explicit widths on your numeric steppers to make they fit though.
in your MXML
<s:NumericStepper valueFormatFunction="stepperFormatter"/>
in your script
protected function stepperFormatter(newValue:Number):String
{
return Math.ceil(newValue).toString()
}
I have two money fields in a SQL database called TotalClaimed and PartialSettlementAmountRecd.
They are declared as Decimals like so:
Public PartialSettlementAmountRecd As Decimal
Public TotalClaimed As Decimal
They both output the repsective amounts perfectly. I need to do a calculation on them, by subtracting PartialSettlementAmountRecd from TotalClaimed. I have tried the following, but it just outputs a random number, not the amount I require.
Dim NewSettAmount As Decimal = (ClaimDetail.TotalClaimed) - (ClaimDetail.PartialSettlementAmountRecd)
Response.Write("New Settlement Amount: £" & NewSettAmount)
Where am I going wrong? Thanks...
shouldn't it be:
Dim NewSettAmount As Decimal = (ClaimDetail.TotalClaimed) - (ClaimDetail.PartialSettlementAmountRecd)
<>
I need to do a calculation on them, by subtracting PartialSettlementAmountRecd from TotalClaimed
(ClaimDetail.PartialSettlementAmountRecd) - (ClaimDetail.TotalClaimed)
Aren't you doing the opposite of what you want? Aren't you subtracting TotalClaimed From PartialSettlementAmountRecd?
I apologise. The random figures I was getting were due to somebody entering a huge number in the Total Claimed field in the database whilst I was testing without telling me! The code actually works now with sensible figures.
I have a SQLite3 table with a column having format DECIMAL(7,2), but whenever I select rows with values not having a non-zero 2nd decimal place (eg. 3.00 or 3.10), the result always has trailing zero(s) missing (eg. 3 or 3.1). Is there any way that I can apply a formatting function in the SELECT statement so that I get the required 2dp? I have tried ROUND(), but this has no effect. Otherwise I have to keep converting the resulting column values into the required format for display (using Python in my case) every time I do a SELECT statement, which is a real pain.
I don't even mind if the result is string instead of numeric, as long as it has the right number of decimal places.
Any help would be appreciated.
Alan
SQLite internally uses IEEE binary floating point arithmetic, which truly does not lend itself well to maintaining a particular number of decimals. To get that type of decimal handling would require one of:
Fixed point math, or
IEEE decimal floating point (rather uncommon), or
Handling everything as strings.
Formatting the values (converting from floating point to string) after extraction is the simplest way to implement things. You could even hide that inside some sort of wrapper so that the rest of the code doesn't have to deal with the consequences. But if you're going to do arithmetic on the value afterwards then you're better off not formatting and instead working with the value as returned by the query, because the format and reconvert back to binary floating point (which Python uses, just like the vast majority of other modern languages) loses lots of information in the reduced precision.
I need to do calculations (division or multiplication) with very large numbers. Currently I am using Double and getting the value round off problems. I can do the same calculations accurately on C# using Decimal type. I am looking for a method to do accurate calculations in VB6.0 and I couldn't find a Decimal type in VB6.0.
What is the data type used for doing arithmetic calculations with large values and without getting floating point round off problems?
Depending on your data type, you can always use Currency, which is like Decimal(19,4), or 15 digits to the left of the decimal point, and 4 to the right.
In VB6, try using the variant datatype, and casting your numbers to that using CDec, as in:
Dim myDec As Variant
myDec = CDec(1.234)
See if that works.