I have a SQLite3 table with a column having format DECIMAL(7,2), but whenever I select rows with values not having a non-zero 2nd decimal place (eg. 3.00 or 3.10), the result always has trailing zero(s) missing (eg. 3 or 3.1). Is there any way that I can apply a formatting function in the SELECT statement so that I get the required 2dp? I have tried ROUND(), but this has no effect. Otherwise I have to keep converting the resulting column values into the required format for display (using Python in my case) every time I do a SELECT statement, which is a real pain.
I don't even mind if the result is string instead of numeric, as long as it has the right number of decimal places.
Any help would be appreciated.
Alan
SQLite internally uses IEEE binary floating point arithmetic, which truly does not lend itself well to maintaining a particular number of decimals. To get that type of decimal handling would require one of:
Fixed point math, or
IEEE decimal floating point (rather uncommon), or
Handling everything as strings.
Formatting the values (converting from floating point to string) after extraction is the simplest way to implement things. You could even hide that inside some sort of wrapper so that the rest of the code doesn't have to deal with the consequences. But if you're going to do arithmetic on the value afterwards then you're better off not formatting and instead working with the value as returned by the query, because the format and reconvert back to binary floating point (which Python uses, just like the vast majority of other modern languages) loses lots of information in the reduced precision.
Related
Why this ("Filter" in jqplay.org):
{"key":633447818234478180}
returns this ("Result" in jqplay.org):
{"key": 633447818234478200}
Original JSON doesn't matter.
Why is it changing 180 into 200? How can I overcome this? Is this a bug? A number too big?
I believe this is because jq can only represent legal JSON data and the number you've given is outside the range that can be represented without loss of precision. See also
What is JavaScript's highest integer value that a number can go to without losing precision?
If you need to work with larger numbers as strings in jq you may want to try this library:
jq-bigintA big integer library for working with possibly-signed arbitrarily long decimal strings. Written by Peter Koppstein (#pkoppstein) and released under the MIT license.
The base R function factor() interprets character elements consisting of blank space as valid factor elements instead of NA. What is the benefit of interpreting blank space character elements like this? Is it a legacy feature that is kept as it is to maintain compatibility?
Example:
factor(c("a","a","","b"))
I realize that this isn't an ordinary problem that can be solved with a reproducible example as a starting point, but I decided to give it a try anyway. The design decision to have factor() interpret blank space character elements like this confounds me. It seems to me that it would simplify things with no clear disadvantages to interpret these elements as NA instead.
What is the benefit of interpreting blank space character elements like this?
Because empty string data usually means “this is an empty string”, and not “this is missing data”.
It depends on the usage of course: an empty “name” field is most likely missing data. But an empty “title” field is just that: no title. How else would you encode lack of a title (assuming “Mr” and “Mrs” have a separate field, which may not be the case).
For factors, having empty labels makes less sense. However, R tends to convert strings to factors quite liberally (especially when reading tabular data from files), and treating all those empty values as NA would cause a lot of mis-annotated data. In general, such implicit conversions should always be lossless, i.e. preserve the whole domain of values being converted.
I am making an editor for a field with numbers. I tried a text field, but since it's a Number datatype coming in, it didn't go smoothly -- despite recasting strings as numbers etc.. it kept giving me NaN as the value. So I decided it would be best to go with a numeric stepper.
When I initially loaded it up it would drop all my decimals and only display my numbers as integers. I changed the stepIncrement to 0.1 and now it does show the decimals (a weird requirement imo).. but when I step up it occasionally gives me a value like '17.700000000000003' when I would expect 17.7. All of the numbers in my data have a single decimal place. I know I can write a dataformatter, but it seems like it shouldn't be necessary in this situation.
Is there another way I could deal with this?
You've stumbled upon the compromise of trying to represent decimal numbers in floating point binary formats like IEEE 754. Not all decimal numbers can be exactly represented. You can read up on this issue in great detail here:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
You can use Number.toFixed(fractionDigits:uint) to display to an arbitrary number of decimal places.
You can use the valueFormatFunction which takes the numeric value and formats it to a string. You will need to set explicit widths on your numeric steppers to make they fit though.
in your MXML
<s:NumericStepper valueFormatFunction="stepperFormatter"/>
in your script
protected function stepperFormatter(newValue:Number):String
{
return Math.ceil(newValue).toString()
}
I need to do calculations (division or multiplication) with very large numbers. Currently I am using Double and getting the value round off problems. I can do the same calculations accurately on C# using Decimal type. I am looking for a method to do accurate calculations in VB6.0 and I couldn't find a Decimal type in VB6.0.
What is the data type used for doing arithmetic calculations with large values and without getting floating point round off problems?
Depending on your data type, you can always use Currency, which is like Decimal(19,4), or 15 digits to the left of the decimal point, and 4 to the right.
In VB6, try using the variant datatype, and casting your numbers to that using CDec, as in:
Dim myDec As Variant
myDec = CDec(1.234)
See if that works.
I have an ASP page where I have 2 variables, strActualRate and strProposed.
The values are:
strActualRate = 33.30
strProposed = 33.3
So when I write the following line to compare:
if strActualRate <> strProposed then
Response.Writr "Both are not equal!"
end if
I am getting the output "Both are not equal", even though both are the same.
I am sure that I need to use some mathematical conversion function to compare.
Can anyone tell me how to solve this ?
Thanks in advance!
If I understand correctly, you think the two values are equal but because VBScript is comparing strings rather than numbers the two are coming back as not equal.
You're correct in the conversion idea, and here's the code:
if CDbl(strActualRate) <> CDbl(strProposed) then
Response.Write "Both are not equal!"
end if
That will convert your string values to numbers to do the comparison.
Your question doesn't really add up, so I'm not really sure what the problem is. I will try to clear up some things about data types and comparison.
You are using the prefix "str" for your variables which suggests that you intend to store string values in them, however you are instead storing numeric values in them. Either you are confused about how hungarian notation is used to keep track of the data type, or the code that you posted does not look like the code that you are actually using.
The numeric value 33.30 is exactly the same as the value 33.3. If you instead would have used the string values "33.30" and "33.3", they would be two strings that are not equal.
If your code is corrected (Response.Write instead of Response.Writr) so that it runs, it will not produce any output at all. As the values are equal, the condifion in the if statement evaluates to false.
If you do in fact assign string values to the variables, the code would output "Both are not equal!". This is just as expected as the strings are not equal. If you have strings and want to compare them as numerical values, you have to comvert them:
If CDbl(strActualRate) <> CDbl(strProposed) Then
Response.Write "Both are not equal!"
End If
Try casting the values to a double in the comparison statement with CDbl()
Are you intending to perform the comparison as strings, floating point numbers or some other method? If you are comparing them as strings, then clearly they are not equal, as one of them has an extra zero on the end. If you are comparing them as floating point numbers, then you generally want to use a comparison that involves taking the difference and checking that it is smaller than some small value. This is because floating point number calculations involve some degree of inaccuracy and comparisons between them can fail because of the underlying representation.