Some interesting decimal problem

Someone was using the AVG function in a column. The result should be 4,7 but answers displayed only 4. He modified the data format and added to the configuration to show 2 decimal places but he saw this value 4,00. He was using the formula like this AVG([Column_name]).

Oracle’s support decided figured out that:

The AVG function works as a division (in a Math context).
If the number you divide is an integer, the result will be an integer.
If the number you divide is a double, the result will be a double, this is how answers works.

In this case, the column had integer values, so the result should be an integer that’s the reason why customer saw a 4 instead 4,7.

If you divide an integer you will get an integer as a result. If you divide a double you will get a double as a result.

And provided the instructions:

1.- Go to column and click on the “Edit Formula” option.
2.- Click on the Function button.
3.- Select the CAST function in the Conversion section and click on the OK button.
4.- The formula should be looked like this CAST(“[column_name]” AS Double).
5.- Click on the Function button again.
6.- Select the AVG function in the Aggregation section and click on the OK button.
7.-The formula should be looked like this Avg(CAST(“[column_name]” AS Double)).
8.- Click on the “Column properties” option.
9.- Click on the “Data Format” tab and check in the “Override Default Data Format”.
10.- Choose the value 2 in the “Decimal Places” section.
11.- Click on the Ok button and click on the “Display Results” button.

One thought on “Some interesting decimal problem”

Leave a Reply