I'm seeing an issue where Fn'Hex2Dec() is returning a negative value when the bitmask has a value in the leftmost position with an 8 character mask.

If I pass the string bitmask for TROP_ATE "0x08000000" it returns the correct value of 134217728.

Now I pass in the value for TROP_EXEC "0x80000000" it returns a negative -2147483648 instead of the expected positive value

And, the equivalent miame.ini - TRACE=INOUT,BASERR,SIGHUP,EXEC is "0x80018020", the expected decimal value is 2147581984, but it returning -2147385312.

Ashell version EL7 6.5.1739.2
fnhex2dec.bsi is the latest v105

Guessing this is an issue in MX_GTOCT.

Right - it's using a 32 bit signed integer and thus treating it as a negative value if the high bit is set. Surprising that it has taken this long to identify!

The fix will show up in 7.0.1752.3 shortly.

I can't update the production server this code needs to run on to 7 at this point. I'll have to modify the function to do the conversion in basic. It'll be interesting to see how much slower it is than in C

Actually, Fn'Hex2Dec() is nearly obsolete ever since ASB started recognizing "&h" and "&o" prefixes when converting from string to numeric. So the function could be reduced to:


If anything, it might even be faster than the MX_GTOCT version!
(Although I should note that this approach only works if the input string has the properly &h or &o prefix! But if it didn't, you could always test for it and add it before doing the assignment to the numeric variable.)

Wait - the above example is misleading. The symbol TROP_EXEC is defined as 0x80000000, but gets resolved into a regular numeric value by the define statement. What you really want is to convert a hex representation to decimal, so the example should be more like:

I can confirm that works as advertised.