overflow (1 Viewer)

gemma-the-husky · Jun 15, 2012

not so much of a problem, but I was messing with some rgb colours and found out that

msgbox(255*256) overflows

and so does msgbox(clng(255*256))

why?

RainLover · Jun 16, 2012

Dave

(255*256) exceeds that allowed for a Variable.

And this happens before CLng, because of the Brackets.

Try

MsgBox (CLng(255) * 256)

Untested.

jdraw · Jun 16, 2012

Not sure where this fits exactly in the discussion but from M$oft it seems the prompt has to be a string ....

MsgBox(prompt [, buttons ] [, title ] [, helpfile ] [, context ] )

The MsgBox function syntax has these arguments:
Argument Description
prompt Required. String expression displayed as the message in the dialog box. The maximum length of prompt is approximately 1024 characters, depending on the width of the characters used. If prompt consists of more than one line, you can separate the lines using a carriage return character (Chr(13)), a linefeed character (Chr(10)), or carriage return – linefeed character combination (Chr(13) & Chr(10)) between each line.

spikepl · Jun 16, 2012

Think about how data types are determined for the result of an arithmetic operation. The result has to be determined prior to any assignment, and so has its datatype.

Dim a as Long

a= 256 * 256 ' this fails too

Banana · Jun 17, 2012

In case of literals, VBA will assume the smallest data type appropriate for the literal. Thus a literal '256' would be an integer. An integer 256 multiplied by integer 256 will overflow.

Code:

?TypeName(256)
Integer

To coerce a literal or a constant value into a desired size, you need to provide a type declaration character. This will succeed:

Code:

?256& * 256&

Or you can use conversion functions as previously suggested for the literals. You cannot use it for a constant value since conversion functions aren't themselves a constant. Thus type declaration character is the only sole for cases such as defining a constant to be used by a Windows API function:

Code:

Const SC_MOVE As Long = &HF010&

If you omit the type character, the value is assumed to be an integer and returns -4080 which is not what SC_MOVE is actually defined in the Windows header files. The correct value should be 61456. By coercing it into long, you can get the correct value.

RainLover · Jun 17, 2012

Banana said:
In case of literals, VBA will assume the smallest data type appropriate for the literal. Thus a literal '256' would be an integer. An integer 256 multiplied by integer 256 will overflow.

Code:

?TypeName(256) Integer

To coerce a literal or a constant value into a desired size, you need to provide a type declaration character. This will succeed:

Code:

?256& * 256&

Or you can use conversion functions as previously suggested for the literals. You cannot use it for a constant value since conversion functions aren't themselves a constant. Thus type declaration character is the only sole for cases such as defining a constant to be used by a Windows API function:

Code:

Const SC_MOVE As Long = &HF010&

If you omit the type character, the value is assumed to be an integer and returns -4080 which is not what SC_MOVE is actually defined in the Windows header files. The correct value should be 61456. By coercing it into long, you can get the correct value.

Banana

My testing appears to be wrong as none of your code works. What am I doing wrong?

Code:

Private Sub TestBanana()
    Print 256& * 256&
End Sub

Banana · Jun 17, 2012

Rain,

IME, "Print" has not worked for me before. I always had to say "Debug.Print" inside procedures. I assume you copied "?256& * 256&" and pasted it into a procedure instead of into Immediate windows. VBA editor changes it to the code that you displayed which then breaks it until you manually add in "Debug." preceding the "Print".

The first two code snippets were tested in Immediate Window, which is why they had the "?" prefixed, rather than "Debug.Print".

RainLover · Jun 17, 2012

Banana

I assume then that the purpose of your post is to promote the use of the Ampersand "&".

GPGeorge · Jun 17, 2012

I have always thought it was a good policy to rein in haphazard positing of assumptions, especially when they are unwarranted. On the other hand, it can be fun, I suppose.

gemma-the-husky · Jun 18, 2012

where all this came from - was that I had a form background in a green colour, and wanted to lighten the colour

the bgcolor on the control showed as 65280

after a bit of thought, i realised that this was pure green, and tried to get 255*256 to confirm it, hence the problem.

so green is rgb(0,255,0)

i tried rgb(0,128,0) but that was darker, not lighter

instead i needed rgb(128,255,128)

RainLover · Jun 18, 2012

GP

We can't agree with every thing posted here but calling Banana's post a " haphazard positing of assumptions" without reason is a bit much.

I have not read a lot of Banana's posts but from what I have read I believe him to be genuine in his comments.

Perhaps this is a " haphazard positing of assumptions". LOL

RainLover · Jun 18, 2012

Banana said:
IME, "Print" has not worked for me before. I always had to say "Debug.Print" inside procedures. I assume you copied "?256& * 256&" and pasted it into a procedure instead of into Immediate windows. VBA editor changes it to the code that you displayed which then breaks it until you manually add in "Debug." preceding the "Print".

The first two code snippets were tested in Immediate Window, which is why they had the "?" prefixed, rather than "Debug.Print".

Obviously I wasn't thinking. I just assumed that you were following the method as originally posted.

I must remind myself, "One should never assume especially when it comes to Coding."

GPGeorge · Jun 18, 2012

Plus ça change, plus c'est la même chose, eh?

RainLover · Jun 19, 2012

George

Big apologies. I did not realize who you were until someone sent me a PM.

How are things at UA. Did you get banned or something. In other words what are you doing here.

What's all this garbage you are posting?

rein in haphazard positing of assumptions

What has that got to do with the price of eggs in Alaska during the wet season. And more importantly what the hell does it mean.

RainLover · Jun 26, 2012

GPGeorge said:
Plus ça change, plus c'est la même chose, eh?

George

What happened?

Did you go walkabout?

Still trying to work out what you said.

Hope I did not scare you away. I wonder if your email address is current. Not to worry. If I don't hear from you soon I will post something in UA as a prompt.

overflow (1 Viewer)

gemma-the-husky

Super Moderator

RainLover

VIP From a land downunder

jdraw

Super Moderator

spikepl

Eledittingent Beliped

Banana

split with a cherry atop.

RainLover

VIP From a land downunder

Banana

split with a cherry atop.

RainLover

VIP From a land downunder

GPGeorge

Grover Park George

gemma-the-husky

Super Moderator

RainLover

VIP From a land downunder

RainLover

VIP From a land downunder

GPGeorge

Grover Park George

RainLover

VIP From a land downunder

RainLover

VIP From a land downunder

Similar threads

Users who are viewing this thread