Discussion:
Stack imbalance
(too old to reply)
GiuseppeDini
2009-09-13 21:51:44 UTC
Permalink
I'm a calling a c++ function in a dll from vb.net and I used different
calling conventions.
I fixed the error even if it didn't result in any problem.
I would like to understand why it was painless and I'm wondering how
Stack imbalance occurs.
Do parameters reamins on the stack? And if so aren't the mistakes
corrected when the vb.net function in which the dll call takes place
gets its end?
--
Giuseppe
MarkusSchaber
2009-09-14 06:13:56 UTC
Permalink
Hi, Guiseppe,
Post by GiuseppeDini
I'm a calling a c++ function in a dll from vb.net and I used different
calling conventions.
I fixed the error even if it didn't result in any problem.
I would like to understand why it was painless and I'm wondering how  
Stack imbalance occurs.
Do parameters reamins on the stack? And if so aren't the mistakes
corrected when the vb.net function in which the dll call takes place
gets its end?
That depends on which calling conventions you mixed up. Calling
conventions generally may differ in at least the following points:

- Which parameters are passed on the stack, and which are pased in
registers.
- Whether caller or calee cleans up the stack.
- The order in which paramters are put on the stack or which registers
are used.
- How exceptions are handled (especially between native and managed
code or through COM).

So if you "only" mixed up the exception handling, and your function
did not actually throw an exception, nothing fatal happened.

Which exact calling conventions did you mix up? And how many
parameters of which type does the function have?
GiuseppeDini
2009-09-14 17:26:30 UTC
Permalink
Post by MarkusSchaber
That depends on which calling conventions you mixed up. Calling
- Which parameters are passed on the stack, and which are pased in
registers.
- Whether caller or calee cleans up the stack.
- The order in which paramters are put on the stack or which registers
are used.
- How exceptions are handled (especially between native and managed
code or through COM).
So if you "only" mixed up the exception handling, and your function
did not actually throw an exception, nothing fatal happened.
Which exact calling conventions did you mix up? And how many
parameters of which type does the function have?
c++ dll:

extern "C" __declspec(dllexport) char * read_cliente ();



vb.net exe:
Public Shared Function x() As String
Dim ptr As System.IntPtr = read_cliente()
Dim str As String =
Runtime.InteropServices.Marshal.PtrToStringAnsi(ptr)
deletes(ptr)
Return str
End Function



<System.Runtime.InteropServices.DllImportAttribute("stringprovider.dll",
CharSet:=Runtime.InteropServices.CharSet.Ansi)> _
Public Shared Function read_cliente() As System.IntPtr
End Function

regards
--
Giuseppe
MarkusSchaber
2009-09-15 07:01:03 UTC
Permalink
Hi,
extern "C" __declspec(dllexport) char *  read_cliente ();
So it was a plain C function (thus throwing no exceptions) which does
not take any parameters. This combination will work with most calling
conventions "by accident".

http://en.wikipedia.org/wiki/X86_calling_conventions has more infos on
that topic.

HTH,
markus
GiuseppeDini
2009-09-15 07:08:52 UTC
Permalink
Post by MarkusSchaber
Hi,
extern "C" __declspec(dllexport) char *  read_cliente ();
So it was a plain C function (thus throwing no exceptions) which does
not take any parameters. This combination will work with most calling
conventions "by accident".
http://en.wikipedia.org/wiki/X86_calling_conventions has more infos on
that topic.
thank you.
--
Giuseppe
Giovanni Dicanio
2009-09-17 10:40:07 UTC
Permalink
Post by MarkusSchaber
Calling
- Which parameters are passed on the stack, and which are pased in
registers.
- Whether caller or calee cleans up the stack.
[..]

Markus: my understanding was exactly like yours, but I did a simple
VB.NET test and got confused.

The test consists in a simple WinForm app with a button that, when
clicked, calls a native function with a pure C interface, exported by a DLL.

The native function uses __cdecl calling convention, and is defined in
the DLL as follows:

<code language="CPP">

extern "C" __declspec(dllexport)
int __cdecl NativeFunction1(int a, int b, int c, int d)
{
return (a + b - c - d);
}

</code>

The complete VS2008 project can be found here:

http://www.geocities.com/giovanni.dicanio/temp/VBCallerTest.zip

I tried specifying on the managed (VB.NET) side both StdCall and CDecl
calling conventions, and the code seems to run fine in both cases.
But what looks odd to me is that the code generated by VB.NET compiler
seems to me the *same* in both cases of StdCall and CDecl.
So, is VB.NET just ignoring the calling convention specification?
What am I missing here?

Thanks,
Giovanni


----------------------------------------
Case of StdCall Calling Convention
----------------------------------------
<System.Runtime.InteropServices.DllImportAttribute("NativeDll.dll",
CallingConvention:=CallingConvention.StdCall)> _
Private Shared Function NativeFunction1(ByVal a As Integer, ByVal b
As Integer, ByVal c As Integer, ByVal d As Integer) _
As Integer
End Function
Private Sub ButtonCallTest1_Click(ByVal sender As System.Object,
ByVal e As System.EventArgs) Handles ButtonCallTest1.Click
00000000 push ebp
00000001 mov ebp,esp
00000003 push edi
00000004 push esi
00000005 push ebx
00000006 sub esp,54h
00000009 mov esi,ecx
0000000b lea edi,[ebp-38h]
0000000e mov ecx,0Bh
00000013 xor eax,eax
00000015 rep stos dword ptr es:[edi]
00000017 mov ecx,esi
00000019 xor eax,eax
0000001b mov dword ptr [ebp-1Ch],eax
0000001e mov dword ptr [ebp-3Ch],ecx
00000021 mov dword ptr [ebp-40h],edx
00000024 cmp dword ptr ds:[000D9CF8h],0
0000002b je 00000032
0000002d call 67986DB1
00000032 xor edx,edx
00000034 mov dword ptr [ebp-50h],edx
00000037 xor edx,edx
00000039 mov dword ptr [ebp-4Ch],edx
0000003c xor edx,edx
0000003e mov dword ptr [ebp-48h],edx
00000041 xor edx,edx
00000043 mov dword ptr [ebp-44h],edx
00000046 xor edx,edx
00000048 mov dword ptr [ebp-54h],edx
0000004b xor edx,edx
0000004d mov dword ptr [ebp-58h],edx
00000050 nop
' Call C code
Dim a As Integer = 10
00000051 mov dword ptr [ebp-44h],0Ah
Dim b As Integer = 20
00000058 mov dword ptr [ebp-48h],14h
Dim c As Integer = 1
0000005f mov dword ptr [ebp-4Ch],1
Dim d As Integer = 2
00000066 mov dword ptr [ebp-50h],2
Dim x As Integer
Dim y As Integer

x = NativeFunction1(a, b, c, d)
0000006d push dword ptr [ebp-4Ch]
00000070 push dword ptr [ebp-50h]
00000073 mov edx,dword ptr [ebp-48h]
00000076 mov ecx,dword ptr [ebp-44h]
00000079 call FF8C98E4
0000007e mov dword ptr [ebp-5Ch],eax
00000081 mov eax,dword ptr [ebp-5Ch]
00000084 mov dword ptr [ebp-54h],eax

Debug.Assert(a = 10)
00000087 cmp dword ptr [ebp-44h],0Ah
0000008b sete cl
0000008e movzx ecx,cl
00000091 call 65CEC380
00000096 nop
Debug.Assert(b = 20)
00000097 cmp dword ptr [ebp-48h],14h
0000009b sete cl
0000009e movzx ecx,cl
000000a1 call 65CEC380
000000a6 nop
Debug.Assert(c = 1)
000000a7 cmp dword ptr [ebp-4Ch],1
000000ab sete cl
000000ae movzx ecx,cl
000000b1 call 65CEC380
000000b6 nop
Debug.Assert(d = 2)
000000b7 cmp dword ptr [ebp-50h],2
000000bb sete cl
000000be movzx ecx,cl
000000c1 call 65CEC380
000000c6 nop

y = NativeFunction1(a, b, c, d)
000000c7 push dword ptr [ebp-4Ch]
000000ca push dword ptr [ebp-50h]
000000cd mov edx,dword ptr [ebp-48h]
000000d0 mov ecx,dword ptr [ebp-44h]
000000d3 call FF8C98E4
000000d8 mov dword ptr [ebp-60h],eax
000000db mov eax,dword ptr [ebp-60h]
000000de mov dword ptr [ebp-58h],eax
Debug.Assert(y = x)
000000e1 mov eax,dword ptr [ebp-58h]
000000e4 cmp eax,dword ptr [ebp-54h]
000000e7 sete cl
000000ea movzx ecx,cl
000000ed call 65CEC380
000000f2 nop

End Sub
000000f3 nop
000000f4 nop
000000f5 lea esp,[ebp-0Ch]
000000f8 pop ebx
000000f9 pop esi
000000fa pop edi
000000fb pop ebp
000000fc ret 4

----------------------------------------
----------------------------------------



----------------------------------------
Case of CDecl Calling Convention
----------------------------------------
<System.Runtime.InteropServices.DllImportAttribute("NativeDll.dll",
CallingConvention:=CallingConvention.Cdecl)> _
Private Shared Function NativeFunction1(ByVal a As Integer, ByVal b
As Integer, ByVal c As Integer, ByVal d As Integer) _
As Integer
End Function
Private Sub ButtonCallTest1_Click(ByVal sender As System.Object,
ByVal e As System.EventArgs) Handles ButtonCallTest1.Click
00000000 push ebp
00000001 mov ebp,esp
00000003 push edi
00000004 push esi
00000005 push ebx
00000006 sub esp,54h
00000009 mov esi,ecx
0000000b lea edi,[ebp-38h]
0000000e mov ecx,0Bh
00000013 xor eax,eax
00000015 rep stos dword ptr es:[edi]
00000017 mov ecx,esi
00000019 xor eax,eax
0000001b mov dword ptr [ebp-1Ch],eax
0000001e mov dword ptr [ebp-3Ch],ecx
00000021 mov dword ptr [ebp-40h],edx
00000024 cmp dword ptr ds:[00209CF8h],0
0000002b je 00000032
0000002d call 667F6DB1
00000032 xor edx,edx
00000034 mov dword ptr [ebp-50h],edx
00000037 xor edx,edx
00000039 mov dword ptr [ebp-4Ch],edx
0000003c xor edx,edx
0000003e mov dword ptr [ebp-48h],edx
00000041 xor edx,edx
00000043 mov dword ptr [ebp-44h],edx
00000046 xor edx,edx
00000048 mov dword ptr [ebp-54h],edx
0000004b xor edx,edx
0000004d mov dword ptr [ebp-58h],edx
00000050 nop
' Call C code
Dim a As Integer = 10
00000051 mov dword ptr [ebp-44h],0Ah
Dim b As Integer = 20
00000058 mov dword ptr [ebp-48h],14h
Dim c As Integer = 1
0000005f mov dword ptr [ebp-4Ch],1
Dim d As Integer = 2
00000066 mov dword ptr [ebp-50h],2
Dim x As Integer
Dim y As Integer

x = NativeFunction1(a, b, c, d)
0000006d push dword ptr [ebp-4Ch]
00000070 push dword ptr [ebp-50h]
00000073 mov edx,dword ptr [ebp-48h]
00000076 mov ecx,dword ptr [ebp-44h]
00000079 call FE8698E4
0000007e mov dword ptr [ebp-5Ch],eax
00000081 mov eax,dword ptr [ebp-5Ch]
00000084 mov dword ptr [ebp-54h],eax

Debug.Assert(a = 10)
00000087 cmp dword ptr [ebp-44h],0Ah
0000008b sete cl
0000008e movzx ecx,cl
00000091 call 64B5C380
00000096 nop
Debug.Assert(b = 20)
00000097 cmp dword ptr [ebp-48h],14h
0000009b sete cl
0000009e movzx ecx,cl
000000a1 call 64B5C380
000000a6 nop
Debug.Assert(c = 1)
000000a7 cmp dword ptr [ebp-4Ch],1
000000ab sete cl
000000ae movzx ecx,cl
000000b1 call 64B5C380
000000b6 nop
Debug.Assert(d = 2)
000000b7 cmp dword ptr [ebp-50h],2
000000bb sete cl
000000be movzx ecx,cl
000000c1 call 64B5C380
000000c6 nop

y = NativeFunction1(a, b, c, d)
000000c7 push dword ptr [ebp-4Ch]
000000ca push dword ptr [ebp-50h]
000000cd mov edx,dword ptr [ebp-48h]
000000d0 mov ecx,dword ptr [ebp-44h]
000000d3 call FE8698E4
000000d8 mov dword ptr [ebp-60h],eax
000000db mov eax,dword ptr [ebp-60h]
000000de mov dword ptr [ebp-58h],eax
Debug.Assert(y = x)
000000e1 mov eax,dword ptr [ebp-58h]
000000e4 cmp eax,dword ptr [ebp-54h]
000000e7 sete cl
000000ea movzx ecx,cl
000000ed call 64B5C380
000000f2 nop

End Sub
000000f3 nop
000000f4 nop
000000f5 lea esp,[ebp-0Ch]
000000f8 pop ebx
000000f9 pop esi
000000fa pop edi
000000fb pop ebp
000000fc ret 4

----------------------------------------
----------------------------------------
Ben Voigt [C++ MVP]
2009-09-21 18:17:50 UTC
Permalink
Post by Giovanni Dicanio
I tried specifying on the managed (VB.NET) side both StdCall and CDecl
calling conventions, and the code seems to run fine in both cases.
But what looks odd to me is that the code generated by VB.NET compiler
seems to me the *same* in both cases of StdCall and CDecl.
So, is VB.NET just ignoring the calling convention specification?
What am I missing here?
You have to look at the machine code generated during JIT compile to see the
difference.
Post by Giovanni Dicanio
Thanks,
Giovanni
----------------------------------------
Case of StdCall Calling Convention
----------------------------------------
<System.Runtime.InteropServices.DllImportAttribute("NativeDll.dll",
CallingConvention:=CallingConvention.StdCall)> _
Private Shared Function NativeFunction1(ByVal a As Integer, ByVal b As
Integer, ByVal c As Integer, ByVal d As Integer) _
As Integer
End Function
Private Sub ButtonCallTest1_Click(ByVal sender As System.Object, ByVal
e As System.EventArgs) Handles ButtonCallTest1.Click
00000000 push ebp
00000001 mov ebp,esp
00000003 push edi
00000004 push esi
00000005 push ebx
00000006 sub esp,54h
00000009 mov esi,ecx
0000000b lea edi,[ebp-38h]
0000000e mov ecx,0Bh
00000013 xor eax,eax
00000015 rep stos dword ptr es:[edi]
00000017 mov ecx,esi
00000019 xor eax,eax
0000001b mov dword ptr [ebp-1Ch],eax
0000001e mov dword ptr [ebp-3Ch],ecx
00000021 mov dword ptr [ebp-40h],edx
00000024 cmp dword ptr ds:[000D9CF8h],0
0000002b je 00000032
0000002d call 67986DB1
00000032 xor edx,edx
00000034 mov dword ptr [ebp-50h],edx
00000037 xor edx,edx
00000039 mov dword ptr [ebp-4Ch],edx
0000003c xor edx,edx
0000003e mov dword ptr [ebp-48h],edx
00000041 xor edx,edx
00000043 mov dword ptr [ebp-44h],edx
00000046 xor edx,edx
00000048 mov dword ptr [ebp-54h],edx
0000004b xor edx,edx
0000004d mov dword ptr [ebp-58h],edx
00000050 nop
' Call C code
Dim a As Integer = 10
00000051 mov dword ptr [ebp-44h],0Ah
Dim b As Integer = 20
00000058 mov dword ptr [ebp-48h],14h
Dim c As Integer = 1
0000005f mov dword ptr [ebp-4Ch],1
Dim d As Integer = 2
00000066 mov dword ptr [ebp-50h],2
Dim x As Integer
Dim y As Integer
x = NativeFunction1(a, b, c, d)
0000006d push dword ptr [ebp-4Ch]
00000070 push dword ptr [ebp-50h]
00000073 mov edx,dword ptr [ebp-48h]
00000076 mov ecx,dword ptr [ebp-44h]
00000079 call FF8C98E4
0000007e mov dword ptr [ebp-5Ch],eax
00000081 mov eax,dword ptr [ebp-5Ch]
00000084 mov dword ptr [ebp-54h],eax
Debug.Assert(a = 10)
00000087 cmp dword ptr [ebp-44h],0Ah
0000008b sete cl
0000008e movzx ecx,cl
00000091 call 65CEC380
00000096 nop
Debug.Assert(b = 20)
00000097 cmp dword ptr [ebp-48h],14h
0000009b sete cl
0000009e movzx ecx,cl
000000a1 call 65CEC380
000000a6 nop
Debug.Assert(c = 1)
000000a7 cmp dword ptr [ebp-4Ch],1
000000ab sete cl
000000ae movzx ecx,cl
000000b1 call 65CEC380
000000b6 nop
Debug.Assert(d = 2)
000000b7 cmp dword ptr [ebp-50h],2
000000bb sete cl
000000be movzx ecx,cl
000000c1 call 65CEC380
000000c6 nop
y = NativeFunction1(a, b, c, d)
000000c7 push dword ptr [ebp-4Ch]
000000ca push dword ptr [ebp-50h]
000000cd mov edx,dword ptr [ebp-48h]
000000d0 mov ecx,dword ptr [ebp-44h]
000000d3 call FF8C98E4
000000d8 mov dword ptr [ebp-60h],eax
000000db mov eax,dword ptr [ebp-60h]
000000de mov dword ptr [ebp-58h],eax
Debug.Assert(y = x)
000000e1 mov eax,dword ptr [ebp-58h]
000000e4 cmp eax,dword ptr [ebp-54h]
000000e7 sete cl
000000ea movzx ecx,cl
000000ed call 65CEC380
000000f2 nop
End Sub
000000f3 nop
000000f4 nop
000000f5 lea esp,[ebp-0Ch]
000000f8 pop ebx
000000f9 pop esi
000000fa pop edi
000000fb pop ebp
000000fc ret 4
----------------------------------------
----------------------------------------
----------------------------------------
Case of CDecl Calling Convention
----------------------------------------
<System.Runtime.InteropServices.DllImportAttribute("NativeDll.dll",
CallingConvention:=CallingConvention.Cdecl)> _
Private Shared Function NativeFunction1(ByVal a As Integer, ByVal b As
Integer, ByVal c As Integer, ByVal d As Integer) _
As Integer
End Function
Private Sub ButtonCallTest1_Click(ByVal sender As System.Object, ByVal
e As System.EventArgs) Handles ButtonCallTest1.Click
00000000 push ebp
00000001 mov ebp,esp
00000003 push edi
00000004 push esi
00000005 push ebx
00000006 sub esp,54h
00000009 mov esi,ecx
0000000b lea edi,[ebp-38h]
0000000e mov ecx,0Bh
00000013 xor eax,eax
00000015 rep stos dword ptr es:[edi]
00000017 mov ecx,esi
00000019 xor eax,eax
0000001b mov dword ptr [ebp-1Ch],eax
0000001e mov dword ptr [ebp-3Ch],ecx
00000021 mov dword ptr [ebp-40h],edx
00000024 cmp dword ptr ds:[00209CF8h],0
0000002b je 00000032
0000002d call 667F6DB1
00000032 xor edx,edx
00000034 mov dword ptr [ebp-50h],edx
00000037 xor edx,edx
00000039 mov dword ptr [ebp-4Ch],edx
0000003c xor edx,edx
0000003e mov dword ptr [ebp-48h],edx
00000041 xor edx,edx
00000043 mov dword ptr [ebp-44h],edx
00000046 xor edx,edx
00000048 mov dword ptr [ebp-54h],edx
0000004b xor edx,edx
0000004d mov dword ptr [ebp-58h],edx
00000050 nop
' Call C code
Dim a As Integer = 10
00000051 mov dword ptr [ebp-44h],0Ah
Dim b As Integer = 20
00000058 mov dword ptr [ebp-48h],14h
Dim c As Integer = 1
0000005f mov dword ptr [ebp-4Ch],1
Dim d As Integer = 2
00000066 mov dword ptr [ebp-50h],2
Dim x As Integer
Dim y As Integer
x = NativeFunction1(a, b, c, d)
0000006d push dword ptr [ebp-4Ch]
00000070 push dword ptr [ebp-50h]
00000073 mov edx,dword ptr [ebp-48h]
00000076 mov ecx,dword ptr [ebp-44h]
00000079 call FE8698E4
0000007e mov dword ptr [ebp-5Ch],eax
00000081 mov eax,dword ptr [ebp-5Ch]
00000084 mov dword ptr [ebp-54h],eax
Debug.Assert(a = 10)
00000087 cmp dword ptr [ebp-44h],0Ah
0000008b sete cl
0000008e movzx ecx,cl
00000091 call 64B5C380
00000096 nop
Debug.Assert(b = 20)
00000097 cmp dword ptr [ebp-48h],14h
0000009b sete cl
0000009e movzx ecx,cl
000000a1 call 64B5C380
000000a6 nop
Debug.Assert(c = 1)
000000a7 cmp dword ptr [ebp-4Ch],1
000000ab sete cl
000000ae movzx ecx,cl
000000b1 call 64B5C380
000000b6 nop
Debug.Assert(d = 2)
000000b7 cmp dword ptr [ebp-50h],2
000000bb sete cl
000000be movzx ecx,cl
000000c1 call 64B5C380
000000c6 nop
y = NativeFunction1(a, b, c, d)
000000c7 push dword ptr [ebp-4Ch]
000000ca push dword ptr [ebp-50h]
000000cd mov edx,dword ptr [ebp-48h]
000000d0 mov ecx,dword ptr [ebp-44h]
000000d3 call FE8698E4
000000d8 mov dword ptr [ebp-60h],eax
000000db mov eax,dword ptr [ebp-60h]
000000de mov dword ptr [ebp-58h],eax
Debug.Assert(y = x)
000000e1 mov eax,dword ptr [ebp-58h]
000000e4 cmp eax,dword ptr [ebp-54h]
000000e7 sete cl
000000ea movzx ecx,cl
000000ed call 64B5C380
000000f2 nop
End Sub
000000f3 nop
000000f4 nop
000000f5 lea esp,[ebp-0Ch]
000000f8 pop ebx
000000f9 pop esi
000000fa pop edi
000000fb pop ebp
000000fc ret 4
----------------------------------------
----------------------------------------
Loading...