Failed to inspect html elements under IFrame

Add-in Express™ Support Service
That's what is more important than anything else

Failed to inspect html elements under IFrame
Elements cannot be accessed under some IFrames 
ravinukala




Posts: 103
Joined: 2012-10-05
Hi,
We are not able to access the document under an iframe.

We have reported a similar issue and its support id is (ID:253642138).

You can reproduce this by using the following steps
URL: michaelkors.com
Goto "My Account" - Now you should see Sign In
Under Sign In inspect "Email Address" Text box.

This returns the conatiner Iframe of Email Address but not the Email Address element itself. You can see the same behavior with other elements like Password, Forgot password under Sign In.

We have seen similar issues in the past and we dont understand how we can resolve this. As this is happening on the login screen of the application, our customer is stuck and is not able to use our IE add in.

Need urgent help...

Thanks,
Ravi Nukala
Posted 03 Aug, 2016 09:36:07 Top
Sergey Grischenko


Add-in Express team


Posts: 7233
Joined: 2004-07-05
Hi Ravi,

Please try the code below:


using System;
using System.Text;
using System.Drawing;
using Accessibility;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.ComTypes;
using System.ComponentModel;
using System.Windows.Forms;
using IE = Interop.SHDocVw;


.......
        private void adxiehtmlDocEvents1_OnClick(object sender, object eventObject, ADXCancelEventArgs e)
        {
            int x = Control.MousePosition.X;
            int y = Control.MousePosition.Y;

            HitTest(x, y);
        }

        private void HitTest(int x, int y)
        {
            mshtml.IHTMLElement elemObj = null;

            try
            {
                SystemAccessibleObject accObj = SystemAccessibleObject.FromPoint(x, y);
                if (accObj.IAccessible != null)
                {
                    elemObj = MSAAToIHTMLElement(accObj.IAccessible);
                }
            }
            catch (Exception)
            {
            }

            if (elemObj != null)
            {
                // do something
            }
        }

        private mshtml.IHTMLElement MSAAToIHTMLElement(IAccessible pacc)
        {
            if (pacc == null)
                return null;

            IServiceProvider spServiceProvider = pacc as IServiceProvider;
            if (spServiceProvider == null)
                return null;

            object spHtmlElement;
            Guid guid = typeof(mshtml.IHTMLElement).GUID;

            int hRes = spServiceProvider.QueryService(ref guid, ref guid, out spHtmlElement);
            if (hRes != 0)
                return null;

            return spHtmlElement as mshtml.IHTMLElement;
        }

        [ComImport(), ComVisible(true), Guid("6D5140C1-7436-11CE-8034-00AA006009FA"),
        InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]
        internal interface IServiceProvider
        {
            [return: MarshalAs(UnmanagedType.I4)]
            [PreserveSig]
            int QueryService(ref Guid guidService, ref Guid riid, [MarshalAs(UnmanagedType.Interface)] out object ppvObject);
        }

        private class SystemAccessibleObject : IDisposable
        {
            private IAccessible iacc;
            private int childID;
            private bool disposed = false;

            public IAccessible IAccessible
            {
                get
                {
                    return iacc;
                }
            }

            public int ChildID
            {
                get
                {
                    return childID;
                }
            }

            public SystemAccessibleObject(IAccessible iacc, int childID)
            {
                if (iacc == null) 
                    throw new ArgumentNullException();

                if (childID != 0)
                {
                    try
                    {
                        object realChild = null;

                        try
                        {
                            realChild = iacc.get_accChild(childID);
                        }
                        catch (Exception)
                        {
                        }

                        if (realChild != null)
                        {
                            iacc = (IAccessible)realChild;
                            childID = 0;
                        }
                    }
                    catch (ArgumentException)
                    {
                    }
                }

                this.iacc = iacc;
                this.childID = childID;
            }

            public void Dispose()
            {
                if (!this.disposed)
                {
                    this.disposed = true;
                    this.iacc = null;
                }

                GC.SuppressFinalize(this);
            }

            public static SystemAccessibleObject FromPoint(int x, int y)
            {
                IAccessible iacc;
                object ci;

                POINT position = new POINT();
                position.x = x;
                position.y = y;

                IntPtr result = AccessibleObjectFromPoint(position, out iacc, out ci);

                if (result != IntPtr.Zero)
                    throw new Exception("AccessibleObjectFromPoint returned " + result.ToInt32());

                return new SystemAccessibleObject(iacc, (int)(ci ?? 0));
            }

            public static SystemAccessibleObject FromWindow(IntPtr window, AccessibleObjectID objectID)
            {
                IAccessible iacc = (IAccessible)AccessibleObjectFromWindow(window, (uint)objectID, new Guid("{618736E0-3C3D-11CF-810C-00AA00389B71}"));
                return new SystemAccessibleObject(iacc, 0);
            }

            public static SystemAccessibleObject MouseCursor
            {
                get
                {
                    return FromWindow(IntPtr.Zero, AccessibleObjectID.OBJID_CURSOR);
                }
            }

            public static SystemAccessibleObject Caret
            {
                get
                {
                    try
                    {
                        return FromWindow(IntPtr.Zero, AccessibleObjectID.OBJID_CARET);
                    }
                    catch (COMException)
                    {
                        return null;
                    }
                }
            }

            public static string RoleToString(int roleNumber)
            {
                StringBuilder sb = new StringBuilder(1024);
                uint result = GetRoleText((uint)roleNumber, sb, 1024);
                if (result == 0) throw new Exception("Invalid role number");
                return sb.ToString();
            }

            public static String StateToString(int stateNumber)
            {
                if (stateNumber == 0) return "None";
                int lowBit = stateNumber & -stateNumber;
                int restBits = stateNumber - lowBit;
                String s1 = StateBitToString(lowBit);
                if (restBits == 0) return s1;
                return StateToString(restBits) + ", " + s1;
            }

            public static string StateBitToString(int stateBit)
            {
                StringBuilder sb = new StringBuilder(1024);
                uint result = GetStateText((uint)stateBit, sb, 1024);
                if (result == 0) throw new Exception("Invalid role number");
                return sb.ToString();
            }

            public string Description
            {
                get
                {
                    return iacc.get_accDescription(childID);
                }
            }

            public string Name
            {
                get
                {
                    return iacc.get_accName(childID);
                }
            }

            public object Role
            {
                get
                {
                    return iacc.get_accRole(childID);
                }
            }

            public int RoleIndex
            {
                get
                {
                    object role = Role;
                    if (role is int)
                    {
                        return (int)role;
                    }
                    else
                    {
                        return -1;
                    }
                }
            }

            public string RoleString
            {
                get
                {
                    object role = Role;
                    if (role is int)
                    {
                        return RoleToString((int)role);
                    }
                    else if (role is string)
                    {
                        return (string)role;
                    }
                    else
                    {
                        return role.ToString();
                    }
                }
            }

            public Rectangle Location
            {
                get
                {
                    int x, y, w, h;
                    iacc.accLocation(out x, out y, out w, out h, childID);
                    return new Rectangle(x, y, w, h);

                }
            }

            public string Value
            {
                get
                {
                    return iacc.get_accValue(childID);
                }
            }

            public int State
            {
                get
                {
                    return (int)iacc.get_accState(childID);
                }
            }

            public string StateString
            {
                get
                {
                    return StateToString(State);
                }
            }

            public bool Visible
            {
                get
                {
                    return (State & 0x8000) == 0;
                }
            }

            public SystemAccessibleObject Parent
            {
                get
                {
                    if (childID != 0) return new SystemAccessibleObject(iacc, 0);
                    IAccessible p = (IAccessible)iacc.accParent;
                    if (p == null) return null;
                    return new SystemAccessibleObject(p, 0);
                }
            }

            public string KeyboardShortcut
            {
                get
                {
                    try
                    {
                        return iacc.get_accKeyboardShortcut(childID);
                    }
                    catch (ArgumentException) { return ""; }
                    catch (NotImplementedException) { return ""; }
                    catch (COMException) { return null; }
                }
            }

            public string DefaultAction
            {
                get
                {
                    try
                    {
                        return iacc.get_accDefaultAction(childID);
                    }
                    catch (COMException) { return null; }
                }
            }

            public void DoDefaultAction()
            {
                iacc.accDoDefaultAction(ChildID);
            }

            public SystemAccessibleObject[] SelectedObjects
            {
                get
                {
                    if (childID != 0) return new SystemAccessibleObject[0];
                    object sel;
                    try
                    {
                        sel = iacc.accSelection;
                    }
                    catch (NotImplementedException)
                    {
                        return new SystemAccessibleObject[0];
                    }
                    catch (COMException)
                    {
                        return new SystemAccessibleObject[0];
                    }
                    if (sel == null) return new SystemAccessibleObject[0];
                    if (sel is IEnumVARIANT)
                    {
                        IEnumVARIANT e = (IEnumVARIANT)sel;
                        e.Reset();
                        List<SystemAccessibleObject> retval = new List<SystemAccessibleObject>();
                        object[] tmp = new object[1];
                        while (e.Next(1, tmp, IntPtr.Zero) == 0)
                        {
                            if (tmp[0] is int && (int)tmp[0] < 0) break;
                            retval.Add(ObjectToSAO(tmp[0]));
                        }
                        return retval.ToArray();
                    }
                    else
                    {
                        if (sel is int && (int)sel < 0)
                        {
                            return new SystemAccessibleObject[0];
                        }
                        return new SystemAccessibleObject[] { ObjectToSAO(sel) };
                    }
                }
            }

            private SystemAccessibleObject ObjectToSAO(object obj)
            {
                if (obj is int)
                {
                    return new SystemAccessibleObject(iacc, (int)obj);
                }
                else
                {
                    return new SystemAccessibleObject((IAccessible)obj, 0);
                }
            }

            public IntPtr Window
            {
                get
                {
                    IntPtr hwnd;
                    WindowFromAccessibleObject(iacc, out hwnd);

                    return hwnd;
                }
            }

            public SystemAccessibleObject[] Children
            {
                get
                {
                    if (childID != 0) return new SystemAccessibleObject[0];

                    int cs = iacc.accChildCount, csReal;
                    object[] children = new object[cs * 2];

                    uint result = AccessibleChildren(iacc, 0, cs * 2, children, out csReal);
                    if (result != 0 && result != 1)
                        return new SystemAccessibleObject[0];
                    if (csReal == 1 && children[0] is int && (int)children[0] < 0)
                        return new SystemAccessibleObject[0];
                    SystemAccessibleObject[] retval = new SystemAccessibleObject[csReal];
                    for (int i = 0; i < retval.Length; i++)
                    {
                        retval[i] = ObjectToSAO(children[i]);
                    }
                    return retval;
                }
            }

            #region Equals and HashCode

            public override bool Equals(System.Object obj)
            {
                if (obj == null)
                {
                    return false;
                }
                SystemAccessibleObject sao = obj as SystemAccessibleObject;
                return Equals(sao);
            }

            public bool Equals(SystemAccessibleObject sao)
            {
                if ((object)sao == null)
                {
                    return false;
                }
                return childID == sao.childID && DeepEquals(iacc, sao.iacc);
            }

            private static bool DeepEquals(IAccessible ia1, IAccessible ia2)
            {
                if (ia1.Equals(ia2)) return true;
                if (Marshal.GetIUnknownForObject(ia1) == Marshal.GetIUnknownForObject(ia2)) return true;
                if (ia1.accChildCount != ia2.accChildCount) return false;
                SystemAccessibleObject sa1 = new SystemAccessibleObject(ia1, 0);
                SystemAccessibleObject sa2 = new SystemAccessibleObject(ia2, 0);
                if (sa1.Window != sa2.Window) return false;
                if (sa1.Location != sa2.Location) return false;
                if (sa1.DefaultAction != sa2.DefaultAction) return false;
                if (sa1.Description != sa2.Description) return false;
                if (sa1.KeyboardShortcut != sa2.KeyboardShortcut) return false;
                if (sa1.Name != sa2.Name) return false;
                if (!sa1.Role.Equals(sa2.Role)) return false;
                if (sa1.State != sa2.State) return false;
                if (sa1.Value != sa2.Value) return false;
                if (sa1.Visible != sa2.Visible) return false;
                if (ia1.accParent == null && ia2.accParent == null) return true;
                if (ia1.accParent == null || ia2.accParent == null) return false;
                bool de = DeepEquals((IAccessible)ia1.accParent, (IAccessible)ia2.accParent);
                return de;
            }

            public override int GetHashCode()
            {
                return childID ^ iacc.GetHashCode();
            }

            public static bool operator ==(SystemAccessibleObject a, SystemAccessibleObject b)
            {
                if (System.Object.ReferenceEquals(a, b))
                {
                    return true;
                }
                if (((object)a == null) || ((object)b == null))
                {
                    return false;
                }
                return a.iacc == b.iacc && a.childID == b.childID;
            }

            public static bool operator !=(SystemAccessibleObject a, SystemAccessibleObject b)
            {
                return !(a == b);
            }

            public override string ToString()
            {
                try
                {
                    return Name + " [" + RoleString + "]";
                }
                catch
                {
                    return "??";
                }
            }

            #endregion

            #region PInvoke Declarations

            [DllImport("oleacc.dll")]
            private static extern IntPtr AccessibleObjectFromPoint(POINT pt, [Out, MarshalAs(UnmanagedType.Interface)] out IAccessible accObj, [Out] out object ChildID);
            [DllImport("oleacc.dll")]
            private static extern uint GetRoleText(uint dwRole, [Out] StringBuilder lpszRole, uint cchRoleMax);

            [DllImport("oleacc.dll", ExactSpelling = true, PreserveSig = false)]
            [return: MarshalAs(UnmanagedType.Interface)]
            private static extern object AccessibleObjectFromWindow(
                IntPtr hwnd,
                uint dwObjectID,
                [In, MarshalAs(UnmanagedType.LPStruct)] Guid riid);

            [DllImport("oleacc.dll")]
            private static extern uint GetStateText(uint dwStateBit, [Out] StringBuilder lpszStateBit, uint cchStateBitMax);

            [DllImport("oleacc.dll")]
            private static extern uint WindowFromAccessibleObject(IAccessible pacc, out IntPtr phwnd);

            [DllImport("oleacc.dll")]
            private static extern uint AccessibleChildren(IAccessible paccContainer, int iChildStart, int cChildren, [Out] object[] rgvarChildren, out int pcObtained);

            #endregion
        }

        public enum AccessibleObjectID : uint
        {
            OBJID_WINDOW = 0x00000000,
            OBJID_SYSMENU = 0xFFFFFFFF,
            OBJID_TITLEBAR = 0xFFFFFFFE,
            OBJID_MENU = 0xFFFFFFFD,
            OBJID_CLIENT = 0xFFFFFFFC,
            OBJID_VSCROLL = 0xFFFFFFFB,
            OBJID_HSCROLL = 0xFFFFFFFA,
            OBJID_SIZEGRIP = 0xFFFFFFF9,
            OBJID_CARET = 0xFFFFFFF8,
            OBJID_CURSOR = 0xFFFFFFF7,
            OBJID_ALERT = 0xFFFFFFF6,
            OBJID_SOUND = 0xFFFFFFF5
        }
Posted 04 Aug, 2016 11:11:51 Top
ravinukala




Posts: 103
Joined: 2012-10-05
Thanks Sergey for sharing the above code.
Now we are trying to debug the code from VS2015/.net3.5 to locate the point where we can add the above code. We have also attached the IE process. Also, checked the box "Enable Unmanaged code debugging" for safer side. However, still breakpoint status symbol says "The breakpoint will not currently be hit no symbols loaded for this document"
Is there anyway to debug in normal mode?
Posted 05 Aug, 2016 04:32:20 Top
Andrei Smolin


Add-in Express team


Posts: 18794
Joined: 2006-05-11
Hello Ravi,

Please check section https://www.add-in-express.com/docs/net-deploying-debugging-tips.php#breakpoints.

Applied to IE, that means you need to have a similar .config file in the folder where iexplore.exe locates.


Andrei Smolin
Add-in Express Team Leader
Posted 05 Aug, 2016 04:38:08 Top
ravinukala




Posts: 103
Joined: 2012-10-05
Thanks Andrei. I have already followed this step. But still cannot see luck with breakpoints. Please find its content below.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
<startup>
<supportedRuntime version="v2.0.50727" />
</startup>
</configuration>

Also, it has been registered and process is attached. Still unable to debug the IE addon.
Posted 16 Aug, 2016 05:03:22 Top
ravinukala




Posts: 103
Joined: 2012-10-05
Also, please share the exact point where above code should suppose to go.
Thanks in advance.
Posted 16 Aug, 2016 07:45:38 Top
Andrei Smolin


Add-in Express team


Posts: 18794
Joined: 2006-05-11
Hello Ravi,

The file name of the .config file is iexplore.exe.config?

Please capture a video showing how you attach to the process and send me to the support email address; see {Add-in Express installation folder}\readme.txt. Make sure the email contains a link to this topic. Do you start a 32bit or 64bit IE?

>Also, please share the exact point where above code should suppose to go.

Sorry? The code that Sergey posted starts in the OnClick event of the ADXIEHTMLDocEvents component.


Andrei Smolin
Add-in Express Team Leader
Posted 16 Aug, 2016 09:04:59 Top
ravinukala




Posts: 103
Joined: 2012-10-05
Hi Andrei,

Yes, it is iexplore.exe.config on 64bit IE.

I have now embedded the Sergey code in ADXIEHTMLDocEvents_OnClick event.

I have emailed you the snapshots of 'attach to the process' with subject line "Supporting Attachment : Failed to inspect html elements under IFrame".

Thanks in advance.
Posted 17 Aug, 2016 07:59:17 Top
Andrei Smolin


Add-in Express team


Posts: 18794
Joined: 2006-05-11
Thank you Ravi,

You need to attach to another process. When I start IE with a single tab I have two iexplore.exe processes: the Type column for one of them contains "Managed (v.4.0.XXXX)" or "Managed (v.2.0.XXXX)". You need to attach to this process. Since your screenshot shows "Managed (v.4.0.XXXX)", I suppose you don't need to use .config file as it points to .NET 2.0.


Andrei Smolin
Add-in Express Team Leader
Posted 17 Aug, 2016 08:21:51 Top
ravinukala




Posts: 103
Joined: 2012-10-05
Yes Andrei. The provided screenshot only shows single IE process attachment. However I have attached the other required IE processes as well.
Thank you.
Posted 18 Aug, 2016 01:01:20 Top