How to 'scrape' a window that pops up (1 Viewer)

[dklos]

Hello everyone!

I've been working on this problem for couple of days now and am exhausted from Google'ing.

I am writing a screen scraper to grab temperature data from a NOAA website. I can automatically navigate thru all the pages using the webrowser methods and sometimes having to use the '.Click' to trigger buttons on some of the pages.

Here's the problem. When I do the final 'click' to get the temperature data, NOAA opens up a new explorer window that contains the raw ascii data.

How can I access the new window to scrape it? What object is it hiding in?

Here's some of the code I'm using: (I left out the busy and readystate stuff)

Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.Navigate "https://ols.nndc.noaa.gov/sub-login.html"

ie.Document.all.Item("userid").Value = "myname"
ie.Document.all.Item("password").Value = "mypassword"

ie.Document.all.Item("B1").Click

(some more code for other screens, more clicks and submits)

ie.Document.Forms(0).Elements.all(65).Click

The click on this screen popups a new window containing the ascii data.

How can I get to it? I tried playing with parentwindow some. Is this the path I should take?

Many thanks for any suggestions or hints!

Daniel K.

 

[Treason]

I had a similiar problem once and i think I found the solution in parentwindow somewhere. Check vbforums.com I think thats where I found the answer before. If you get desperate I'll dig out my old project files.

 

[dklos]

I had a similiar problem once and i think I found the solution in parentwindow somewhere. Check vbforums.com I think thats where I found the answer before. If you get desperate I'll dig out my old project files.

Thanks, Treason, for the hint. I hadn't visited vbforums.com yet. I did leave a message in xtremevbtalk, vbcity and this forum.

I searched in vbforums and didn't find any clues. Yours was the only response I have gotten from any of the forums. It must be a difficult question I asked. Lots of people viewing... but only one reply.

If I don't get any hints within the next couple of days, I'll ask if you could possible find the code you had used.

Many thanks!

Daniel K.

 

[dklos]

From what I can gather (and by leaving messages in many different forum sites), once IE creates a new window, I lost access to it thru the webbrowser in VB.

I did figure out how to find the window using findwindow and can loop thru the child windows, but I can't figure out how to get to the html source code or figure out where the object is hiding.

Daniel

 

[Treason]

Hey Daniel,

I know what it's like. When I worked on the very same solution information was very scarce. I dug up my access db. One thing I am noticing is that I did not attempt to automate Internet Explorer, instead I used the webbrowser control.

You are not gonna like this answer. I grabbed the data from the local Internet cache.. Here's an example:

Option Compare Database
Option Explicit

Private Declare Function DeleteUrlCacheEntry Lib "Wininet.dll" _
   Alias "DeleteUrlCacheEntryA" _
  (ByVal lpszUrlName As String) As Long
  
Private WithEvents objWebPage As WebBrowser_V1

Sub Go()

wb.Navigate ("http://www.somesite.com")
DoCmd.SetWarnings False
Set objWebPage = wb.Object
Dim x As Integer
    wb.Navigate ("http://www.somesite.com/default.aspx")
    Wait
    wb.Document.Forms.IndexPageForm.Search.Value = DBN
    wb.Document.Forms.IndexPageForm.Search.Onblur
    Wait
    'This button click spawns the popup that loads the data I need
    wb.Document.Forms.IndexPageForm.SubmitButton.Click
    Wait
Call DeleteUrlCacheEntry("http://www.somesite.com/popup.aspx")
'Now I go back to that popup's url
wb.Navigate ("http://www.somesite.com/popup.aspx")
'And grab what I need
    Open "C:\html.txt" For Output As 1
    Print #1, wb.Document.documentelement.InnerText
    Close #1
end sub

Sub Wait()
    Do
    DoEvents
    Loop Until wb.ReadyState = READYSTATE_COMPLETE And wb.Busy = False
End Sub

'This may help also
Private Sub objWebPage_NewWindow(ByVal URL As String, ByVal Flags As Long, ByVal TargetFrameName As String, PostData As Variant, ByVal Headers As String, Processed As Boolean)
wb.Navigate2 URL 'Navigate to the page in the same window
Processed = True 'Don't open a new window
End Sub

This code is sloppy but It may give you some ideas...

Good luck

 

[dklos]

I didn't know you could ie.navigate to a page that it already open.

In my case, the popup is a dynamic page, so I can't navigate to the website directly for it.

I did get my project done... in a roundabout way. I find the window using findwindow, clicked on view source and then automatically found the notepad window and grabbed the 'edit' text from it.

I will try navigating to an existing open page and see what happens.