Problem getting HTML data from Web object

[johnraus]

Hi, I have a little problem getting HTML source from a secured website.
I need a Table with Data thats on this website
(HTTPS with logins/coockies and sessions and ...?)

This website is very tricky, but browsing with Web object is no problem ! event right click and view source works, but it shuold be an automated task.
I tried HTTPS.DownloadSecure / HTTP.SubmitSecure with correct ports and Logins to get file locally but no solution.

Now I created a new Generic TextOnly Printer, to store a file locally and use Web.Print to store the Text Data in a file. Then I process this data to a Table and back to HTML again ... ... But this is very sloppy and timeconsuming since the table is messed up by Text2File printing.

Does anybody have a more proper solution, to get the HTML source from the Web object ... ?

[Worm]

do you want the source or the content on the page?

[johnraus]

do you want the source or the content on the page?

I would like to have the source.
This is because it's easier to get Data from the tables in the webpage

[johnraus]

do you want the source or the content on the page?

Some additional info:

I would like to login on a Website (the tricky one, see first post) with the Web Object

in AMS I'll make a button wich navigates to particular page on this server(this is already O.K.), get source code of displayed page, extract some data from HTML source code, run some statistics and create a local HTML file with a report of the calculations.

This is why I would like to be able to access HTML Source code from the page currently being displayed in the web object

This would make my AMS project really usefull.

PS
the second issue would be cool to automate a search on this website and see the results in the Web.Object,
but then I would need to POST Data to the WEB.OBJECT
the search URL is like:
https://aaa.bbb.com:1234/ABC/search_doc.jsp
Post DATA like:
Parameter: (Mime Type) Value application/x-www-form-urlencoded
Parameter: docid Value:123456789-012
tried HTTP.Submitsecure, but it seems it's not the same session as the WEB OBJECT so this fails, The only thing I seem to be able to automate is Browsing this website with the web object.

Any Help would really be arappriciated

[Worm]

Getting the source is simple enough once you're on the page

Web.LoadURL("Web1", "javascript:alert(document.all[0].outerHTML)")

How to get that info into AMS is the hard part. I tried passing it in the URL, but it is truncated. I thought maybe it could be set as a cookie, then retrieved within AMS some other way, but haven't had much of a chance to try that.

Another thought would be to use:

Web.LoadURL("Web1", "javascript:document.execCommand(\"SaveAs\",\"false\", \"C:\\\\htmlsource.txt\")")

In conjunction with the KeyStrokes dll to automate the save, but that doesn't seem real clean either. I'll see what I can come up with, but that's the best I have so far.


")

[Worm]

This isn't pretty, but it seems to do the trick. There has to be a better way.

Cookies are limited to 4K, so thats out...

[Worm]

Here's a much cleaner solution... it uses the luaCom plugin from (http://www.icynorth.com)

[johnraus]

Here's a much cleaner solution... it uses the luaCom plugin from (http://www.icynorth.com)

Thnx Worm for your interest, and helping me ...

I'll Try all of your solutions right now, so I'll be busy for a while

[Worm]

Okay, sorry to keep going on this, but this is too cool to not share.

I extended the luaCom project a little more. The issue before was that your user couldn't really interact with the "automated" Internet Explorer Window. It hit me that I might be able to embed the Internet Explorer Window in AMS to emulate the web object.

Guess What!? I can, and did! With a couple handy API calls to remove the title bar and to set it's parent, I was able to take the Internet Explorer Window and make it look like the Web Object within the AMS project. Now, you can control the Internet Explorer Window within your AMS project via luaCom, plus the users can interact with it too.

Anyhow, this should give you a seamless solution. This goes to prove to me, yet again, how powerful AMS can be.

[Intrigued]

[Can't resist]

"You don't know the power of the Code Side! Come Worm, join me... we will rule the galaxy together... as goofy guy 1 and ... goofy guy2!"

[/Can't resist]

Hard.Message("Dee", "Har-Har!")

(btw, this is pretty nifTAY!)

[johnraus]

Okay, sorry to keep going on this, but this is too cool to not share.

I extended the luaCom project a little more. The issue before was that your user couldn't really interact with the "automated" Internet Explorer Window. It hit me that I might be able to embed the Internet Explorer Window in AMS to emulate the web object.

Guess What!? I can, and did! With a couple handy API calls to remove the title bar and to set it's parent, I was able to take the Internet Explorer Window and make it look like the Web Object within the AMS project. Now, you can control the Internet Explorer Window within your AMS project via luaCom, plus the users can interact with it too.

Anyhow, this should give you a seamless solution. This goes to prove to me, yet again, how powerful AMS can be.

Yep, this is to COOL to be true THNX very much, you even provided solution before I could ask the question, because your previous "luaComGetSource" started another inctance of IE and I would end at the logonpage for that website, this one should solve this issue.

I'm not very expirienced so where can I find more info on controlling this Internet Explorer Window with the LUAcom plugin for (Back-Fwd button, get displayed url (like e_URL on loaded) or even Post data to this IE window.

[johnraus]

WOW I just discovered info on controlling IE instance, in AMS project
This is more than I asked for ....

MSDN info in iternetexplorer object

This is really COOooooollll
THNX again Worm for your effort

on minor drawback, is when th IE object has focus, the onmouse-over effect in AMS is gone till I click AMS window, but I can live with this. The control of this IE object is a boost for my creativity for me for now ... Lots of hours of playtime to go to expand my project ....

[Worm]

one minor drawback, is when th IE object has focus, the onmouse-over effect in AMS is gone till I click AMS window, but I can live with this.

No need to live with it when it can be overcome...

Here's the fix, I used my mouse.dll to determine which window the pointer is over, then use the User32.dll function SetFocus to put the focus on the correct window.

When/If you do more automation with IE, I'd love to see what you come up with.

I agree with you on this one... it opens up a whole new avenue with web integration. I'm very interested in how well this will perform. Please keep me updated.

[johnraus]

Hey Worm, you did it again
I'll keep you posted...

[Worm]

Bug fix:

In On Preload the action:
oIE.Resizable = false;

needs to come after the DLL.CallFunction statements. Otherwise the IE window is resizable inside the AMS app. Although that's kinda cool, its not the what you're looking to do.