I’m trying to log into a website which authenticates via email and password. Here’s the start of the web app and how I successfully read the headers:
Dim socket1 As New HTTPSocket
Dim result As String
' Request login screen
' http://v2.gedmatch.com/login1.php
Dim ih as InternetHeaders = (socket1.GetHeaders("http://v2.gedmatch.com/login1.php", 0))
I’m now having difficulty into how I have to give the cookie which is in one of the headers back to the website when I do the POST command to login. I can get the count for my “ih” with “ih.Count” but as it’s not a dictionary I can’t read out the single key/value pairs. Can someone tell me how you have done this part?
It’s my understanding (and please correct me if I’m wrong here) that once I have the cookie I will use SetRequestHeader (http://documentation.xojo.com/index.php/HTTPSocket.SetRequestHeader) to give the cookie back with the POST command. In the help it’s been written to do it with this line of code:
I’m having difficulty in between these two pieces of code, how can I get from having the header information in “ih” (which has a dictionary called …Headers in there apparently with 9 entries) and transform it in the right way to use the Join(Array(cookie1, cookie2) which is a construct I don’t understand either, maybe someone can point out to me what lines I’m missing in between.
My code continues with this:
Dim form As New Dictionary
form.Value("email") = "my_email"
form.Value("password") = "my_password"
socket1.SetFormData(form)
result = socket1.Post("http://v2.gedmatch.com/login2.php", 30) // Synchronous
When I read the “result” then it says “Error 53: Not logged in” which I think is due to the fact that I’m not giving the cookie back with my POST command. Is this how it’s done right with a website that needs authorization? Do I have to give any other headers back to website with the POST command?
I’ve searched the forum and while there are bits and pieces everywhere, there is no forum entry explaining it to a newbie like me in it’s entirety.
There is also the possibility that they only allow the login post to come from their website, in which case no amount of hacking around will allow this. Httpsockets are not the same as a browser.
Well, that’s a possibility, but I was thinking that forms are sometimes submitted by JavaScript and fields are added dynamically depending on what the user inputs. Trying to bypass someone’s login screens is generally frowned upon anyway.
You must have misunderstood me, @Greg O’Lone . I don’t want to bypass the login screen, I’m a paying user on this website but I want to automate a couple of tasks and better keep track of the response over time.
I’d consult the host first. For instance, I could write a scrapper that checks to see if I have notifications here since I don’t get email notices and the popup notices here fail usually, but imagine all those requests… now imagine making the tool publicly available…yikes! Now periodic checks may be permitted. The Xojo forum server would be bogged right down… not good. Some hosts/sites autoban users if X-amount of requests are made in a specified amount of time. Or, implement hidden fields to determine if the user is a bot since Javascript won’t fire within an httpsocket. It’s a bruteforce security measure. No problem with Russia, but daily many of my sites are bombarded by such with Russian IPs. We setup an autoban with auto - reporting of the IP to the originating ISP. The bombardments have grown fewer and fewer in number Whenever accessing materials through unprovided methods, it’s always polite to ask first. Even in that case, they may give your account special privileges (ie turn off bot detection).
Thanks @Ashot Khachatryan , that was exactly the missing piece in between. In the meantime I also found this excellent explanation what is exchanged between website and browser: https://en.wikipedia.org/wiki/HTTP_cookie#Cookie_attributes
I’ve added one line to Ashot’s example:
socket1.SetRequestHeader("Set-Cookie", strCookie)
@Matthew Combatti , thanks for your advice and comment. My actions are those that a user is usually doing. Actually the web app will bring requests down as currently I’ll accidentally do actions again (that I did some weeks or days ago) as there is no way to track which result I already worked on (and they show 1500 matches), so impossible to keep track without a database keeping track.
Best solution would be an API, I hope they will eventually publish one.
Whenever accessing materials through unprovided methods, it’s always polite to ask first. Even in that case, they may give your account special privileges (ie turn off bot detection).