Can my local programs be recognized as though they are ME?

Problems with the Windows version of XAMPP, questions, comments, and anything related.

Can my local programs be recognized as though they are ME?

Postby tedrika » 12. December 2017 00:16

Very new to this. Here is my issue:

I have to log into a site X.com in order to use certain search features. I'm a paying member and am registered.

I want to automatically search once an hour for certain things.

It occurred to me that it would be easier to set up a cron to do the searches and send the results to my email or text.

So, I thought I could install XAMPP locally and that would work since it is 'local', which I thought equated to as being my computer or -- 'me'.

So I created a simple scrape program that goes to X.com?search=a+b|c+d However X.com doesn't recognize the program as being ME and indicates the scrape program is not authorized since it isn't logged in. Is that because the XAMPP has a different IP address or something else?
Whatever the reason, is there a way I can set things up to accomplish my objective?
tedrika
 
Posts: 5
Joined: 12. December 2017 00:03
XAMPP version: 7.1.11
Operating System: Windows 10

Re: Can my local programs be recognized as though they are M

Postby Nobbie » 12. December 2017 14:49

tedrika wrote:Is that because the XAMPP has a different IP address or something else?


It is because HTTP is a transcational protocoll which does not know status. There is no "logged" in status or similar, that used to be realized as a fake with the help of cookies (small storage in the clients browser). If you want to create an automatically working script, which is running under Xampp, you have to emulate the behaviour of a browser, which includes the full HTTP header (as part of HTTP), which includes the cookies. You have to store and to apply the cookie values correctly. And first of all, you need to emulate the login process in order to receive the desired value of the cookie (and the name of the cookie, which is used for login by that page). The remote server does not "recognize" your PC or similar, it only recognizes your browser by the value of the transmitted cookies.

Its quite a task to do so, but its not impossible.
Nobbie
 
Posts: 13170
Joined: 09. March 2008 13:04

Re: Can my local programs be recognized as though they are M

Postby tedrika » 12. December 2017 21:14

Nobbie, thanks for your reply. It looks like there are about 20 cookies for me. I"m not real familiar with this area, but I have used programs to create cookies..You said it is quite a task to do overall, so I guess this is not as simple as somehow transmitting the same 20 cookies within the simple scrape script?

Nobbie wrote:
tedrika wrote:Is that because the XAMPP has a different IP address or something else?


It is because HTTP is a transcational protocoll which does not know status. There is no "logged" in status or similar, that used to be realized as a fake with the help of cookies (small storage in the clients browser). If you want to create an automatically working script, which is running under Xampp, you have to emulate the behaviour of a browser, which includes the full HTTP header (as part of HTTP), which includes the cookies. You have to store and to apply the cookie values correctly. And first of all, you need to emulate the login process in order to receive the desired value of the cookie (and the name of the cookie, which is used for login by that page). The remote server does not "recognize" your PC or similar, it only recognizes your browser by the value of the transmitted cookies.

Its quite a task to do so, but its not impossible.
tedrika
 
Posts: 5
Joined: 12. December 2017 00:03
XAMPP version: 7.1.11
Operating System: Windows 10

Re: Can my local programs be recognized as though they are M

Postby Nobbie » 12. December 2017 21:48

tedrika wrote:I guess this is not as simple as somehow transmitting the same 20 cookies within the simple scrape script?


it depends. You have to rebuild a standard HTTP Header, which also contains all cookies. Each cookie is simply represented like "COOKIENAME=COOKIEVALUE" within the header. Either build the response from scratch, or use the cURL library from PHP. Anyway, you have to do something.
Nobbie
 
Posts: 13170
Joined: 09. March 2008 13:04

Re: Can my local programs be recognized as though they are M

Postby tedrika » 12. December 2017 22:11

ok, I've used cURL for a few things before. I'm not sure what 'rebuild a standard HTTP' means but I'll play around a bit with it. The scrape program simply reads the url with file_get_contents($url) maybe I need to use curl instead to get the contents and send a header along with it? If that works it seems I could have done that from my shared host and not needed xampp for local.

Nobbie wrote:
tedrika wrote:I guess this is not as simple as somehow transmitting the same 20 cookies within the simple scrape script?


it depends. You have to rebuild a standard HTTP Header, which also contains all cookies. Each cookie is simply represented like "COOKIENAME=COOKIEVALUE" within the header. Either build the response from scratch, or use the cURL library from PHP. Anyway, you have to do something.
tedrika
 
Posts: 5
Joined: 12. December 2017 00:03
XAMPP version: 7.1.11
Operating System: Windows 10

Re: Can my local programs be recognized as though they are M

Postby tedrika » 13. December 2017 01:23

Ok, I got something to work, and it seems to be working from my shared host..not too bad. Thanks again for your help!
tedrika
 
Posts: 5
Joined: 12. December 2017 00:03
XAMPP version: 7.1.11
Operating System: Windows 10

Re: Can my local programs be recognized as though they are M

Postby Nobbie » 13. December 2017 11:22

tedrika wrote:I'm not sure what 'rebuild a standard HTTP' means but I'll play around a bit with it.


The cURL Library already does this job for you, the HTTP header is the conversation between Browser and Apache about the file, which is requested and it contains lots of informations (like CONTENT-TYPE, FILESIZE etc.) including the cookies. As you use cURL, you cannot "see" the HTTP header, these are simply a few lines of text before the HTML output begins (the Browser displays only the HTML and used the HTTP Header for the conversation).

If you succeeded with cURL, there is a good chance to get an even simpler solution. If you have a linux server (or any PC running linux), there is a command line tool "wget". This is a very powerfull tool to download remote sites, it also can pass data like passwords and userids for logins. That wget has been ported to Windows (as far as remember), you may Google for "wget Windows" if you dont run linux.

If you are lucky, you can simply use this little program for your approach, i think its worth a try. The success depends on the way how the authorization of the remote site is realized, if it is a standard Basic Authorization via HTTP, you are actually done. Simply start wget with the proper parameters (the URL, the login credentials). Finally there is no need for Xampp, even not for any Apache server.
Nobbie
 
Posts: 13170
Joined: 09. March 2008 13:04

Re: Can my local programs be recognized as though they are M

Postby tedrika » 13. December 2017 22:46

thanks for that additional info Nobbie. I'm familiar with wget but haven't used it much. It looks like the CURL is going to do the job for me, and it includes a program to analyze the results as needed. Thanks again.
tedrika
 
Posts: 5
Joined: 12. December 2017 00:03
XAMPP version: 7.1.11
Operating System: Windows 10


Return to XAMPP for Windows

Who is online

Users browsing this forum: No registered users and 101 guests