[FIXED] Login and scrape a remote website using Node.JS

Issue

I have a login page https://www.mywebsite.com/login, and a landing page to scrape https://www.mywebsite.com/home.

I want to create a script, preferred written in Node.JS back end server for running in the background without user interaction, that will login through login page, and navigate to the landing page after login, redirect with cookies validated through login.

So basically, it is a script that follows redirection from login page, through landing page without losing the cookies just like a modern browser does.

I have ideas in mind what I'm looking for but I don't know what's the best thing or best approach to use in order to achieve my goals. Any help would be appreciated.

I prefer Node.js as recommendation but open for many other possibilities.

Solution

I would suggest looking into Phantom.js, which is a headless browser that might support the functionality that you are looking for. Check out their cookie documentation as well. I hope this is helpful!

Answered By - BarzTM