Web-Scraping - Data Science

In this blog we will be learning how to extract data from any website and and analyze it for better purpose. Now, There are many online shopping companies which provides such facility to purchase such items. But the time taking task is to find perfect Mobile-phone from various available option. You can select your choice easily for that here “Web-scrapping” comes in to the picture. as word suggests i.e. collecting data from web.

On some of the website it is illegal to use web-scrapping. Now lets dive in the technical part. For the scrapping we will be using Python now lets install required libraries, run all the commands in either cmd or you can use Pycharm IDE ,

1. Selenium: It is extensively used for browser automation . Installation : pip install selenium. Now check the version of chrome and open this link and download chrome driver of version same as of your chrome.

2. Beautiful Soup : It is a Python library for pulling data out of HTML and XML files. Installation : pip install beautifulsoup4

3. Pandas : Pandas is a software library written for the Python programming language for data manipulation and analysis. Installation : pip install pandas

Steps To get the final csv file with required data.

1. Open the web from which you want to extract the data. for demo i am using flipcart for collecting info on gamming pcs. Note down the URL of page. eg.“https://www.amazon.in/s?i=electronics&bbn=1389432031&rh=n%3A1389432031%2Cp_85%3A10440599031%2Cp_36%3A400000-1000000&pd_rd_r=67b37192-12f5-443d-a571-2c12224ee356&pd_rd_w=CMTUy&pd_rd_wg=0ziES&pf_rd_p=04635875-775c-4bf0-b81d-c9df5ebfba3d&pf_rd_r=M4F0Y5X8TZENK95EM25N&qid=1626802564&rnid=1318502031&ref=pd_gw_unk”

2. Right click in web page and select inspect for inspect element.

3. For the product and note down the class name which we will use to get the data from web page using beautiful soup in python.

4. Write down the python code for web scrapping and run it and store the data in CSV file. Which we can use for various purpose.

Output :

Stored Data in CSV file