In this tutorial, we will cover how to extract information from a matrimonial website using R.  We will do web scraping which is a process of converting data available in unstructured format on the website to structured format which can be further used for analysis.
We will use a R package called rvest which was created by Hadley Wickham. This package simplifies the process of scraping web pages.
Web Scraping in R
Install the required packages
To download and install the rvest package, run the following command. We will also use dplyr which is useful for data manipulation tasks.
install.packages(&quot;rvest&quot;)install.packages(&quot;dplyr&quot;)
Load the required Libraries
To make the libraries in use, you need to submit the program below.
library(rvest)library(dplyr)
Scrape Information from Matrimonial WebsiteFirst we need to understand the structure of URL. See the URLs below.https://www.jeevansathi.com/punjabi-brides-girlshttps://www.jeevansathi.com/punjabi-grooms-boys
The first URL takes you to the webpage wherein girls' profiles of Punjabi community are shown whereas second URL provides details about boys' profiles' of Punjabi community.
We need to split the main URL into different elements so that we can access it. 
Main_URL = Static_URL + Mother_Tongue + Brides_GroomsCheck out the following R code how to prepare the main URL. In the code, you need to provide the following details -
Whether you are looking for girls'/boys' profiles. Type bride to see girls' profiles. Enter groom to check out boys' profiles.
Select Mother Tongue. For example, punjabi, tamil etc.
# Looking for bride/groomBride_Groom = &quot;bride&quot;# Possible Values : bride, groom# Select Mother TongueMother_Tongue = &quot;punjabi&quot;# Possible Values# punjabi# tamil# bengali# telugu# kannada# marathi# URLif (tolower(Bride_Groom) == &quot;bride&quot;) {html = paste0('https://www.jeevansathi.com/',tolower(Mother_Tongue),'-brides-girls')} else {html = paste0('https://www.jeevansathi.com/',tolower(Mother_Tongue),'-grooms-boys')}
See the output : [1] &quot;https://www.jeevansathi.com/punjabi-brides-girls&quot;
Extract Profile IDsFirst you need to select parts of an html document using css selectors: html_nodes(). Use SelectorGadget which is a chrome extension available for free. It is the easiest and quickest way to find out which selector pulls the data that you are interested in.How to use SelectorGadget : Click on a page element that you would like your selector to match (it will turn green). It will then generate a minimal CSS selector for that element, and will highlight (yellow) everything that is matched by the selector.text = read_html(html) %__% html_nodes(&quot;.profileContent .color11 a&quot;) %__% html_text()profileIDs = data.frame(ID = text)
ID1 ZARX03452 ZZWX55733 ZWVT21734 ZAYZ61005 ZYTS68856 ZXYV98497 TRZ84758 VSA72849 ZXTU196510 ZZSA687711 ZZSZ654512 ZYSW480913 ZARW219914 ZRSY072315 ZXAT280116 ZYXX881817 ZAWA856718 WXZ214719 ZVRT887520 ZWWR953321 ZYXW4043The basic functions in rvest are very user-friendly and robust. Explanation of these functions are listed below -
read_html() :  you can create a html document from a URL
html_nodes() : extracts pieces out of HTML documents.
html_nodes(&quot;.class&quot;) : calls node based on CSS class
html_nodes(&quot;#class&quot;) : calls node based on , , id
html_text() : extracts only the text from HTML tag
html_attr() : extracts contents of a single attribute
Difference between .class and #class1. .class targets the following element:
2. #class targets the following element:
Get Detailed Information of Profiles
The following program performs the following tasks -
Loop through profile IDs
Pull information about Age, Height, Qualification etc.
Extract details about appearance
Fetch 'About Me' section of profiles
# Get Detailed Informationfinaldf = data.frame()for (i in 1:length(profileIDs$ID)){ID = profileIDs[i,1]link = paste0(&quot;https://www.jeevansathi.com/profile/viewprofile.php?stype=4&amp;username=&quot;, ID)FormattedInfo = data.frame(t(read_html(link) %__% html_nodes(&quot;.textTru li&quot;) %__% html_text()))# Final TableFormattedInfo = data.frame(ProfileID = ID, Description = read_html(link) %__% html_nodes(&quot;#myinfoView&quot;) %__% html_text(), Appearance = read_html(link) %__% html_nodes(&quot;#section-lifestyle #appearanceView&quot;) %__% html_text(), FormattedInfo)finaldf = bind_rows(finaldf, FormattedInfo)}# Assign Variable Namesnames(finaldf) = c(&quot;ProfileID&quot;, &quot;Description&quot;, &quot;Appearance&quot;, &quot;Age_Height&quot;, &quot;Qualification&quot;, &quot;Location&quot;, &quot;Profession&quot;, &quot;Mother Tongue&quot;, &quot;Salary&quot;, &quot;Religion&quot;, &quot;Status&quot;, &quot;Has_Children&quot;)
Download Display PicTo download display pic, you first need to fetch image URL of profile and then hit download.file( ) function to download it. In the script below, you need to provide a profile ID.# Download Profile Pic of a particular ProfileID = &quot;SWU6683&quot;text3 = read_html(html) %__% html_nodes(&quot;.vtop&quot;) %__% html_attr('src')pic = data.frame(cbind(profileIDs, URL = text3[!is.na(text3)]))download.file(as.character(pic$URL[match(ID, pic$ID)]), &quot;match.jpg&quot;, mode = &quot;wb&quot;)# File saved as match.jpg
DisclaimerWe have accessed only publicly available data which does not require login or registration. The purpose is  not to cause any damage or copy the content from the website.
Other Functions of rvest( )You can extract, modify and submit forms with html_form(), set_values() and submit_form(). Refer the case study below -You can collect google search result by submitting the google search form with search term. You need to supply search term. Here, I entered 'Datascience' search term.library(rvest)url       = &quot;http://www.google.com&quot;pgsession = html_session(url)            pgform    = html_form(pgsession)[[1]]# Set search termfilled_form = set_values(pgform, q=&quot;Datascience&quot;)session = submit_form(pgsession,filled_form)# look for headings of first pagesession %__% html_nodes(&quot;.g .r a&quot;) %__% html_text()
[1] &quot;Data science - Wikipedia&quot; [2] &quot;Data Science Courses | Coursera&quot; [3] &quot;Data Science | edX&quot; [4] &quot;Data science - Wikipedia&quot; [5] &quot;DataScience.com | Enterprise Data Science Platform Provider&quot; [6] &quot;Top Data Science Courses Online - Updated February 2018 - Udemy&quot; [7] &quot;Data Science vs. Big Data vs. Data Analytics - Simplilearn&quot; [8] &quot;What Is Data Science? What is a Data Scientist? What is Analytics?&quot; [9] &quot;Online Data Science Courses | Microsoft Professional Program&quot; [10] &quot;News for Datascience&quot; [11] &quot;Data Science Course - Cognitive Class&quot;