<p><div><div><div><div data-width="100" data-show-faces="false" data-send="true" data-layout="button_count" data-height="21" data-href="https://www.r-bloggers.com/r-in-open-data-complaints-in-the-field-of-freedom-of-information-data-set-from-data-gov-rs/"></div></div><div></div></div><div></div></div>
<div>
(This article was first published on <strong><a href="http://www.exactness.net/post/157147338755"> The Exactness of Mind</a></strong>, and kindly contributed to <a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers)</a> </div>
<p>The <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic/OpenDataR/tree/master/ComInfoPublicImportance">notebooks</a> (R, Rmd, and HTML files are provided in <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic">my GitHub repository</a>) focus on an exploratory analysis of the open data set on the complaints in the field of freedom of information, provided at the <a target="_blank" rel="nofollow" href="http://data.gov.rs/sr/">Open Data Portal of the Republic of Serbia</a> that is currently under development. The data set was kindly provided to the Open Data Portal by the <a target="_blank" rel="nofollow" href="http://www.poverenik.rs/en.html">Commissioner for Information of Public Importance and Personal Data Protection</a> of the Republic of Serbia. Many more open data sets will be indexed and uploaded to the <a target="_blank" rel="nofollow" href="http://data.gov.rs/sr/">Open Data Portal of the Republic of Serbia</a> in the forthcoming weeks and months.</p>
<p>You should view this as an exercise in data wrangling and visualization with {ggplot2} and {igraph} primarily. As of the data set: (a) no metadata and no documentation were provided; (b) the translation of legal terms from Serbian to English is mine, meaning: a lot of Google Translate suggestions were used (I’m a psychologists, not a lawyer or a legal expert); © mixture of latin and cyrilic alphabet was detected in the data; (d) thorough cleaning takes place here in <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic/OpenDataR/blob/master/ComInfoPublicImportance/ComInfoPublicImportance_PartA.Rmd">Part A</a>; exploratory analysis + data visualizations are be presented <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic/OpenDataR/blob/master/ComInfoPublicImportance/ComInfoPublicImportance_PartB.Rmd">Part B</a>. R programming language provides a fantastic infrastructure for data wrangling (cleaning, preparation, re-structuring; in a nutshell, all necessary data management process than needs to be taken care of before any attempts at EDA or statistical modeling). In <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic/OpenDataR/blob/master/ComInfoPublicImportance/ComInfoPublicImportance_PartA.Rmd">Part A</a> I have used {dplyr} in combination with {tidyr} and {base} functions to inspect and clean up the data set (as much as I could); in <a target="_blank" rel="nofollow" href="https://github.com/GoranMilovanovic/OpenDataR/blob/master/ComInfoPublicImportance/ComInfoPublicImportance_PartB.Rmd">Part B</a>, {ggplot2} and {igraph} functionality was added to visualize some of the interesting patterns from the data set.</p>
<p>This is also a good reality check for all those who are contemplating a Data Science career. Similarly to what I had to do here, you will be often faced with data sets with no documentation and no metadata, and than you will need to do explorations before doing real EDA in order to try to figure out the semantics of the data; many times, you will be forced to combine structured and abstract approaches to clean data with manual procedures; you will be driven mad by inconsistencies and unavailability, but it will still be up to you to do what you can in order to try to squeeze out something useful from the data that you have at your disposal. It’s no joke: data wrangling and related procedures will be stealing a huge amount of time from you. Statistical modeling comes almost as a reward after what you’ve been through since you’ve been introduced to the data set…</p>
<p>Here are some examples with {ggplot2} and {igraph} from this case study.</p>
<figure data-orig-height="960" data-orig-width="1344"><img alt="image" src="https://i0.wp.com/68.media.tumblr.com/d89aa171c5e845f222c4b6152446bd23/tumblr_inline_ol9ox5Hhxz1qa0hyw_540.png?resize=456%2C326" data-orig-height="960" data-recalc-dims="1" data-orig-width="1344"></figure>
<p><b><i>Figure 1. </i>Number of complaints filed per applicant group 2005 – 2016. {ggplot2} w. facet_wrap().</b></p>
<figure data-orig-height="960" data-orig-width="1344"><img alt="image" src="https://i1.wp.com/68.media.tumblr.com/c9476a5bb300cc8a45600fe0d415e847/tumblr_inline_ol9oxgLXiC1qa0hyw_540.png?resize=456%2C326" data-orig-height="960" data-recalc-dims="1" data-orig-width="1344"></figure>
<p><b><i>Figure 2. </i>Each applicant group (blue circles) in this directed graph points towards the domains (gold circles) in respect to which it has sent its complaints to the Commissioner for Information of Public Importance and Personal Data Protection. {igraph}.</b></p>
<figure data-orig-height="960" data-orig-width="1344"><img alt="image" src="https://i1.wp.com/68.media.tumblr.com/ec07847bea6fd5ade62ddb2821d88137/tumblr_inline_ol9oxsVxCh1qa0hyw_540.png?resize=456%2C326" data-orig-height="960" data-recalc-dims="1" data-orig-width="1344"></figure>
<p><b><i>Figure 3. </i>Each applicant group in this directed graph points towards the top three authority groups in respect to which it has sent the maximum numbers of complaints to the Commissioner for Information of Public Importance and Personal Data Protection (applicant groups represented by blue and authority groups by red circles): {igraph}.</b></p>
<p>
<div>
<div>To <strong>leave a comment</strong> for the author, please follow the link and comment on their blog: <strong><a href="http://www.exactness.net/post/157147338755"> The Exactness of Mind</a></strong>.</div>
<hr>
<a rel="nofollow" href="https://www.r-bloggers.com/">R-bloggers.com</a> offers <strong><a rel="nofollow" href="https://feedburner.google.com/fb/a/mailverify?uri=RBloggers">daily e-mail updates</a></strong> about <a title="The R Project for Statistical Computing" rel="nofollow" href="https://www.r-project.org/">R</a> news and <a title="R tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/tutorial">tutorials</a> on topics such as: <a title="Data science" rel="nofollow" href="https://www.r-bloggers.com/search/data%20science">Data science</a>, <a title="Big Data" rel="nofollow" href="https://www.r-bloggers.com/search/Big%20Data">Big Data, <a title="R jobs" rel="nofollow" href="https://www.r-users.com/">R jobs</a>, visualization (<a title="ggplot and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/ggplot2">ggplot2</a>, <a title="Boxplots using lattice and ggplot2 tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/boxplot">Boxplots</a>, <a title="Maps and gis" rel="nofollow" href="https://www.r-bloggers.com/search/map">maps</a>, <a title="Animation in R" rel="nofollow" href="https://www.r-bloggers.com/search/animation">animation</a>), programming (<a title="RStudio IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/RStudio">RStudio</a>, <a title="Sweave and literate programming" rel="nofollow" href="https://www.r-bloggers.com/search/sweave">Sweave</a>, <a title="LaTeX in R" rel="nofollow" href="https://www.r-bloggers.com/search/LaTeX">LaTeX</a>, <a title="SQL and databases" rel="nofollow" href="https://www.r-bloggers.com/search/SQL">SQL</a>, <a title="Eclipse IDE for R" rel="nofollow" href="https://www.r-bloggers.com/search/eclipse">Eclipse</a>, <a title="git and github, Version Control System" rel="nofollow" href="https://www.r-bloggers.com/search/git">git</a>, <a title="Large data in R using Hadoop" rel="nofollow" href="https://www.r-bloggers.com/search/hadoop">hadoop</a>, <a title="Web Scraping of google, facebook, yahoo, twitter and more using R" rel="nofollow" href="https://www.r-bloggers.com/search/Web+Scraping">Web Scraping</a>) statistics (<a title="Regressions and ANOVA analysis tutorials" rel="nofollow" href="https://www.r-bloggers.com/search/regression">regression</a>, <a title="principal component analysis tutorial" rel="nofollow" href="https://www.r-bloggers.com/search/PCA">PCA</a>, <a title="Time series" rel="nofollow" href="https://www.r-bloggers.com/search/time+series">time series</a>, <a title="finance trading" rel="nofollow" href="https://www.r-bloggers.com/search/trading">trading</a>) and more...
<img height="1" alt="" width="1" src="http://feeds.feedburner.com/~r/RBloggers/~4/_NzkOiMkimg"></a></div>