<p><span>Thanks to <a rel="nofollow" href="https://www.dallasopendata.com/City-Services/FY-2017-Dallas-Animal-Shelter-Data/sjyj-ydcj" target="_blank">Dallas OpenData</a> anyone has access to the city animal shelter records.  If you lost or found a pet it could be that <a rel="nofollow" href="http://www.writersdigest.com/editor-blogs/questions-and-quandaries/grammar/how-to-handle-animal-pronouns-he-she-or-it" target="_blank">he or she</a> spent some time in a shelter – I personally took lost dogs there twice</span><span>. It’s unfortunate but every year tens of thousands of animals find their way to shelters with significant fraction never finding way out. </span><br><span><br></span><span>City of Dallas animal shelter dataset (I will share in another post detailed account on how the data is consumed and processed in R) contains 5 types of animals but solid lead belongs to dogs:</span><br><span><br></span><br>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/3.bp.blogspot.com/-Rp4fcn8bQDg/WYK3gZhnEtI/AAAAAAAA6ew/zP944BntOQgzzs75l4y5kmUu9RFMNa0DwCLcBGAs/s1600/dallas-animal-shelters-animal-totals.png?ssl=1" target="_blank"><img border="0" data-original-height="512" src="https://i0.wp.com/3.bp.blogspot.com/-Rp4fcn8bQDg/WYK3gZhnEtI/AAAAAAAA6ew/zP944BntOQgzzs75l4y5kmUu9RFMNa0DwCLcBGAs/s640/dallas-animal-shelters-animal-totals.png?resize=450%2C512&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td>Admissions by Animal Types</td>
</tr>
</tbody>
</table>
<p><span><br></span><span>For consistency and plausibility of analysis we will focus only on the records with dogs</span><span>. </span><br><span><br></span><span>More exactly, each animal gets admitted to a shelter with certain intake type and gets discharged with certain outcome: </span><br><span><br></span><br>
<table cellspacing="0" align="center" cellpadding="0">
<tbody>
<tr>
<td><a rel="nofollow" href="https://i0.wp.com/3.bp.blogspot.com/-5ctV5cIfgjM/WYK5MOgze9I/AAAAAAAA6e8/IoF-bhgwF2Usl51UbEuoM86S33UIRrJKgCLcBGAs/s1600/dallas-animal-shelters-intake-totals.png?ssl=1" target="_blank"><img border="0" data-original-height="512" src="https://i0.wp.com/3.bp.blogspot.com/-5ctV5cIfgjM/WYK5MOgze9I/AAAAAAAA6e8/IoF-bhgwF2Usl51UbEuoM86S33UIRrJKgCLcBGAs/s640/dallas-animal-shelters-intake-totals.png?resize=450%2C512&ssl=1" data-recalc-dims="1" data-original-width="450"></a></td>
</tr>
<tr>
<td>Dogs Admitted by Intake Types</td>
</tr>
</tbody>
</table>
<p><span><br></span><span>Top 3 reasons why dogs turn up at shelters are </span><span><b>Confiscated</b> (abused, no owner, etc.), </span><span><b>Owner Surrender</b> (willingly brought in by owner), and </span><span><b>Stray</b> (lost or abandoned)</span><span>.</span><br><span><br></span>
<div><a rel="nofollow" href="https://i1.wp.com/1.bp.blogspot.com/-WDdMC3FkaEs/WYK6IG3ZyuI/AAAAAAAA6fA/ihiU6qsb0hMS3MASadUaUA-ZmVTZALR_wCLcBGAs/s1600/dallas-animal-shelters-outcome-totals.png?ssl=1" target="_blank"><img border="0" data-original-height="512" src="https://i0.wp.com/1.bp.blogspot.com/-WDdMC3FkaEs/WYK6IG3ZyuI/AAAAAAAA6fA/ihiU6qsb0hMS3MASadUaUA-ZmVTZALR_wCLcBGAs/s640/dallas-animal-shelters-outcome-totals.png?resize=450%2C512&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<p><span><br></span><span>Dogs leave shelters (either alive or dead) for 4 main reasons (outcomes): </span><span><b>Adoption</b> (good),</span><span> </span><span><b>Euthanized</b> (bad), <b>Returned to Owner</b> (good), and <b>Transfer</b> (neutral). </span><br><span><br></span><span>So what is the relationship between top intake types and outcomes? Which and to what extent intake types drive outcomes? The good news there is some causality effect because each stay begins with intake type and ends with outcome. But given that there are still a few different ways to investigate the relationship.</span><br><span><br></span><span><span>Let’s begin with the most general (in that case) but visually appealing visualization called sankey diagram (or just sankey)</span><span>. It is </span></span><span><span>a specific type of flow diagram,</span><span> in which the width of the arrows is shown proportionally to the flow quantity:</span></span><br><span><br></span><iframe allowfullscreen="" src="https://rpubs.com/grigory/IntakeToOutcomeSankey" name="myiFrame" width="800px" height="600px"></iframe><span><br></span><span><br></span><br><span>Each dog shelter stay contributed to the size of one of the pipes flowing from an intake type to an outcome. At least now we know conditional probabilities of a dog leaving shelter with certain outcome given it was admitted with intake type.</span><br><span><br></span><span>Next, we go beyond total aggregates (counts) used in the sankey above to correlate intake types and outcomes over time (monthly trends). W</span><span>e computed correlations between monthly totals of dogs brought in and removed from Dallas animal shelters for each pair of top animal intake types (<b>Confiscated</b>, <b>Owner Surrender</b>, and <b>Stray</b>) and outcomes (<b>Adoption</b>, <b>Euthanized</b>, <b>Returned to Owner</b>, and <b>Transfer</b>): 12 coefficients in total:</span><br><span><br></span>
<div><a rel="nofollow" href="https://i0.wp.com/4.bp.blogspot.com/-3UPVMKvVWF4/WYQKPTSRsYI/AAAAAAAA6gM/lYCJuomL5F8RN2tPQfVT0L7Onyvg0v0QACLcBGAs/s1600/dallas-animal-shelters-multiple-dog-monthly-trends-corrs.png?ssl=1" target="_blank"><img border="0" data-original-height="683" src="https://i0.wp.com/4.bp.blogspot.com/-3UPVMKvVWF4/WYQKPTSRsYI/AAAAAAAA6gM/lYCJuomL5F8RN2tPQfVT0L7Onyvg0v0QACLcBGAs/s640/dallas-animal-shelters-multiple-dog-monthly-trends-corrs.png?resize=450%2C683&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div></div>
<p><span><br></span><span>In this case strong correlation implies (at least to some extent) causation effect due to presence of temporal relationship, consistency, and plausibility criteria (see <a rel="nofollow" href="https://stats.stackexchange.com/a/2536/8464" target="_blank">here</a> and <a rel="nofollow" href="http://epiville.ccnmtl.columbia.edu/assets/pdfs/Hill_1965.pdf" target="_blank">here</a>). </span><span>The most apparent observation is that variations (changes) in dogs <b>Surrendered by Owner</b> are highly correlated with swings in number of dogs being <b>Euthanized</b> which is almost as obvious as unfortunate. Other correlations to note:</span></p>
<ul>
<li><span>No outcomes are affected by variations in <b>Confiscated</b> dogs, but this is likely due to small share of admissions of this type.</span></li>
<li><span>Variation in <b>Stray</b> dogs admitted affect every outcome (more or less) except <b>Euthanized</b> which is sort of surprising (Stray intake type is the largest and is almost twice as big as the 2d largest type Owner Surrender).</span></li>
</ul>
<div><span>But can we do better than correlations for the same data? What if in</span><span>stead of coefficients (which are technically sophisticated aggregates) we observe actual actual monthly trends for each pair? Visual below corresponds to the correlation matrix above by showing actual trends instead of computed correlation coefficients:</span></div>
<div><span><br></span></div>
<div><a rel="nofollow" href="https://i0.wp.com/1.bp.blogspot.com/-Wo7hfF3ILy8/WYQNiCpPGgI/AAAAAAAA6gg/MuIA8A8H6REhZW-Jb0LOtjVHFOGbwDGwwCLcBGAs/s1600/dallas-animal-shelters-multiple-dog-monthly-trends.png?ssl=1" target="_blank"><img border="0" data-original-height="911" src="https://i1.wp.com/1.bp.blogspot.com/-Wo7hfF3ILy8/WYQNiCpPGgI/AAAAAAAA6gg/MuIA8A8H6REhZW-Jb0LOtjVHFOGbwDGwwCLcBGAs/s640/dallas-animal-shelters-multiple-dog-monthly-trends.png?resize=450%2C911&ssl=1" data-recalc-dims="1" data-original-width="450"></a></div>
<div><span><br></span></div>
<div><span>Each row contains the same intake type and each column contains the same outcome type (just like correlation matrix before). Having observed correlations before we can now see both trends over time (months) and volumes (going top down):</span></div>
<ul>
<li><span><b>Confiscated</b> intake trends flat with only significant spike in January 2016. This spike is so unusual, big, and contained that it should encourage an investigation into probable external event or procedural change that may have triggered it.</span></li>
<li><span>Number of <b>Confiscated</b> dogs is relatively low to noticeably affect outcomes. Still, if we can reduce effect of other intake types some relationships are probable.</span></li>
<li><span><b>Owner Surrender</b> trend correlation with <b>Euthanized</b> outcome is as obvious as unfortunate – people bring their old or unhealthy pets for a reason. </span></li>
<li><span><b>Owner Surrender</b> has significant seasonal component spiking in summer possibly due to hot weather or holiday season or both.</span></li>
<li><span><b>Euthanized</b> trends together with <b>Owner Surrender</b> which causes it to large degree.</span></li>
<li><span><b>Stray</b> dogs trend slowly upwards in Dallas and it’s alarming.</span></li>
<li><span><b>Adoption</b> also trends upwards but not steep enough to compensate inflow of dogs into shelters. The targeted campaign to encourage more adoptions of pets in the city is due.</span></li>
<li><span><b>Stray</b> trend dipped in January 2016 exactly when confiscated trend spiked – it could be a coincidence or related – for sure something to consider when investigating.</span></li>
<li><span><b>Euthanized</b> trend correlates strongly with <b>Stray</b> intake until the summer of 2016 when it started to diverge in opposite direction – again some policy or procedural change apparently caused it. Indeed, if we observe other outcomes we notice that <b>Returned to Owner</b> trend began its uptick at around the same time (indeed, it turned out to be the case; for example, see <a rel="nofollow" href="http://www.latimes.com/nation/la-na-stray-dogs-20160915-snap-story.html" target="_blank">here</a> for significant changes in Dallas Animal Services leadership and policies around summer and fall of 2016 after and what caused it).</span></li>
</ul>
<div><span>I will be back with more analysis (survival analysis) and R code for data processing, analysis, and visualizations.</span></div>
	<img width="1" alt="" src="http://feeds.feedburner.com/~r/RBloggers/~4/BGyxjmxG34c" height="1">