Skip navigation

Category Archives: OS X

RStudio is continuing to refine their 👍🏽 new product version string scheme, with the most recent change using something like the following for daily builds:

RStudio-pro-2021.11.0-daily+23.pro1.dmg

The full URL for these Amazon S3-hosted builds is something like:

https://s3.amazonaws.com/rstudio-ide-build/desktop/macos/RStudio-pro-2021.11.0-daily%2B23.pro1.dmg

with the + getting URL encoded to %2B.

Unfortunately, AWS S3’s fairly unintelligent web server really wanted that + encoded to %2B for it to properly map to the retrieved-by-explicit-path-key bucket-backed object store. Since it was not, the download component of RSwitch v2.x failed with a somewhat inaccurate error message.

It was a minor fix that I finally had some spare cycles to make today.

You can download the RSwitch 2.1.2b universal binary here.

I had tried to convert my data-saving workflows to [`feather`](https://github.com/wesm/feather/tree/master/R) but there have been [issues](https://github.com/wesm/feather/issues/155) with it supporting large files (that seem to be near resolution), so I’ve been continuing to use R Data files for local saving of processed/cleaned data.

I make _many_ of these files and sometimes I do it as a one-off effort, thinking that I’ll come back to it quickly. Inevitably, I don’t do that and also end up naming those one-offs badly. I made a small [R helper package](https://github.com/hrbrmstr/rdatainfo) to make it easier to wrap up checking out these files at the command-line (via a `bash` function) but it hit me that it’d be even easier if there was a way to use the macOS Quick Look feature (hitting `` on a file icon) to see the previews.

Thus, [`QuickLookR`](https://github.com/hrbrmstr/QuickLookR) was born.

You need to [download the ZIP file](https://github.com/hrbrmstr/QuickLookR/releases/tag/v0.1.0), unzip it and save the `QuickLookR.qlgenerator` component into `~/Library/QuickLook`. Then `devtools::install_github(‘hrbrmstr/rdatainfo’)` in an R session. If you’ve got R/Rscript in the standard `/usr/local/bin` location, then you should be able to hit `` on any `.rdata`, `.rda` or `.rds` file and see a `str()` preview like this:

Blank_Skitch_Document

I haven’t cracked open Xcode in a while and my Objective-C is super-rusty, but this works on my El Capitan MacBook Pro (though I’m trying to see why some `.rds` files embedded in packages on my system have no previews).

If you have suggestions or issues, please use [github](https://github.com/hrbrmstr/QuickLookR/issues) to file them. For issues, it’d be really helpful if you included a copy of or link to files that don’t work well.

For the next revision, I plan on generating prettier HTML-based previews and linking against `R.framework` to avoid a call out to the system.

If Wes/Hadley really have fixed `feather`, I’ll be making a QuickLook plugin for that file format as well in the very near future.

It’s usually a good thing when my and infosec worlds collide. Unfortunately, this time it’s a script that R folk running on OS X can use to see if they are using a version of XQuartz that has a nasty vulnerability in the framework it uses to auto-update. If this test comes back with the warning, try to refrain from using XQuartz on insecure networks until the developers fix the issue.

**UPDATE**

Thanks to a gist prodding by @bearloga, here’s a script to scan all your applications for the vulnerability:

library(purrr)
library(dplyr)
library(XML)
 
read_plist <- safely(readKeyValueDB)
safe_compare <- safely(compareVersion)
 
apps <- list.dirs(c("/Applications", "/Applications/Utilities"), recursive=FALSE)
 
# if you have something further than this far down that's bad you're on your own
 
for (i in 1:4) {
  moar_dirs <- grep("app$", apps, value=TRUE, invert=TRUE)
  if (length(moar_dirs) > 0) { apps <- c(apps, list.dirs(moar_dirs, recursive=FALSE)) }
}
apps <- unique(grep("app$", apps, value=TRUE))
 
pb <- txtProgressBar(0, length(apps), style=3)
 
suppressWarnings(map_df(1:length(apps), function(i) {
 
  x <- apps[i]
 
  setTxtProgressBar(pb, i)
 
  is_vuln <- FALSE
  version <- ""
 
  app_name <- sub("\\.app$", "", basename(x))
  app_loc <- sub("^/", "", dirname(x))
 
  to_look <- c(sprintf("%s/Contents/Frameworks/Autoupdate.app/Contents/Info.plist", x),
               sprintf("%s/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/Info.plist", x),
               sprintf("%s/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/Autoupdate.app/Contents/Info.plist", x))
 
  is_there <- map_lgl(c(sprintf("%s/Contents/Frameworks/Sparkle.framework/", x), to_look), file.exists)
 
  has_sparkle <- any(is_there)
 
  to_look <- to_look[which(is_there[-1])]
 
  discard(map_chr(to_look, function(x) {
    read_plist(x)$result$CFBundleShortVersionString %||% NA
  }), is.na) -> vs
 
  if (any(map_dbl(vs, function(v) { safe_compare(v, "1.16.1")$result %||% -1 }) < 0)) {
    is_vuln <- TRUE
    version <- vs[1]
  }
 
  data_frame(app_loc, app_name, has_sparkle, is_vuln, version)
 
})) -> app_scan_results
 
close(pb)
 
select(arrange(filter(app_scan_results, has_sparkle), app_loc, app_name), -has_sparkle)

Microsoft’s newfound desire to make themselves desirable to the hipster development community has caused them to make many things [open](https://github.com/Microsoft/) and/or free of late. One of these manifestations is [Visual Studio Code](https://code.visualstudio.com/), an [Atom](https://atom.io/)-ish editor for us code jockeys. I have friends at Microsoft and the Revolution R folks are there now, so I try to give things from Redmond a shot more than I previously would, especially when they make things for Mac.

VS code is so much like Atom (or even [Sublime Text](http://www.sublimetext.com/)) that I won’t go into a full review of it. Suffice it to say it has a file selector pane, editor panes, output panes, snippets, theme support and is pretty extensible. One requirement I appreciate is that it forces you to think of code in terms of projects (you select a directory to edit in) and I also appreciate that they made git a first-class citizen.

Since I do not spend much time building large, compiled applications (this—along with web apps—seems to be VS Code’s sweet spot) there isn’t much initial appeal for me. It also lacks the “intellisense” support for the main language I use (R) so I’m left with basic syntax highlighting (the 90’s called and want their basic editor capabilities back).

None of that would initially drive me away from using something like VS Code and I may end up using it for HTML/CSS/JavaScript projects or even fire it up when I need to do some work in Python or Go. But I won’t be using it for R any time soon. While the aforementioned lack of “intellisense” for R is an issue, I don’t rely on the auto-completion for R but it does occasionally speed up typing and definitely helps with the more esoteric function definitions in equally esoteric packages.

The biggest show-stopper for VS Code is the lack of REPL (a read-eval-print loop) for R. I can fire up an R script in Sublime Text or even Atom and run individual lines of code that are executed in an R session that runs in the background and outputs in an editor pane. It works well but it is (unsurprisingly) a far cry from the tight integration of similar functionality in RStudio. VS Code can run R scripts (it just runs the code through R as you would at the command-line) but has no REPL for R, which means you end up executing the entire script as you go along. No saved state (more on that in a second) means that the beautiful data frame your code created that took 10 minutes to build will take 10 minutes to build every time you tweak model parameters or ggplot2 aesthetics. Granted, you could call R with `–save` but then you have to check for the presence of data structures in your code (so you might as well be programming in non-interactive Python).

An offshoot of the details behind this show-stopper is that you do not get graphics output in a window. You get a single PDF of all plots, just as you would if you ran the R script at the command-line. If you’ve been spoiled by RStudio or even cutting and pasting code from an editor into the R GUI, you will immediately miss the graphics viewer pane.

Unless Microsoft (or some community contributors who desperately want to use R in VS Code for some reason) add some of this functionality to VS Code (including support for seamlessly spinning R scripts and knitting R markdown documents), I cannot recommend using to anyone in the R community.

Having said that, here’s the `tasks.json` configuration if you want to be able to hit `Command-Shift-B` in an R script in VS Code and have it execute and display the output. This configuration is for the official R Project build of R and should work even after a R version upgrade).

{
	"version": "0.1.0",
	"command": "/Library/Frameworks/R.framework/Resources/bin/R",
	"showOutput": "always",
	"args": [
		"--no-restore",
		"--no-save",
		"--quiet",
		"--file=${file}"
	]
}

If you are using VS Code for R (on any platform) your comments would be especially most welcome. It’d be great to hear why you’re using it and how you’ve configured it to help make you as productive as RStudio or ESS has for others.

In a previous post I provided “mouse-heavy” instructions for getting R running on your Mac. A few of the comments suggested that an “all Homebrew” solution may be preferable for some folks. Now, there are issues with this since getting “support” for what may be R issues will be very difficult on the official mailing lists as you’ll immediately be told to “use the official distribution” by some stalwart R folks (this happens on StackOverflow and other forums as well). However, if you have a thick skin and can be somewhat self-sustaining, Homebrew is a superb alternative to setting up your R environment (and other things) on your OS X system.

What is ‘Homebrew’?

Homebrew is the “missing package manager for OS X”. It’s similar to apt, yum and other package managers on linux/BSD that enable you to install open source (and other types of) packages without having to do the download→unarchive→compile→curse→google→compile→curse→google some more→compile→smile→test→install dance manually. MacPorts is another third-party package manager for OS X, but I use Homebrew, so you get Homebrew examples here.

Homebrew’s inventory of packages comes from github repositories that contain “Formulas” for where to get package components and how to (literally) make them work on OS X. Here’s a small-ish example (some Formula are fairly long/involved/complex) of a Homebrew Formula for the cowsay utility (what, you don’t like to have ASCII animals give you handy messages?):

class Cowsay < Formula
  desc "Configurable talking characters in ASCII art"
  homepage "https://web.archive.org/web/20120225123719/http://www.nog.net/~tony/warez/cowsay.shtml"
  url "http://ftp.acc.umu.se/mirror/cdimage/snapshot/Debian/pool/main/c/cowsay/cowsay_3.03.orig.tar.gz"
  sha256 "0b8672a7ac2b51183780db72618b42af8ec1ce02f6c05fe612510b650540b2af"

  bottle do
    cellar :any_skip_relocation
    revision 1
    sha256 "c041ce7fbf41fd89bf620ae848e3b36fe1e69ab3e2dfca18bc2f2e79cfe8063a" => :el_capitan
    sha256 "ffacfb987481394174267fd987dea52607825e3542d1ea3d0b7aa4ccf7ea5cc5" => :yosemite
    sha256 "12c41b969af30817a4dc7ec25572fe1b707b9d4dcb46d8cc06d22264594219c1" => :mavericks
  end

  # Official download is 404:
  # url "http://www.nog.net/~tony/warez/cowsay-3.03.tar.gz"

  def install
    system "/bin/sh", "install.sh", prefix
    mv prefix/"man", share
  end

  test do
    output = shell_output("#{bin}/cowsay moo")
    assert output.include?("moo")  # bubble
    assert output.include?("^__^") # cow
  end
end

It has:

  • a description of what the package is
  • the official location of the program/libraries “home”
  • where the main URL of the contents of the program/library is
  • weird hex strings to help the Homebrew ecosystem now pwn you
  • instructions for how to install (with optional patching of problematic code on particular setups)
  • test/validation instructions

(You can/should overlook the fact they use icky Ruby for this whole thing.)

There are thousands of Formula in the main Homebrew repository and you can “tap” other (properly organized) GitHub repositories for other (usually task-specific) formula. We’ll need to do this for R.

Finally, the Homebrew community has also come up with the notion of Casks where actual binary OS X programs (and other things) can be installed. We’ll use this as well for some ancillary components.

Apart from the ease of initial setup, the truly compelling part of using Homebrew is that all it takes to update components/libraries is to do a:

brew update && brew upgrade

from a Terminal prompt. You should get into the habit of issuing those commands daily-ish.

Yes, you will need to become comfortable in the Terminal (or, preferably, iTerm 2) to use the Homebrew ecosystem (though there are some efforts to make this more GUI-user friendly).

Using Homebrew to Create & Maintain Your R Installation

I won’t provide much (if any) color commentary to the commands below. Suffice it to say that in a few short lines of a script, you’ll end up having:

  • R (with gfortran and the vast majority of required support libraries for many packages)
  • Oracle Java (a later step in the sequence ensures R knows about your Java install)
  • X11 (XQuartz)
  • MacTex
  • RStudio
  • extra SVG, XML, curl, geo-munging and C++ support libraries
  • A cool font for RStudio (FiraCode, though that’s not necessary)
  • iTerm 2 (optional)
  • GitUp git gui (optional)

If it’s your first time using Homebrew you’ll need to do this:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

NOTE that I’m generally against piping output of curl to run raw on your system without validation, so you can download https://raw.githubusercontent.com/Homebrew/install/master/install and verify it (or ask a security friend to verify it), but you’ll be trusting the Homebrew ecosystem to not pwn your system for the rest of your time using it, so as long as you trust that I gave you the proper URL to get to the installer, cut/paste away.

Once Homebrew is setup I’d recommend copying and pasting each line (below) one-by-one to get familiar with Homebrew operations and messaging.

This can be a pretty scary experience if you’re not used to the command-line, so drop a note in the comments or on Twitter (target @hrbrmstr and use the #rstats #homebrew tags) if some things go awry or get confusing and I’ll try to help as much as I can.

brew tap caskroom/cask
brew install brew-cask

brew install Caskroom/cask/xquartz
brew cask install java

brew tap homebrew/science
brew install R --with-openblas # added --with-openblas based on a spiffy comment by Lincoln Mullen

brew install Caskroom/cask/rstudio


# For latex:
brew cask install mactex

# OR YOU CAN DO
brew install gnupg
brew cask install basictex # suggested by @noamross
# plus the following, provided by Kras.Pten
# read why here: http://rud.is/b/2015/10/22/installing-r-on-os-x-100-homebrew-edition/#comment-10603
sudo tlmgr update –-self 
sudo tlmgr update –-all 
sudo tlmgr install titling framed inconsolata

# DO NOT DO BOTH!

brew install libsvg curl libxml2 gdal geos boost

R CMD javareconf JAVA_CPPFLAGS=-I/System/Library/Frameworks/JavaVM.framework/Headers

brew tap caskroom/fonts
brew cask install font-fira-code
brew cask install iterm2
brew cask install gitup           # if you want a GUI for git stuff (h/t @jennybryan)

That’s quite a bit less clicking/typing from what was required in the previous post.

Fin

I validated that entire configuration on a completely fresh installation of El Capitan (OS X 10.11) in a VM. At the end, I had a fully-functioning data-science setup. Hopefully, you will as well.

If you have suggestions for other Homebrew things to add to make a good “data science OS X setup”, drop a note in the comments!

P.S.

Once you have a full Homebrew & “Cask” setup, the way to keep up-to-date with everything is more like:

brew update && brew upgrade brew-cask && brew cleanup && brew cask cleanup

but an complete “how to use Homebrew” guide is beyond the scope of this post.

NOTE: The comments are a must read for this. Some excellent additional advice and “gotchas” by some super-helpful readers.


I was in a conversation with an academic colleague (wicked smart dude) and the subject of installing R came up (NOTE: this will happen to you, too, if you ever have the misfortune to have a face-to-face convo with me ;-). They noted that getting up and running with R was not as seamless as one would like it to be and, to be honest, I have to agree, especially after typing the rest of this post out.

I recently had a similar experience helping folks who use Windows get R & RStudio up and running and that’s even more of a nightmare, especially if you do not have Administrator privileges (or, perhaps I just scare easily).

Prior to these experiences, I never really stopped to consider just how less friendly the installation process of R is when compared to Excel, Tableau or other apps one might use for data analysis and visualization. Hopefully this will becomre a top priority for the R Consortium.

Since this colleague uses OS X, I offered to put together instructions for how to get R & RStudio installed and finally had 5 minutes to crank out a blog post to help the broader community with the information.


Get R

R_for_Mac_OS_X


Verify R itself is working

  • Look in the Applications folder for the R application.
  • Double-click it and you should see an R console window.
  • If that did not work, try installing R again
  • Once you’ve verified R is working, quit the app

R_Console


Download RStudio

RStudio is an integrated development environment for R that will make your life and coding easier.

RStudio_-_Download_RStudio


Verify RStudio & R are working together

  • Look in the Applications folder for the RStudio application.
  • Double-click it and you should see an RStudio window with four panes.

niasra_uow_edu_au_content_groups_public__web__inf__math_documents_mm_uow183084_pdf

From now on, just start RStudio when you want to work in R.


[Optional] Install XQuartz

Some functions in R require an “X11 Server” and/or libraries associated with an X11 server. Apple does not provide this software with OS X anymore so you have to do it on your own via a third-party application called XQuartz.

XQuartz


[Optional] Install Xcode Tools

Some R packages require compilation. That requires utilities not installed on OS X by default. You can wait to do the following until it’s needed, but since you’re already installing things…

  • Get Xcode https://itunes.apple.com/us/app/xcode/id497799835?mt=12 and install it like any “normal” Mac application
  • When the intallation is done, open Xcode then close it just to verify it installed correctly
  • Find and open the Terminal program in the Utilities folder under the Applications folder
  • Paste the following into the Termainal window and hit enter/return (accept any dialog/prompt):
xcode-select --install`
  • Close the Terminal application

[Optional] Set yourself up for easier future compiled package installation

Some R packages need additional libraries to work and most aren’t on your system by default. There are a myriad of ways to get these libraries, and the way I obtain them is via the homebrew utility. You can save yourself the trouble of installing homebrew later by doing the following now:

  • Find and open the Terminal program in the Utilities folder under the Applications folder
  • Paste the following into the Terminal window and hit enter/return:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • Read and accept the various prompts until it’s installed
  • Close the Terminal application

You can now do brew install xyz in the future when a library is needed to support a package. Drop a note in the comments if you’d like this discussed more in a future blog post.


[Optional] [If you have an hour+ to kill] Install MacTeX

R has an academic history and there are many semi-advanced functions that are tied to something called latex. Installing latex for OS X is not hard, just time (and bandwidth) consuming (it’s about the same size as a new OS X installer). If you delve into package creation or do more detailed output work in R, you’ll want to install MacTex sooner than later.


Fin

If you have any changes/additions/etc drop a note in the comments. I may even stick this on github to make it easier to contribute in the future.

I’ve operated a [Weather Underground](http://www.wunderground.com/) [Personal Weather Station](http://www.wunderground.com/weatherstation/about.asp) (PWS) [[KMEBERWI7](http://www.wunderground.com/personal-weather-station/dashboard?ID=KMEBERWI7#history)] off-and-on (hardware issues notwithstanding) for as long as I can remember, and I thought it was about time to finally do an Alfred↔PWS mashup. My personal requirements were quite modest:

– 5 reading history (including most current)
– Ability to copy all the current day’s readings as CSV
– Quickly get to my PWS data w/o a bookmark

[Alfred](http://www.alfredapp.com/) makes all this possible via customized workflows that support many scripting environments, including Python. Here’s a quick preview before going into the details:

pws-alfred-results

It’s a fairly simple workflow:

– Grab today’s “raw” data (and clean it up)
– Select the last 5 entries
– Connect a URL action and use the full CSV as clipboard contents for any copy action

The full Python code is below and [on github](https://github.com/hrbrmstr/alfred-pws), and you can hit that github link or [Packal](http://www.packal.org/workflow/pws-history) for the compiled workflow. It’s been tested on Mavericks, but more eyes are always welcome.

Customizing the workflow for your own/favorite PWS is as simple as changing the `station` variable.

There’s plenty of room for improvement, including

– performing a background download of the PWS data
– using a sparkline graph as the icon
– customizing which data fields are returned
– providing commands to get/set your/favorite PWS
– providing options for the “copy” return type (currently CSV, but potentially XML or JSON)

Don’t hesitate to post issues or pull requests and check back for updates (as I’m sure some of those improvements will be making their way into the workflow).

import re
import csv
import sys
import datetime
from lib import requests
from workflow import Workflow, web
from StringIO import StringIO
 
# retrieve today's history for station "X"
 
def get_wx_data(station):
 
  tdy = datetime.datetime.today()
 
  # construct the URL for "today"
  url = 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=%s&day=%d&month=%d&year=%d&graphspan=day&format=1' % (station, tdy.day, tdy.month, tdy.year)
 
  r = web.get(url) # get the data
 
  r.raise_for_status() # report any errors
 
  return(re.sub("\n\n", "\n", re.sub("^\n|<br>", "", r.text))) # clean up the output & pass it back to main control
 
 
# main workflow control
 
def main(wf):
 
  station = "KMEBERWI7" # change to use your own/favorite station
 
  resp = get_wx_data(station)
 
  # only want last 5 readings, change this to whatever you want
  max = 5
 
  i=0
  for row in reversed(list(csv.reader(StringIO(resp)))):
    wf.add_item(title=row[0] + " | " + row[1] + "F | " + row[3] + "in | " + row[8] + "%", 
                subtitle=station, # so you know where you're getting data from
                arg=station, # passed as query to URL action - http://www.wunderground.com/personal-weather-station/dashboard?ID={query}#history
                valid=True, # it can be opened in the browser
                icon="/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/ToolbarInfo.icns", # info icon
                copytext=resp) # get the whole CSV file in a copy
    if (i==max): break
    i += 1
 
  # output to alfred
 
  wf.send_feedback()
 
if __name__ == u"__main__":
  wf = Workflow()
  sys.exit(wf.run(main))

Over on the [Data Driven Security Blog](http://datadrivensecurity.info/blog/posts/2014/Apr/making-better-dns-txt-record-lookups-with-rcpp/) there’s a post on how to use `Rcpp` to interface with an external library (in this case `ldns` for DNS lookups). It builds on [another post](http://datadrivensecurity.info/blog/posts/2014/Apr/firewall-busting-asn-lookups/) which uses `system()` to make a call to `dig` to lookup DNS `TXT` records.

The core code is below and at both the aforementioned blog post and [this gist](https://gist.github.com/hrbrmstr/11286662). The post walks you though creating a simple interface and a future post will cover how to build a full package interface to an external library.