Skip navigation

Google Spreadsheet “importHTML” Rocks For Quick Analytics

I usually take a peek at the Internet Traffic Report (ITR) a couple times a day as part of my routine and was a bit troubled by all of the red today:

I wanted to do some crunching on the data, and I deliberately do not have Word or Excel on my new MacBook Pro (for reasons I can detail if asked). A SELECT / CUT / PASTE into TextWrangler did not really thrill me and I knew there had to be a way to get non-marked-up, columnar data into a format I could mangle and share easily.

Enter, Google Shreadsheet’s importHTML function.

If you don’t have the forumla bar enabled in Google Spreadsheets, just go to View->Formula Bar to enable it. Once there, enter the following in the formula bar to get the data from the ITR into a set of columns that will auto-update every time you reference the sheet.

=importHTML("http://www.internettrafficreport.com/namerica.htm","table",0)

(as you can see, it’s not case sensitive, either)

Yes, I know Excel can do this. I could have done a quick script whack the pasted data in TextWrangler. You can do something similar in R with htmlTreeParse + xpathApply and Perl has HTML::TableContentParser (and other handy modules), but this was a fast, easy way to get me to a point where I could do the basic analytics I wanted to perform (and, sometimes, all you need is quick & easy).

Official Google Help page on importHTML.

USDA Driving Towards Both Food Insecurity And Cyber-Insecurity?

It’s rare that two of my passions—food and information security—intersect, but thanks to the USDA’s announcement of their Blueprint For Stronger Service, I can touch on both in one post.

In 2011, the Obama administration challenged all departments to reduce costs in a effort dubbed the “Campaign to Cut Waste“. In response, the USDA has managed to trim annual expenses by $150 million through a number of efforts. One such effort is to close 259 domestic USDA offices (you can see which states are impacted below).

I’m going to expand on why this is a bad idea over at #nom later this week, but 2011 was not a good year in terms of controlling food poisoning in the United States and I don’t think closing offices will make for better oversight.

Other efforts focus on the elimination of redundancies and inefficiencies. The Blueprint has 27 initial (or to-be-implemented immediately) improvements that include the following:

  • Consolidate more than 700 cell phone plans into about 10
  • Standardize civil rights training and purchases of cyber security products
  • Centralize civil rights, human resource, procurement, and property management functions

So, they were either getting gouged by suppliers (unlikely since there is negotiated pricing for the government) or the USDA’s “cyber-security” strategy was severely fragmented (and, thus, broken) enough that even finance folks could see the problem. Regardless of the source, it had to be pretty bad to make it to the top three of 27 immediate items (and called out in every sub-department press release) and even more so amongst over 160 initiatives that are being or have been put in place.

I still cannot find the details of the plan or budget analysis that went into the focus on cyber security products (links appreciated if you have them), but as private organizations continue their efforts to defend against existing and emerging threats, it might be worth a look at your strategy and spend a bit more closely. Would your infosec department be included in a similar list if your organization went through such a sweeping cost-cutting analysis program? Is your portfolio of security products as optimized as it can be? Could you use a budget sweep as an opportunity to leap frog your security capabilities (e.g. move to whitelisting vs signature-based anti-malware) vs just pressure your existing vendors and re-negotiate contracts?

Unfortunately, the government being the government, I’m now even more concerned that the USDA may need to worry about increased infections on both the food-level and the “cyber” level.

Getting Things Done : A Cobbler’s Tale

Starting sometime mid-year in 2011, I began having more ‘stuff’ to do than even my eidetic memory could help with. It’s not that I forgot things, per se, but the ability to mentally recall and prioritize work, family, personal and other tasks finally required some external assistance and I resolved to find a GTD system by the end of January.

Being an OS X user, there are great choices out there (both of those have iOS sister-apps, too). However, I’m not just an OS X user. As I was saying to @myrcurial (and even @reillyusa) the other day, I dislike being locked in to proprietary solutions. Plus, the $120 price tag for OmniFocus (OS X + iPad) seemed like a king’s ransom, especially since I am also an Android user (OmniFocus only has an iOS app) and pay for both Dropbox and various virtual hosts. Believing that I still have some usable skills left, I decided to — as @hatlessec characterized my solution — cobble something together on my own.

Once upon a time, I did maintain a .plan file (when I had sysadmin duties), but really doubted the efficacy of it and finger in the age of the modern web. The thought of machinating SQLite databases, parsing XML files or even digesting bits of JSON seemed overkill for my purposes. Searching through my Evernote clippings, my memory was drawn back to one of my favorite sites, Lifehacker, which has regular GTD coverage. After re-poking around a bit, I decided to settle on @ginatrapani’s @todotxtapps for meeting the following requirements (in order):

  • It uses a plain text file with a simple structure – (no exposit necessary…the link is a quick read and the format will become second nature after a glance)
  • It is Free (mostly) – mobile apps are ~$2.00USD each and if you need more than free Dropbox hosting and want a web interface, there are potential hosting costs. If you count your setup time as money, then add that in, too.
  • It runs on OS X, BSD, Windows & Linux – no platform lock-in
  • It has a thriving community – without being backed by a vendor (like the really #spiffy @omnigroup), a strong developer & user community is extremely important to ensure the longevity of the codebase. Todo.txt has very passionate developers and users who are very active on all fronts.
  • It is very extensible & integrable – I used @alfredapps to give me a quick OS X “GUI CLI” to the todo.sh commands. I built an Alfred keyword for my most used Todo.txt functions along with a generic one to bring up vim in a Terminal.app window for a free-form edit. Alfred’s shell-commands also give me @growlmac integration (so I get some feedback after working with tasks).

    I also integrated it with @geektool. I won’t steal the thunder from other GeekTool/Todo.txt integration posts (like this one). The GeekTool integration puts my todo’s right in front of me all the time on all my desktops.

    By storing my todo directory in @dropbox, it also makes syncing to my web site and mobile devices a snap.

    On my server, I have a simple cron job setup to e-mail me my todo’s at the beginning of the day (again, so it’s in front of me wherever I look).

  • It runs on iOS AND Android – again, no platform lock-in
  • There’s an optional web interface – the one I linked to (there are others) is far from ideal, but it was quick to setup and has no overt security issues. Properly protected behind nginx or apache, you should have no issues if you need to have a web version handy.

So, while the setup is a bit more than just downloading two commercial apps, it has many other benefits and isn’t too much more work if you already have some of the other pieces in place. If you want more info on the Alfred scripts or any other setup component, drop me a note in the comments.

While I’ve read about many GTD solutions and seen many user-stories of how they met their GTD needs, I’d be interested in what tools you use to ‘get things done’…

Feed Link Has Moved

Feedburner has borked the old RSS feed for the site and has completely disassociated me from it (meaning it’s no longer in my Google Feedburner admin options and they won’t let me re-claim it).

So… the new feed link is http://rud.is/b/feed/atom/.

Apologies for any inconvenience.

Businessweek Infographic Illustrates The Pounding We Took In 2011

Another #spiffy tip from @MetricsHulk:

Evan Applegate put together a great & simple infographic for Businessweek that illustrates the number and size of 2011 data breaches pretty well.


(Click for larger version)

The summary data (below the timeline bubble chart) shows there was a 37.4% increase in reported incidents and over 260 million records exposed/stolen for the year. It will be interesting to see how this compares with the DBIR.

Improve Your Security Metrics For $14.00USD

IT Security Metrics : A Practical Framework for Measuring Security & Protecting Data has has solid reviews by Richard Bejtlich (@TaoSecurity), David J. Elfering (@icxc) & Dr. Anton Chuvakin (@anton_chuvakin), amongst others. You can get it (for a short time) for just about fourteen Washingtons by doing the following.

First, go to this Amazon link and enter “ETXTBOOK” (no quotes) as the code, you’ll get a credit of $10.00USD for Amazon Kindle textbooks. That credit expires on January 9th, 2012, btw.

Now, if you view IT Security Metrics : A Practical Framework for Measuring Security & Protecting Data on Amazon and order it (again, by January 9th, 2012), it will cost you a whole ~$14.00USD

NEISTU 3.0 : Rise of the Tweetup

It’s that time again! Coming to Nashua, NH on 2012-01-21 : The New England Information Security Tweetup 3.0!

You can use the handy graphic below to promote the tweetup or just give folks this handy link.

(Click to enlarge)

Three Resolutions For Web Developers

I’m on a “three things” motif for 2012, as it’s really difficult for most folks to focus on more than three core elements well. This is especially true for web developers as they have so much to contend with on a daily basis, whether it be new features, bug reports, user help requests or just ensuring proper caffeine levels are maintained.

In 2011, web sites took more hits then they ever have and—sadly—most attacks could have been prevented. I fear that the pastings will continue in 2012, but there are some steps you can take to help make your site less of a target.

Bookmark & Use OWASP’s Web Site Regularly

I’d feel a little sorry for hacked web sites if it weren’t for resources like OWASP, tools like IronBee and principles like Rugged being in abundance, with many smart folks associated with them being more than willing to offer counsel and advice.

If you run a web site or develop web applications and have not inhaled all the information OWASP has to provide, then you are engaging in the Internet equivalent of driving a Ford Pinto (the exploding kind) without seat belts, airbags, doors and a working dashboard console. There is so much good information and advice out there with solid examples that prove some truly effective security measures can really be implemented in a single line of code.

Make it a point to read, re-read and keep-up-to-date on new articles and resources that OWASP provides. I know you also need to beat the competition to new features and crank out “x” lines of code per day, but you also need to do what it takes to avoid joining the ranks of those in DataLossDB.

Patch & Properly Configure Your Bootstrap Components

Your web app uses frameworks, runs in some type of web container and sits on top of an operating system. Unfortunately, vulnerabilities pop up in each of those components from time to time and you need to keep on top of those and determine which ones you will patch and when. Sites like Secunia and US-CERT aggregate patch information pretty well for operating systems and popular server software components, but it’s best to also subscribe to release and security mailing lists for your frameworks and other bootstrap components.

Configuring your bootstrap environment securely is also important and you can use handy guides over at the Center for Internet Security and the National Vulnerability Database (which is also good for vulnerability reports). The good news is that you probably only need to double-check this a couple times a year and can also integreate secure configuration baselines into tools like Chef & Puppet.

Secure Data Appropriately

I won’t belabor this point (especially if you promise to read the OWASP guidance on this thoroughly) but you need to look at the data being stored and how it is accessed and determine the most appropriate way to secure it. Don’t store more than you absolutely need to. Encrypt password fields (and other sensitive data) with more than a plain MD5 hash. Don’t store any credit card numbers (really, just don’t) or tokenize them if you do (but you really don’t). Keep data off the front-end environment and watch the database and application logs with a service like Loggly (to see if there’s anything fishy going on).

I’m going to cheat and close with a fourth resolution for you: Create (and test) a data breach response plan. If any security professional is being honest, it’s virtually impossible to prevent a breach if a hacker is determined enough and the best thing you can do for your user base is to respond well when it happens. The only way to do that is have a plan and to test it (so you know what you are doing when the breach occurs). And, you should run your communications plan by other folks to make sure it’s adequate (ping @securitytwits for suggestions for good resources).

You want to be able to walk away from a breach with your reputation as intact as possible (so you’ll have to keep the other three resolutions anyway) with your users feeling fully informed and assured that you did everything you could to prevent it.

What other security-related resolutions are you making this year as a web developer or web site owner and what other tools/services are you using to secure your sites?