rud.is

Just When You Thought It Was Safe To Make A Slopegraph

Thanks to a nice call-out post link on Flowing Data in my RSS feeds this morning, I found Naomi Robbins’ Effective Graphs Forbes blog, perused the archives a bit and came across her post on arrow charts.

She presented a nice comparison between (ugh) pie charts, arrow charts and slopegraphs. Sadly, both the NPR slopegraph and Peltier’s slopegraph included in the article committed some of the cardinal sins of slopegraphs I have pointed out previously. Let’s take a look (click on each graphic to make them bigger):

Use of binning/rounding without annotation
Use of binning/rounding but not to show rate of change
Stacking labels (presenting rank where none exists)

More faithful representations would be explicit rounding/binning (to only show rate of change):

or the full scale version (warning: huge slopegraph) to accurately show the value differences and rate of change:

The data set is small, so transcription is not really be an issue, but here is is for you if you want to play with it some more.

This is definitely a case where her arrow charts are a solid alternative to slopegraphs, so definitely check out her post.

VERIS Community :: JSON vs XML

2012-07-20 – 21:06
Posted in Information Security, Javascript, VERIS
Comments (2)

You may not be aware of the fact that the #spiffy Verizon Biz folk have some VERIS open source components, one of which is the XML schema for the “Vocabulary for Event Recording and Incident Sharing”.

While most Java-backends will readily slurp up and spit back archaic XML data, the modern web is a JSON world and I wanted to take a stab at encoding the sample incident in JSON format since I’m pretty convinced this type of data is definitely a NoSQL candidate and that JSON is the future.

I didn’t run this past the VZB folk prior to the post, but I think I got it right (well, it validates, at least :-) :

```
{
```
```
  "VERIS_community": {
```
```
    "incident": {
```
```
      "incident_uid": "String",
```
```
      "handler_id": "String",
```
```
      "security_compromise": "String",
```

      "related_incidents": { "related_incident_id": "String" },

```
      "summary": "String",
```
```
      "notes": "String",
```
```
      "victim": {
```
```
        "victim_id": "String",
```
```
        "industry": "000",
```

        "employee_count": "25,001 to 50,000",

```
        "location": {
```
```
          "country": "String",
```
```
          "region": "String"
```
```
        },
```
```
        "revenue": {
```
```
          "amount": "0",
```
```
          "iso_currency_code": "USD"
```
```
        },
```
```
        "security_budget": {
```
```
          "amount": "0",
```
```
          "iso_currency_code": "USD"
```
```
        },
```
```
        "notes": "String"
```
```
      },
```
```
      "agent": [
```
```
        {
```
```
          "motive": "String",
```
```
          "role": "String",
```
```
          "notes": "String"
```
```
        },
```
```
        {
```
```
          "type": "External",
```
```
          "motive": "String",
```
```
          "role": "String",
```
```
          "notes": "String",
```
```
          "external_variety": "String",
```
```
          "origins": {
```
```
            "origin": {
```
```
              "country": "String",
```
```
              "region": "String"
```
```
            }
```
```
          },
```
```
          "ips": { "ip": "String" }
```
```
        },
```
```
        {
```
```
          "type": "Internal",
```
```
          "motive": "String",
```
```
          "role": "String",
```
```
          "notes": "String",
```
```
          "internal_variety": "String"
```
```
        },
```
```
        {
```
```
          "type": "Partner",
```
```
          "motive": "String",
```
```
          "role": "String",
```
```
          "notes": "String",
```
```
          "industry": "0000",
```
```
          "origins": {
```
```
            "origin": {
```
```
              "country": "String",
```
```
              "region": "String"
```
```
            }
```
```
          }
```
```
        }
```
```
      ],
```
```
      "action": [
```

        { "notes": "Some notes about a generic action." },

```
        {
```
```
          "type": "Malware",
```
```
          "notes": "String",
```
```
          "malware_function": "String",
```
```
          "malware_vector": "String",
```
```
          "cves": { "cve": "String" },
```

          "names": { "name": "String" },

          "filenames": { "filename": "String" },

          "hash_values": { "hash_value": "String" },

          "outbound_IPs": { "outbound_IP": "String" },

          "outbound_URLs": { "outbound_URL": "String" }

```
        },
```
```
        {
```
```
          "type": "Hacking",
```
```
          "notes": "String",
```
```
          "hacking_method": "String",
```
```
          "hacking_vector": "String",
```
```
          "cves": { "cve": "String" }
```
```
        },
```
```
        {
```
```
          "type": "Social",
```
```
          "notes": "String",
```
```
          "social_tactic": "String",
```
```
          "social_channel": "String",
```
```
          "email": {
```

            "addresses": { "address": "String" },

            "subject_lines": { "subject_line": "String" },

```
            "urls": { "url": "String" }
```
```
          }
```
```
        },
```
```
        {
```
```
          "type": "Misuse",
```

          "notes": "Notes for a misuse action.",

```
          "misuse_variety": "String",
```
```
          "misuse_venue": "String"
```
```
        },
```
```
        {
```
```
          "type": "Physical",
```

          "notes": "Notes for a physical action.",

```
          "physical_variety": "String",
```

          "physical_location": "String",

```
          "physical_access": "String"
```
```
        },
```
```
        {
```
```
          "type": "Error",
```

          "notes": "Notes for a Error action.",

```
          "error_variety": "String",
```
```
          "error_reason": "String"
```
```
        },
```
```
        {
```
```
          "type": "Environmental",
```

          "notes": "Notes for a environmental action.",

          "environmental_variety": "String"

```
        }
```
```
      ],
```
```
      "assets": {
```
```
        "asset_variety": "String",
```
```
        "asset_ownership": "String",
```
```
        "asset_hosting": "String",
```
```
        "asset_management": "String",
```
```
        "os": "String",
```
```
        "notes": "String"
```
```
      },
```
```
      "attribute": [
```
```
        { "notes": "String" },
```
```
        {
```

          "type": "ConfidentialityPossession",

```
          "notes": "String",
```
```
          "data_disclosure": "String",
```
```
          "data": {
```
```
            "data_variety": "String",
```
```
            "amount": "0"
```
```
          },
```
```
          "data_state": "String"
```
```
        },
```
```
        {
```

          "type": "AvailabilityUtility",

```
          "notes": "String",
```

          "availability_utility_variety": "String",

          "availability_utility_duration": "String"

```
        }
```
```
      ],
```
```
      "timeline": {
```

        "timestamp_first_known_action": {

```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        },
```

        "timestamp_data_exfiltration": {

```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        },
```

        "timestamp_incident_discovery": {

```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        },
```
```
        "timestamp_containment": {
```
```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        },
```

        "timestamp_initial_compromise": {

```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        },
```
```
        "timestamp_investigation": {
```
```
          "year": "2001",
```
```
          "month": "--12",
```
```
          "day": "---17",
```
```
          "time": "14:20:00.0Z"
```
```
        }
```
```
      },
```
```
      "discovery_method": "String",
```
```
      "control_failure": "String",
```
```
      "corrective_action": "String",
```
```
      "loss": {
```
```
        "loss_variety": "String",
```
```
        "loss_amount": {
```
```
          "amount": "0",
```
```
          "iso_currency_code": "USD"
```
```
        }
```
```
      },
```
```
      "impact_rating": "String",
```
```
      "impact_estimate": {
```
```
        "amount": "0",
```
```
        "iso_currency_code": "USD"
```
```
      },
```
```
      "certainty": "String"
```
```
    }
```
```
  }
```
```
}
```

I believe I’d advocate for the “timestamps” to be more timestamp-y in the JSON version (the dashes do not make much sense to me even in the XML version) and any fields with min/max range values to be separated to actual min & max fields. I’m going try to find some cycles to mock up a MongoDB / Node.js sample to show how this JSON format would work. At a minimum, even a rough conversion from XML to JSON when requested by a browser would make it easier for client-side data rendering/manipulation.

If you’re not thinking about using VERIS for documenting incidents or hounding your vendors to enable easier support for it, you should be. If you’re skittish about recording incidents anonymously into the VERIS Community, you should get over it (barring capacity constraints).

“Fibonacci” Graphing

Benchmarking/profiling is one of the fundamental practices for tech folk and Feng Shen’s recent post to Hacker News continues this fine tradition with a look at startup & run times for “fibonacci(40)” in seven computer languages (two ‘C’ variants, Clojure, go, python, node & Java).

Good, quick project, but I didn’t think the chart he made (check the site) comparing the results did proper justice (though python is the clear loser even with Clojure’s horrid startup time) to each language, so I charted out the startup times separately:

Here’s a Google Drive spreadsheet for the data that also has larger views of the charts.

SSH Password Time-series Heatmap In D3

2012-07-12 – 14:33
Posted in Charts & Graphs, d3, Information Security
Tagged d3, heatmap, honeypot, passwords
Comments (1)

In @jayjacobs’ latest post on SSH honeypot passsword analysis he shows some spiffy visualizations from crunching the data with Tableau. While I’ve joked with him and called them “robocharts”, the reality is that Tableau does let you work on visualizing the answers to questions quickly without having to go into “code mode” (and that doesn’t make it wrong).

I’ve been using Jay’s honeypot data for both attack analysis as well as an excuse to compare data crunching and visualization tools (so far I’ve poked at it with R and python) in an effort to see what tools are good for exploring various types of questions.

A question that came to mind recently was “Hmmm…I wonder if there is a patten to the timings of probes/attacks?” and I posited that a time-series view across the days would help illustrate that. To that end, I came up with the idea of breaking the attacks into one hour chuncks and build a day-stacked heatmap which could be filtered by country. Something like this:

I’ve been wanting to play with D3 and exploring this concept with it seemed to be a good fit.

Given that working with the real data would entail loading a ~4MB file every time someone viewed this blog post, I put the working example in a separate page where you can do a “view source” to see the code. Without the added complexity of a popup selector and loading spinner, the core code is about 50 lines, much of which could be condensed even further since it’s just chaining calls in javascript. I cheated a bit and used jQuery, too, plus made some of it dependent on WebKit (the legend may look weird in Firefox) due to time constraints.

The library is wicked simple to grok and makes it easy to come up with new ways to look at data (as you can see from the examples gallery on the D3 site).

Unfortunately, no real patterns emerged, but I’m going to take a stab at taking the timestamps (which is the timestamp at the destination of the attack) and align it to the origin to see if that makes a difference in the view. If that turns up anything interesting, I’ll make another quick post on it.

Given that much of data (“big” or otherwise) analysis is domain knowledgable folk asking interesting questions, are there any folks out there who have questions that they’d like to see explored with this data set?

Security & Privacy Of Mountain Lion’s Dictation Feature

2012-07-12 – 09:43
Posted in Apple, Compliance, Information Security, Mountain Lion
Comments (3)

With Gizmodo doing a post hyping Mountain Lion’s new dictation feature it’s probably a good time to note that folks in regulated environments or who just care about security & privacy a bit more than others should not enable or use this feature for the dictation of sensitive information.

From Apple’s own warning on the matter:

When you use the keyboard dictation feature on your computer, the things you dictate will be recorded and sent to Apple to convert what you say into text. Your computer will also send Apple other information, such as your first name and nickname; and the names, nicknames, and relationship with you (for example, “my dad”) of your address book contacts. All of this data is used to help the dictation feature understand you better and recognize what you say. Your User Data is not linked to other data that Apple may have from your use of other Apple services.

It’s much like what happens with Siri, Dragon Dictation or a myriad of other iOS and modern desktop apps/browser extensions. Thankfully, it performs the transfers over SSL, but that still won’t help you if your dictating health, financial or other regulated/NPPI/PII data.

While the feature is cool and does work pretty well, it’s important to make sure you and your users know what it does, how it works and where they can/cannot use it.

Honeypot Analytics : 500 Pretty Passwords

2012-07-09 – 14:23
Posted in Charts & Graphs, Information Security
Tagged honeypot
Comments (2)

I had a few moments this past weekend to play with an idea for visualizing the passwords used against the honeypot @jayjacobs set up. While it’s not as informative as Jay’s weekend endeavors:

I'm messing around with how to visualize 60k passwords, how's this? : http://t.co/ocAjQdxm (cc @hrbrmstr )

— jayjacobs (@jayjacobs) July 7, 2012

it is pretty, and it satisfied my need to make a word cloud out of useful data.

The image below is of the top 500 passwords used against the honeypot and requires an SVG-capable browser and also requires horizontal scrolling, so you can view or download it standalone if there are any issues. For those generally SVG-challenged, there’s also a slightly less #spiffy PNG version to view as well.

Beach Blanket Tweetup
(a.k.a. GraniteSec Tweetup #5)

Brylcream? Check.
Decked out in your best Frankie Avalon or Annette Funicello beach fashions? Check.

Now, polish up your cruiser and get ready ‘cuz @GraniteSec is heading back to Fort Foster [info] [map] at Kittery Point, Maine for it’s fifth family & friends tweetup!

We can’t promise mermaids or sky-diving surfers, but you’re sure to have a groovy time hanging loose and chowing down with other hip cats & chicks.

For all the low down, check out granitesec.org (redirect to the latest tweetup Eventbrite page forthcoming) or rap with @hrbrmstr, @ITSecurity or @1_tjw on Twitter.

(And, if any of you younger cats are having trouble with the lingo/references, check this out, man…)

Also, if you want to help promote the tweetup, you can use the above 500x350px graphic or this 160x400px graphic and link to this post or http://granitesec.org/ which will have an Eventbrite redirect up in the near future.

2012 WhiteHat Security Website Security Statistics Report Redux

2012-07-02 – 12:38
Posted in AppSec, Charts & Graphs, Information Security, Metrics
Leave a Comment

This is an inaugural post for @MetricsHulk, on the condition that there are few – if any – “ALL CAPS” bits. Q3&4 tend to be “report season”, and @MetricsHulk usually has some critiques, praises, opines and suggestions (some smashes, too) to offer as we are inundated with a blitz of infographics.

The always #spiffy @WhiteHatSec released their 2011 Web Site Security stats report [direct link (PDF)] last week (here’s one of their teaser tweets):

The Avg # of Serious* vulns found per website per year was 79, way less
than 230 in 2010 & down from 1,111 in 2007. http://t.co/Q5GIwZKi

— Synopsys Software Integrity (@SynopsysAppsec) June 27, 2012

With over 7,000 sites and hundreds of diverse organizations represented in the report, it is a great resource for folks to see how they stack up (more on that in a bit). Security folks should also take some encouragement from the report since:

Real vulnerabilities are down (significantly)
WAFs can help
Vulnerabilities are getting fixed faster (when found)

@WhiteHatSec does a fine job summarizing key & extended findings (hint: read the report), and they are awesomely up-front and honest with regard to the findings (see pages 4 & 5 for their analysis on why the ‘good stats’ might be so good).

The report is chock-full of data. Real. Data. The only way it could have been better data-wise is if they provided a Google Docs bundle of raw numbers. (NOTE: I didn’t get all the data in there, but it has decent amount from the report)

I do think there is some room for improvement. Take, for example, the – sigh – donut chart on page 9. I might be inclined to refrain from comment if this was one of those hipster infographics that seem to be everywhere these days. A pie chart isn’t much better, but at least we’re able to process the relative sizes a bit better when the actual angles are present. Here’s a before/after makeover for your comparison/opine (click for larger version):

We get an immediate sense of scale from the bars and it removes the need for the “Frosted Lucky Charms” color-wheel effect. The @WhiteHatSec folk use bars (very appropriately) almost everywhere else, so I’m not sure what the design decision was for deviating for this part of the report.

The next bit that confused me was Figure 18 (page 15). I’m having difficulty both figuring out where the “79” value comes from (I can’t get to it by averaging the values presents) and grok’ing the magnitude of the differences from the bubbles. So, here’s another before/after makeover for your comparison/opine (click for larger version):

Finally, I think Figure 23 & 24 could do with a bit of a slopegraph makeover, as the spirit of the visualization is to show year-over-year differences. The first two slopegraphs used the “Tufte binning technique“, so you’ll need to refer to the companion data tables if you want exact numbers for comparison (the trend is more important, IMO).

Average Days Open

Average Days to Close

Remediation Rates by Year

(You can also download easier to read PDFs of the slopegraphs)

Absolutely no one should take the makeover suggestions as report slander. As stated at the beginning of the post, @WhiteHatSec is open about the efficacy of their data and analysis, plus they provide actual data. The presentation of stats & trending by industry and vulnerability type should help any organization with an appsec program figure out if they are doing better or worse the others in their sector and see if they are smashing bugs with similar success. It also gives the general infosec community a view that we would otherwise not have. I would encourage other organizations to follow @WhiteHatSec’s example, even if it means more donut charts (mmm…donuts).

What information did you glean from the WhiteHat report, or what makeovers would you encourage for the next one?