Skip navigation

Category Archives: Leadership

A Note to Our Community On How To Hide Your Content From Search Engines

UPDATE 2019-04-17 — The example at the bottom which shows that the, er, randomly chosen site has the offending <meta> tag present is an old result. As of this update timestamp, that robots noindex tag is not on the site. Since the presence status of that tag is in flux, it will continue to be monitored.

Say your organization has done something pretty terrible. Terrible enough that you really didn’t want to acknowledge it initially but eventually blogged about it, and haven’t added a blog post in a long time so that entry is at the top of your blog index page which Google can still index and will since it’s been linked to from this site which has a high rating internally in their massive database.

If you wanted to help ensure nobody finds that original page, there are lots of ways to do that.

First, you could add a Disallow entry in your robots.txt for it. Ironically, some organizations don’t go that route but do try to prevent Google (et al) from indexing their terms of use and privacy policy, which might suggest they don’t want to have a historical record that folks could compare changes to, and perhaps are even planning changes (might be good if more than just me saves off some copies of that now).

Now, robots.txt modifications are fairly straightforward. And, they are also super easy to check.

So, what if you wanted to hide your offense from Google (et al) and not make it obvious in your robots.txt? For that, you can use a special <meta> tag in the header of your site.

This is an example of what that looks like:

datacamp

but that may be hard to see, so let’s look at it up close:

<meta name="robots" content="noindex" class="next-head" />
<title class="next-head">A note to our community (article) - DataCamp</title>
<link rel="canonical" href="https://www.datacamp.com/community/blog/note-to-our-community" class="next-head" />
<meta property="og:url" content="https://www.datacamp.com/community/blog/note-to-our-community" class="next-head" />

That initial <meta> tag will generally be respected by all search engines.

And, if you want to really be sneaky, you can add a special X-Robots-Tag: noindex HTTP header to your web server for any page you want to have no permanent record of and sneak past even more eyes.

Unfortunately, some absolute novices who did know how to do the <meta> tag trick aren’t bright enough to do the sneakier version and get caught. Here’s an example of a site that doesn’t use the super stealthy header approach:

datacamp

FIN

So, if you’re going to be childish and evil, now you know what you really should do to try to keep things out of public view.

Also, if you’re one of the folks who likes to see justice be done, you now know where to check and can use this R snippet to do so whenever you like. Just substitute the randomly chosen site/page below for one that you want to monitor.

library(httr)
library(xml2)

httr::GET(
  url = "https://www.datacamp.com/community/blog/note-to-our-community"
) -> res

data.frame(
  name = names(res$all_headers[[1]]$headers), # if there are more than one set (i.e. redirects) you'll need to iterate
  value = unlist(res$all_headers[[1]]$headers, use.names = FALSE)
) -> hdrs

hdrs[hdrs[["name"]] == "robots",]
## [1] name  value
## <0 rows> (or 0-length row.names)

httr::content(res) %>% 
  xml_find_all(".//meta[@name='robots']")
## {xml_nodeset (1)}
## [1] <meta name="robots" content="noindex" class="next-head">\n

readLines("https://www.datacamp.com/robots.txt")
## [1] "User-Agent: *"                                                              
## [2] "Disallow: /users/auth/linkedin/callback"                                    
## [3] "Disallow: /terms-of-use"                                                    
## [4] "Disallow: /privacy-policy"                                                  
## [5] "Disallow: /create/how"                                                      
## [6] "Sitemap: http://assets.datacamp.com/sitemaps/main/production/sitemap.xml.gz"

Thank you for reading to the end of this note to our community.

Predictions? Humbug! Resolve Is Where It’s At

This is the time of year when pundits and armchair/amateur analysts make predictions for the coming year. Given that only a tiny fraction of them predicted the Sonage of 2011 (not Sony specifically or the level of pwnage) or the RSA/Lockeed [↑, ↑, ↓, ↓, ←, →, ←, →, B, A] multi-faceted “supply chain” attack (most just predicting increased “nation state” hacks) or the decimation of trust in certificate authorities (not that we really trusted them before), it is hardly worth the time reading or seriously considering any post presuming to posit what will occur in 2012 (wait…I can’t resist…and it even fits in 140: “2012 Infosec Prediction: There will be more attacks just like the one this year if not worse in scale and/or magnitude #protip“).

Instead, why not get some resolve and take charge of what will happen in the coming year? “Resolution” & “resolve” have their roots in the Latin “resolvere“, which has a host of contextual meanings. One highly appropriate one is “to find the answer or solution to“. So, rather than pontificate, here are some “resolves” for you for 2012:

  • Resolve to not buy any more products and to make serious use (beyond the typical 5% you are) of the ones you have. That may require ensuring your staff has appropriate training to automate where applicable and tweak appropriately where possible. It may also require a good amount of thinking. In most shops, the last thing needed is more tools. Figure out the best way to use the tools you have. Not only will it improve the efficacy of current investments, it will free up more capital for your business units to invest & grow.
  • Resolve to actually have meaningful dialoge with your Internal Audit department. I’ve rarely come across an auditor who is truly evil (they do exist, tho). Most want to Do The Right Thing™, but many lack the technical skillset to turn that desire into a reality. You should make it a goal in 2012 to have you and your Internal Audit department toe-tapping from the same risk dance card.
  • Resolve to join at least one cross-industry information sharing group. Even if it’s just kvetching at a local ISSA meeting, you should not underestimate the cathartic benefit of knowing you’re not alone. Joining or help to build a full-on entity like the ACSC, however, will even reap even larger dividends.
  • Resolve to understand the business model of each of your business units (if you have more than one) and find a way to get a handle on their pain points (the ones you or your IT department are causing). Go out on sales calls; shadow call centers; watch highly experienced and effective folks as they get their jobs done by working around IT & security barriers you’ve helped put in place. You’ll come back with business justifications for all sorts of things (like adaptive authentication or revamping your outdated identity & access management model)
  • Speaking of sitting… Resolve to spend three or more total business days at your IT Help Desk (great advice for non-security IT folk, too). You will first-hand observe the gaps in many of your processes (which you should then fix) and will also be able to put real faces & names to the pile of call statistics you ignore every month. I can also guarantee that you will then be spending a great deal of time revamping your incident response plan/procedures (you will see things you really won’t believe).
  • Speaking of statistics… Resolve to pick three meaningful things to start measuring and find a way to collect the data, get access to the data and publish the data (including sharing it to Internal Audit and getting it in front of senior management). A great place to start is the CIS Consensus Information Security Metrics. Your goal is to have at least one action item per month from this exercise (or pick different things to measure).
  • Resolve to kick the effectiveness of your security awareness program up a few notches. Create an internal “YouTube” service that shows real attacks from end-to-end. Make your messages personal by tying in social media awareness, safe browsing practices and patch management with messages of how to help folks keep their kids safe online or themselves safe as they do online banking. Make the learning experience engaging (just like you demand of your kids’ teachers).
  • Resolve to be the first organization of 2012 that has a sane password policy. (This one won’t be easy)
  • Resolve to expand beyond the mystical forumlae for CVSS & CWSS and create the foundation for a true risk-centric security program. If you are looking for help/guidance, this rogues’ gallery is a good place to start. WARNING: you will actually have to talk to business/finance people. (*shudder*)
  • Resolve to partner with just one development team and one Ops team and help get them rugged and visible.

Finally, resolve to do just one of the items on that list and you’ll be doing more good in 2012 than all of the prognosticators combined.

The 5 Dysfunctions Of A Team

The following excerpt is from The Five Dysfunctions of a Team: A Leadership Fable by Patrick Lencioni.

I wonder how many of you recognize traits like these on your own team(s), past or present. I can certainly point to these as being core reasons various teams I’ve been on or led have been ineffective and unsuccessful. The book seems like a good, short read, too.

  • Dysfunction One – Absence of Trust

    When team members do not trust one another, they are unwilling to be vulnerable within the team. It is impossible for a team to build a foundation for trust when team members are not genuinely open about their mistakes and weaknesses.

  • Dysfunction Two – Fear of Conflict

    Failure to build trust sets the stage for the second dysfunction. Teams without trust are unable to engage in passionate debate about ideas. Instead, they are guarded in their comments and resort to discussions that mask their true feelings.

  • Dysfunction Three – Lack of Commitment

    Teams that do not engage in healthy conflict will suffer from the third dysfunction. Because they do not openly surface their true opinions or engage in open debate, team members will rarely commit to team decisions, though they may feign agreement in order to avoid controversy or conflict.

  • Dysfunction Four – Avoidance of Accountability

    A lack of commitment creates an atmosphere where team members do not hold one another accountable. Because there is no commitment to a clear action plan, team members hesitate to hold one another accountable on actions and behaviors that are contrary to the good of the team.

  • Dysfunction Five – Inattention to Results

    The lack of accountability makes it possible for people to put their own needs above the team’s goals. Team members will focus on their own career goals or recognition for their departments to the detriment of the team.

A weakness in any one area can cause teamwork to deteriorate. The model is easy to understand, and yet can be difficult to practice because it requires high levels of discipline and persistence.
and persistence.