Mastodon
Go back

Metricon: Verification versus Validation

Speaker: Jennifer Bayuk

Based on work for Stevens Institute of Technology.

How do professional systems engineers work?

History:

  1. Mainframe
  2. physical security (punch cards)
  3. cables to terminals
  4. network to workstations (some data moves there & on floppies) *spike in misuse & abuse
  5. modems and dedicated links to external providers/partners
  6. added midrange servers (including e-mail)
  7. added dial-back procedures to modem
  8. e-mail & other issues begat firewalls
  9. firewalls begat the “port 80” problem
  10. modems expanded to the remote access issue
  11. remote access issue begat multi-factor auth
  12. then an explosion of midrange begat more malware
  13. internal infestation from web sites & more e-mail
  14. added proxy servers
  15. made anti-virus ubiquitous
  16. kicked in SSL on web servers that now host critical biz apps
  17. (VPN sneaks in for vendors & remote access)
  18. more customers begat identity management
  19. increasing attacks begat IDS
  20. formalized “policies” in technical security enforcement devices
  21. now we have data & access everywhere, begets log management
  22. data loss begat disk encryption on servers & workstations
  23. increasingly common app vulns begat WAFs

Reference: Stevens Inst. “systems thinking”

Use systemogram to show what systems are supposed to do (very cool visualization for differing views of “security systems thinking”)

applied that systemogram model to a real world example of Steven’s school computer lab

Shows the “Vee Model” (her diagram is more thorough - GET THE PRESENTATION)

Advantages of this approach include:

Must advance and move beyond threat->countermeasure insidious cycle.

Traditional requirements process involves gathering functional requirements, interface definition and system-wide “ilities” - need to get it in before the interface level (high-level “black box”)

The major vulnerabilities are at the functional decompositional level

Many security vulns are introduced at the interface level as well

Unfortunately, it’s usually put at the system-wide level (as they do with availability ,etc)

What Do Security Requiremens Look Like Today?

V&V: Verification: did we build it right? Validation: was it built right? (akin to correctness & effectiveness)

There are more similarities than system architects really want to believe or understand.

Much of security metrics are really verification vs validation

Validation Criteria


Join the conversation:follow@hrbrmstr@rud.ison the fediverse to read and reply to this post from your own instance.

No comments yet.


Previous Post
For Everyone Who Thought I Was Just A Zombie
Next Post
Metricon: Measuring Metrics Programs (Why Aren't We?)