Software Testing

Prologue

  • "By the year 2000 all software will be delivered bug-free", unknown Software Engineering manager approximately 1990

  • By now we expected that software development would be flawless and that software testing would be unnecessary

  • Reality -- Software errors are a more serious problem today than ever before!

  • The most dramatic -- zero-days

  • A zero-day bug is one that exploits a previously-unknown vulnerability in a computer application

  • It is called a "zero-day" because the programmer has had no days to fix the flaw

  • Once a fix is available, it is no longer a "zero-day"

  • It is becoming increasingly typical that individuals or companies who discover zero-days sell them to the organization involved or to a government agency or to ....????

  • Zero-day attacks occur during the vulnerability window that exists between when vulnerability is first found and when software engineers develop and apply a fix

  • Malware writers often exploit zero-day vulnerabilities

  • Web browsers are a target because of their widespread distribution and use

  • Attackers can also send e-mail attachments, which exploit vulnerabilities in the application opening the attachment

  • Determining the size of the vulnerability window can be difficult, since attackers do not announce when the vulnerability is first discovered

  • Developers may not even know if a vulnerability is being exploited when they fix it

  • By one estimate, "hackers exploit security vulnerabilities in software for 10 months on average before details of the holes surface"

  • But, this window can be several years long

  • In 2008 Microsoft announced a vulnerability in the Internet Explorer, which affected some versions that were released in 2001

  • A software bug can be worth big money

  • There is a market for them because company data, medical records, social lives, governments, ... now live on computers

  • A lot of people are interested in that data

  • Bugs are often what they use to get at it

  • These vulnerabilities -- actually, the knowledge needed to exploit them -- are expensive, highly refined munitions that form the core of an extremely sophisticated weapons system

  • Washington Post analysis of the Edward Snowden leaks revealed an NSA budget that included $25.1 million for "covert purchases of software vulnerabilities"

  • NSA buys and uses zero-days

  • Nightmare scenario is an attack on public infrastructure

  • "Zero-day vulnerabilities, if you're able to identify one of them, can do serious harm," Mary Galligan, former Special Agent in charge of Cyber and Special Operations in the FBI's New York office

  • Such things as manufacturing processes, the electrical grid, water supply, etc. are vulnerable to zero-day exploits

  • Big software companies are increasingly finding it cost-effective to buy up their own bugs and fix them before anybody else can exploit them

  • In 2010 Google offered rewards for vulnerabilities in Chrome

  • Its total payouts have reached $3.3 million

  • Microsoft will pay up to $100,000 for a serious security flaw in Windows

  • Facebook has paid more than $1.5 million for its bugs

  • Coding practices and standards are getting better, but not fast enough

  • National Vulnerability Database currently lists over 60,000 vulnerabilities

  • In March the federal government notified 3000 U.S. companies that they had been hacked last year

  • The more we ask computers to do for us, the more important it is that they be secure

  • But the more computers have to do, the more complex their software has to be, and therefore the more bugs they have

  • This is a vicious cycle!

  • This is the environment we live in today and why you are taking a Software Testing course....