Technology Evaluation Study Plan

Read your chosen Technology Selection paper to learn more about the selected technology. Next, consider
what type of formal evaluation study could be used to learn more about this technology and how it is
likely to interact with people, processes, and technologies. Then, design a formal evaluation study which
could be used to obtain more information about one or more of the following:
o characteristics (features and capabilities) of the technology
o interactions among technologies, people, environments, and processes (use cases or scenarios)
o risks or vulnerabilities associated with adoption of this technology
o costs and benefits associated with adoption of this technology
Choose Your Evaluation Method
Your evaluation study design must use one of the following:
o Case Study
o Delphi Study (panel of subject matter experts)
o Quasi-Experiment (e.g. penetration testing or pilot testing in a controlled environment)
o Pilot Implementation (in a demonstration environment)
See the Technology Evaluation Methods module in the Week 2 conference for detailed descriptions of
each of these types of evaluation methods.
Design Your Study
Identify the specific questions that your formal evaluation study will address. These questions must be
security-focused and should address: threats, vulnerabilities, attacks, countermeasures, risks, risk
mitigations, etc. Your design should include a description of the specific security issues which will be
tested or security capabilities which will be evaluated. Use standard terminology when writing about
security issues (see the rubric).
Develop Your Evaluation Study Plan
Use your study design to prepare a high-level plan for your evaluation study. Your plan must include the
following:
o Introduction
description of the emerging technology and justification for including it in an evaluation study
o Research Question(s)
These must be security focused (i.e. focused on cybersecurity objectives such as confidentiality,
integrity, availability, etc.) and should address: threats, vulnerabilities, attacks, countermeasures, risks,
risk mitigations, etc.
Use “how” or “what” questions (writing good “why” questions is beyond the scope of this course).
Examples
What vulnerabilities exist that could be attacked to compromise confidentiality?
How could an attacker compromise availability?
For each research question, provide a brief description of a scenario or use case which could be used to
answer the question. Your description should be one paragraph (no longer).
o Methods
high level design of the study (focus upon the evaluation model and your research questions)
description of how the technology will be incorporated or used in the study (including specific security
issues which will be tested or security capabilities which will be evaluated)
notional system or network architecture diagram showing the pilot test environment (only if you are
doing a plan study
o Limitations or Special Considerations
any special considerations or security concerns which must be addressed (e.g. “clean room,” test data
sanitization, or isolation environment to prevent the pilot study from causing harm to operational systems)
o Timeline of Events (Notional)
A notional timeline (expressed in days or months after start date) for your stu