We are pleased to offer six in-person courses during the 2023 RAAINS Workshop, covering a wide variety of AI topics. These courses will be held at MIT Lincoln Laboratory on the afternoon of Monday, 13 November 2023, from 1:00-5:00PM. When registering for the RAAINS Workshop, please specify your course preference(s) and we will do our best to accommodate them based on space and time availability.
In addition, all RAAINS participants are highly encouraged to take our online course, Artificial Intelligence Foundations, prior to attending the RAAINS Workshop. Descriptions of the available courses can be found below.
Virtual Course
The Artificial Intelligence Foundations course provides valuable background for the in-person RAAINS courses and technical sessions. It is self-paced and available to all participants free of charge. The course contains nine different modules that introduce learners to various concepts in the world of AI, including data requirements and conditioning, deep learning, computer vision, computing and hardware requirements, and human-machine teaming. To access the course, click on the Register button at the top right to create an account and complete the enrollment process (email verification is required; please use an email address that is not likely to block messages from [email protected]). For any questions, please reach out to [email protected]
In-Person Courses
Early Critical Considerations of Ethical AI Systems
Instructors: Ms. Ngaire Underhill, Ms. Isabelle Hurley
Duration: 2 hours
Machine Learning (ML) and Artificial Intelligence (AI) systems are increasingly being applied to national security problems. Advanced, careful, and robust consideration of AI/ML design and implementation, however, is critical. These considerations can include: potential misuse of AI, AI system deprecation, AI design nuances, and the role of human machine teaming in the AI pipeline. Strategic consideration of these issues in the inception and design phase of an AI/ML system will save resources, time, and energy, as well as enable the creation of a more accurate and well-designed system. Disregarding these critical design questions can lead to mission failure as well as unintended and undesired moral, legal, and financial consequences. This course will walk participants through a guided example in which strategic questions regarding system design and implementation are identified, discussed, and evaluated. Course participants will subsequently be challenged to perform this exercise on a variety of potential AI/ML applications across different domains. Upon completion, participants will gain experience and knowledge in strategic thinking about the implementation and design of AI/ML systems.
Human-Machine Teaming for Systems Engineering
Instructors: Ms. Kimberlee Chang, Dr. Vincent Mancuso, Dr. Sarah McGuire
Duration: 3 hours
There is growing recognition of the importance of human machine teaming (HMT) to enable effective AI/ML technologies as they are incorporated into increasingly critical systems. HMT bridges the gap between humans and AI to enable human and system partners to efficiently communicate, coordinate, and adapt in complex scenarios. In this course, we will provide an overview of human–machine teaming (HMT) as it relates to AI development and testing. We will discuss the concept of HMT and how to select the right level of autonomy for the intended application. We will also introduce the concept of human-AI teaming testbeds and the importance of applying multidimensional metrics that allow performance benchmarking and inform further development. Upon completion, participants will have an understanding of key HMT considerations when designing AI systems and methods that can be utilized during the design process.
Artificial Intelligence for Cyber
Instructors: Mr. Jensen Dempsey, Dr. Timothy Reid, Dr. William Stephenson, Dr. Ashley Suh
Duration: 2 hours
Cyber security professionals are overburdened with repetitive and time-consuming tasks. Artificial Intelligence (AI) systems show promise at automating and speeding up repetitive tasks in many industries. However, there are unique challenges in applying AI to cyber problems. These challenges can include combating adversarial attacks, ensuring data integrity, enabling real-time responsiveness, and facilitating human-machine collaboration while promoting trust in the AI. This course will begin with an overview of the potential capabilities and challenges of using AI for the cyber domain. We will then provide insights into a selection of current state-of-the-art AI techniques – such as Reinforcement Learning (RL), Large Language Models (LLMs), and Explainable AI (XAI) – that developers, evaluators, and cyber operators can use to augment current practices in the cyber domain. Finally, we will solidify these concepts with a case study from the National Security regime. Upon completion, participants will gain knowledge about how state-of-the-art AI technologies are applied to cyber security problems and awareness of the challenges associated with applying such AI technologies.
Counter Influence Operations Using AI and Causal Inference, with Ethical Considerations
Instructors: Dr. Edward Kao, Dr. Erika Mackin, Ms. Amna Greaves
Duration: 2 hours
Effective monitoring of and response to disinformation campaigns waged by nation states require accurate influence assessments and automated analytics on the vast amount of (social) media data. Lincoln Laboratory has developed a data-driven, human-machine teaming framework to address this technology need. This framework leverages: (1) the recent advancement of AI models to rapidly discover influence operation narratives, and (2) a novel extension of causal inference on social networks to quantify the influence of actors and pathways between communities. These analytical products can inform mission planners of emerging threat narratives and prominent actors, as well as effective channels for counter messaging. Additionally, as influence operations impact multiple facets of our society, an approach to conduct responsible and ethical technology development is proposed. This course will present and demonstrate: (1) novel applications of Natural Language Processing techniques with a focus on the Transformer models to detect and summarize narrative content within a large text corpus, (2) the technical foundation of network causal inference and its applications to provide more accurate measures of effectiveness (MOEs) in influence campaigns, and (3) a risk innovation framework for responsible and ethical research. Upon completion, participants will gain technical understanding and experience real-world applications of key technologies for counter influence operations and their ethical considerations.
A Practical Guide to Applied Generative AI
Instructors: Dr. Pooya Khorrami, Mr. Evan Young, Dr. Swaroop Vattam, Dr. Olga Simek, Dr. Miriam Cha, Mr. Keegan Quigley, Ms. Trang Nguyen
Duration: 4 hours
Generative AI is the key to unlocking the full potential of artificial intelligence, allowing machines to create new and original content. Within a short span, Generative AI has changed the technology landscape and promises to unveil transformational use cases in commercial and national security arenas. This course will introduce participants to the basics of Generative AI, including various types of models and how they work across different modalities. It will begin by introducing participants to Generative AI for image and video generation, specifically, the types of models/architectures used (e.g., Generative Adversarial Networks / GANs, Diffusion models), how the models are applied, and what areas they have impacted. Participants will also be introduced to Large Language Models (LLMs), covering topics such as the history of LLMs, different types of LLMs, how LLMs work in practice, and the risks and challenges posed by LLMs. With a foundation in visual and textual generative modeling, participants will also be introduced to Cross-modal Generative AI with a look at the range of input and output modalities, common techniques for cross-modal generation, and a deeper dive into both Image-to-Image and Image Captioning models. Upon completion of this course, participants will gain a deeper technical understanding of Generative AI, discover applications in diverse domains, and become familiar with challenges and risks posed by this transformative technology.
Test and Evaluation of AI Systems
Instructors: Dr. Michael Yee, Dr. Lei Hamilton
Duration: 4 hours
Artificial Intelligence (AI) is powering advances in many different domains such as computer vision, natural language, and autonomous systems. Although AI systems (using deep neural networks) often surpass human-level performance, they have been found to be vulnerable to both naturally arising challenges and malicious attacks. For example, they can fail when the environment changes (e.g., when new objects are sensed under different lighting conditions) or when inputs are corrupted by noise. They are also susceptible to small input modifications that drastically change system output despite being imperceptible to humans (known as “adversarial examples”). Moreover, the confidence scores reported by AI systems with their predictions can be poorly calibrated, leading to issues with user trust. This course will provide an overview of core concepts in AI test and evaluation (T&E), including what can go wrong when applying trained models to new input data, important T&E dimensions to consider, and tools and strategies for evaluation. Participants will work individually or in teams on several hands-on exercises that apply concepts learned in the course using Jupyter notebooks. Group discussions will facilitate sharing of experiences and lessons learned. Upon completion, participants will be able to explain the importance of evaluating AI systems for vulnerabilities, describe common types of vulnerabilities and various methods for identifying them, and use several tools for evaluating AI models. This understanding is critical for building robust AI systems that can be trusted to perform as expected upon deployment.