In the first half of this column on ethical artificial intelligence usage, we looked at a couple of different models: The EU AI Act (Regulatory/Mandatory) and ISO/IEC 42001 (International Standard).
Let’s spend this column looking in-depth at a third model: the NIST AI Risk Management Framework (Voluntary/U.S. Standard)
This is put together by NIST (National Institute of Standards and Technology), is 48 pages and can be found at NIST AI Framework.
What I like about the NIST framework is that it treats the problem as a “socio-technical” system. This means it rejects the idea that you can secure AI just by changing software or firmware (the technical part).
Instead, it requires you to manage how that code interacts with the messy, unpredictable world of human behavior (the social part). An example of that would be if a system detects something correctly but displays it in such a way that a human may misinterpret it or jump to an inaccurate conclusion.
Four Key Functions of the NIST AI Framework
In this model, there are four key functions that have to be done for a project. Since projects always vary in scope and technology, this is easily adaptable to just about any situation.
- GOVERN (Culture): Just like any type of organization, you have to identify and establish this policy first. This will target things like who is accountable, what the policies or rules specifically are, and how you maintain diversity or balance in the governance. These are very typical controls in any business or organization. If this isn’t done correctly, the other three parts will always fail, so start here.
- MAP (Context): In this phase, you have to define exactly what the AI is going to be doing or how it is used, and the risk and impact associated with it. This is a qualitative exercise, so as an example, if the AI is facial recognition, using it to unlock a computer or phone is fairly low risk compared to a facial recognition system used to arrest suspects. Once you determine the “what,” then you map it to the “what could go wrong” part, considering things like financial damage, reputation and loss of sales, physical harm to anyone, a social rights violation, etc. There is no right or wrong; it’s a list of “what ifs” that need to be taken into consideration.
- MEASURE (Testing): This moves from qualitative to quantitative. Here you have to measure the accuracy and effectiveness of the solution. As an example, if you are deploying a weapons detection system, you must measure how many times it fails or works. The way to determine what to measure for is by using the MAP to see if the “what ifs” are a problem or not. A very simple example would be if in the MAP you determined a “what if” is: “How does cold weather affect the detection of weapons? So if everyone is wearing heavy clothes, does it still work?”
Many companies approach this like a red team exercise to try to break it or have it make mistakes that were in the MAP. The more specific you can make it, the better, especially when you consider how AI has been trained.
- MANAGE (Action): This is where you have to make the decisions about the application of AI and how it affects all the stakeholders. In this part, you have to take the results of the Measure section and determine the priority of the deficiency and if any of them are egregious enough to stop a project or take a system offline.
Probably the most common action is how to mitigate the deficiency. In order to mitigate the deficiency, you apply controls to it such as an additional filter or system, or more commonly putting a human or multiple humans in the loop so mistakes are reduced to a level where the risk is acceptable.
Diving Deeper into NIST
Trustworthiness Characteristics: NIST defines trustworthy not by one characteristic but by seven distinct ones: valid and reliable; safe, secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair.
These are all trade-offs and you can’t maximize all seven at once. As an example, to make a model Secure (hard to hack), you might make it less Explainable (hiding the inner workings). The Govern part will determine which tradeoff is acceptable or not and then how to mitigate or manage it.
Practical Tools: NIST understands that many of these are somewhat abstract and not easy to understand let alone do, so NIST added two helpers to get you through the process:
- The Playbook: This is a comprehensive list of things you can refer to to help you with the MAP. So as an example, instead of just coming up with ideas on how to map a risk, some Playbook suggestions are “interview several subject matter experts or review the list of incidents over the last five years.” Much like a food menu, you don’t have to eat everything on it.
- Profiles: NIST releases specific profiles for specific technologies. This is found in the Generative AI Profile (NIST AI 600-1) which adds 12 specific risks to AI and its implementation. This covers everything from hallucinations (where it makes stuff up) to data poisoning. These risks can and should be used in the process to help Measure and Map.
AI is a very complicated and confusing area for most. There is a lot of smoke and mirrors and a lot of new startups all trying to get in on the action, but make no mistake, this is our future in the physical security and monitoring area.
My goal was not to make you all-knowing on how to evaluate, manage, and execute an AI solution, but to let you know there are several ways, guides, and standards on how to properly do it.
The more we understand about AI the better off we are as an industry will be. AI can be an amazing tool when used by the right hands and used appropriately. Used incorrectly really bad things can happen, so please take the challenge and dig into AI.
If this is as foreign as a new language for you, start with something as easy as using Google’s Gemini or ChatGPT to do internet searches for you or have it summarize a pile of spreadsheets for you. Start to use it to get yourself accustomed to how it works, or in some cases, how it doesn’t work.
It can do great things and the more you know the better.
