Amazon Bedrock Guardrails overview

- [Instructor] Let's take a look at Bedrock Guardrail architecture. What's interesting about it is it's a parallel policy evaluation engine with sub-100 millisecond latency. This means that we have content filters, we have PII detection, we have topic controls, and it's all done in a way that makes it scalable. That's really the key consideration for the Guardrails. It allows you to do real time content classification. For example, you could do no, low, medium, high, right? And you could put these into different buckets, and if it's a high confidence, maybe that's temporarily removed from some operation. If it's none, then that can instantly go through. In terms of integration with agents, which is essentially code that can run serverless, or knowledge base, which is essentially a RAG-based solution, it's a great way to also have these additional security measures in place. And if you look at support for text and image, it gives you different options for different modalities and it's working across all the foundation models that are on the platform. Now, input validation is done through Dynamic UUID, and you can look at this via the scoping that you apply on the API request, and you can look at doing things like pre-inference validation, also doing parallel policy evaluation with this 20,000 character limit per request, and also you can have configurable blocked messaging with innovation tracing. So this allows you to do things in a more real-time manner. If you look at output safety controls, this allows you to do things in a synchronous manner. So what this means is that you're able to look at all the things that are happening, and potentially have a response limit of 5,000 characters per chunk and look at metrics like processing latency or chunk size or intervention count. And this 5,000 character limit is probably the biggest thing to be aware of when you're dealing with this. In terms of the confidence threshold mapping, this is another component to think about. If it's high, if it's medium, if it's low, again, it could go to some bucket and you could have different operations that are acted upon depending on which bucket it goes into. And you can also do PII masking with regex as well in terms of safety control. So can you imagine how this would take place? Is that you could create a control that never lets PII ever come out of the system. In terms of content filtering components, you could have filter types, you know, looking at things like insults or violence, et cetera, depending on the kinds of outputs that they go to, and you also have image modality support. Beyond just movies, you could have PNG, JPEG, and there's a four megabyte limit. In terms of conference intervals as well, these can be mapped to CloudWatch metrics, and then in terms of CLI, you could also do this dynamically by using a AWS CLI command, like AWS, space, bedrock runtime. Let's go ahead and apply the Guardrail ID to it. In terms of security boundary implementation, this is another component to think about, is dealing with these boundaries including encryption or key policy, et cetera, and the KMS key policy allows you to deal with data and decrypt it and encrypt it and have different grants, so this could be at that layer as well. In terms of cross-account access, you also could integrate with cross-account access and you could also have security enforcements as well by using these Guardrails. In terms of edge case, these are other patterns to consider in terms of solving problems, like do you want to do a malformed input recovery or a timeout or potentially even a retry strategy, like exponential backoff. These are all things that you can measure by looking at the different metrics. In terms of performance monitoring, this is something that happens with any production system, but you could use CloudWatch metrics and look at things like latency interventions, coverage, and some of the key metrics, as I mentioned, would be latency, maybe the total count of interventions and even the policy evaluation time, and then the dimensions would be GuardrailID, so like what is the specific kind of Guardrail that you're working with? What is the PolicyType, what's the ModelId? So you could drill down into what's happening at a Guardrail perspective or what's happening with a Guardrail with a particular model. Maybe one model is performing better than another model, and you can also set up alarms as well. Now, if we look at digital rights control, this is also a pretty sensitive topic, is what can be done to control digital rights? And you can see that there's things like consent tracking as well, like did someone agree to have their content used for a particular purpose, maybe for training, or did they not? Also, we have PII masking control as well. So the ability to anonymize data like names or email or ID, and then the audit trail. So I think this is maybe one of the more important features of digital rights control is like looking at all of the different operations from an API level, and then making sure that that applies with maybe the legal policy of an organization. Now, in terms of responsible AI, this is something that is also really important to think about, is, how can organizations think about responsible AI? You could look at essentially biometric data protection, making sure that certain key biometric data, images or files are not stored. Also, algorithmic harm prevention as well. So are you preventing viral untruths? Are these some things that you could do at a policy level to stop? And then in terms of audit patterning as well, so can you actually audit to make sure that certain things are not happening in your system? In terms of monitoring and compliance, this is the last thing I'll discuss about this, is that you can look at it from a dashboard perspective and you could create a dashboard called GuardrailMetrics and you could look at different things like interventions or latency or policy coverage types, and this could be automatically reported and potentially data governance components of the company can take a look at that monitoring and compliance. So I think in terms of the final end goal of something like a Guardrail is to actually have comprehensive controls and granularity into what's happening when you're using AI systems, and can you actually make a defense of them? And as more and more regulation is happening in the AI space, I think the monitoring and compliance are going to be one of the more interesting areas to take a look at, and so it is important to use tools like this Guardrail ecosystem.