The 11 LangSmith security checks Black Cat runs
Black Cat SSPM evaluates 11 security policies against your LangSmith configuration on every scan, classifies each finding by risk, and provides remediation steps. Below is the full list, grouped by category.
autonomy
mediumLangSmith Agent Unmonitored
Add human feedback and monitoring
lowLangSmith Agent High Run Volume
Review high-volume autonomous operation
mediumLangSmith Agent No Evaluation
Add evaluation dataset for quality monitoring
highLangSmith Agent High Risk Unblocked
Block or restrict high-risk LangSmith agents with scores above 70
data access
lowLangSmith Agent Dataset Access
Review dataset contents for sensitive data
mediumLangSmith Agent Retriever Tools
Review vector store and retriever tool access
lifecycle
lowLangSmith Agent Stale
Review stale agent projects
mediumLangSmith Agent High Error Rate
Investigate and fix high error rate
highLangSmith Agent Orphaned Owner
Reassign the LangSmith agent to an active owner
permission
mediumLangSmith Agent High Tool Diversity
Review and reduce the number of tools used by the agent
highLangSmith Agent External API Tools
Review external API tool access