Transformer Models

Transformer-based models for detecting software vulnerabilities are limited by their token input size, potentially missing crucial data. This project introduces a slicing method that focuses on relevant program points to improve detection accuracy, yielding better performance metrics compared to traditional function-based approaches.

Introduced an explanation method for the transformer-based CodeBERT, which factors in the information flow across layers using Markov chains and integrated gradients for better insight into source code vulnerability predictions.