Sequential anomaly detection with labeling costs

In a number of data analytics domains, there is a need for detecting the situations when something outside of the normal conditions is happening. The goal of this project is to develop novel algorithms that learn to distinguish these normal and anomalous patterns through minimal interaction with a human user, while allowing complex data patterns such as time-series data. We further take into account a specific asymmetry of labeling costs that is inherent in the problem. Our proposed methods use prediction with expert advice techniques, adapting them to the structure of the aforementioned problem. We avoide statistical assumptions about the data generating mechanism, while allowing the use of domain knowledge through designing a set of experts used by our algorithms. Our algorithms enhance the effectiveness of Darkhorse Analytics’ anomaly detection system, which is currently based on manual definition of thresholds on specific key values, and also reduce the cost of manual analysis of the data sets by requiring less interaction with human users.

Faculty Supervisor:

Csaba Szepesvári, András György

Student:

Pooria Joulani

Partner:

Darkhorse Analytics

Discipline:

Computer science

Sector:

Information and communications technologies

University:

University of Alberta