Machine Learning Approach to Crime Prediction and Identification of Hotspots
Abstract— Crime prediction is becoming a fast-growing area of research in the field of data science due to an increase in the availability of crime data and the adoption of data-driven approaches by police departments all around the world. This practicum aims to find spatial and temporal hotspots along with predicting the type of crime using association rule mining and machine learning classification models by using Denver crime data along with GDELT data which is a database for news events from all over the world. The model is able to identify criminal hotspots and provides a basic structure for high-level crime prediction models. The results from this practicum can help the police force improve their response time to incidents, allows them to be better prepared while handling situations and help recognize areas of risk.
Keywords— crime prediction, hotspots, association rule mining, crime classification
I. Introduction
Crime is a persistent problem all over the world that affects the quality of life and economic growth [1,4,5]. Crime negatively affects the image of the associated localities resulting in residents relocating and further discourages business growth. It also burdens the taxpayers due to the increased need for the police, judiciary and prison systems [5].?
By studying how world economies and societies are structured, it is reasonable to assume that crime will persist and most likely increase in the near future. Another observation is that crime occurs more frequently in urban areas rather than rural. This is due to higher population density and larger economic disparity in urban localities. Crime statistics also have a correlation with factors such as unemployment and income [2].?This makes it essential to study incidents, identify solutions to reduce and prevent crime in urban surroundings.
Crime has been analyzed from a behavioural, psychological and sociological perspective since the beginning of the 20th century. The studies have focused on the relation of criminal behaviour with their environment and actions [1]. Based on data available, credible theories such as ‘centrography’, ‘journey to crime’ and ‘routine activity theory’ have also been established [8]. But now, with the growth of data and information available over the last decade, there are new opportunities and possibilities to advance the fields of crime analysis [3,14].?Nowadays, information and data are critical tools in law enforcement [10]. As computer-based systems are increasingly used in tracking crimes, data analysts have started helping uniformed officers in solving them [13].?????
Data-driven approaches to crime prevention in recent years has made it easier to observe large-scale trends and patterns. But specific crime patterns are still not clearly visible by these approaches. Thus, crime analytics is still fundamentally a difficult manual job for analysts [3]. Another problem to take into consideration is that the patterns and trends observed in small geographic clusters may not align with large-scale trends or assumptions such as the ‘broken windows' theory [1]. Due to the popularity of the idea of solving crime based on the criminal and his motives being central to police work over decades, methods based on geographical profiling have not been expanded on or utilized frequently by law enforcement [1,8]. Even data-driven approaches initially just compared crimes being committed to past crimes to observe patterns [3]. But now with the advent of machine learning algorithms, crime and its patterns are more easily observable through multiple metrics [4].
Machine learning is the science of automating the learning process for a model or a system without explicitly programming it. The concept of machine learning has been applied to the concepts of self-driving cars, web searches and pattern recognition [4]. Various machine learning models can now be used for criminal analysis and decision-making [10]. This allows for the identification of factors influencing crime. These factors can then be correlated with crimes being committed in either an area of a city or a whole country. Using machine learning and data mining methods also saves time in an otherwise laborious process. ?Time is a crucial factor in law enforcement. With the support of a large amount of data available and machine learning algorithms, the concept of predictive policing in law enforcement is starting to take shape [3].
This practicum attempts to obtain a model which can identify crime hotspots and predict crime to a level of accuracy that can be expanded upon to obtain a high-level prediction model.
Section 2 of this report covers the literature. Section 3 expands on the methodology of the project and section 4 describes the results. Section 5 covers the conclusion and future work of the practicum.
II. Related Work
A. Datasets
The exploration of crime analysis has been carried out from quite a few viewpoints using a few different types of datasets. Some researchers used a London-based crime dataset along with a dataset of geo-localized information such as transportation, households and London borough profiles as well as a behavioural dataset computed from mobile network activity in the city [1]. Some papers used a crime dataset based solely on homicides in Brazilian cities along with 10 urban indicators such as child labour, elderly population and illiteracy [2]. Another approach went with crime in the city of Vancouver along with neighbourhood data [4].
A few papers used the Denver crime dataset to identify types of crime occurring in different areas and also to identify patterns and correlations between various factors. These are then used to either predict future crime or to identify hotspots where particular types of crime are occurring. This dataset records the reported offences in the city of Denver and contains information regarding the location, time and type of crime [17,18,19].
A very popular dataset that has been used in crime analysis using machine learning and data mining approaches is the ‘Crime and Communities’ dataset attained from the UCI machine learning repository. This dataset is prepared from the socio-economic data of the 1990 census, law enforcement data of the 1990 US LEMAS survey and the 1995 FBI UCR crime data. It has a total of 128 attributes with each instance having data from a different state [5,9,10,11].?
?????One paper used crime data in Taiwan along with spatial-temporal data to split city maps into grids, as they were applying a “broken windows” approach to observe crime patterns [7]. One paper used metrics such as victim characteristics such as gender, age and jobs as well as characteristics of the crime site such as location, ownership etc. in order to create geographical profiles for investigating crime [8]. Another paper used spatial-temporal features by applying Google Places API to theft data for Taoyuan city, Taiwan for grid-based crime prediction and to observe crime displacement in the city [12]. Some researchers worked with the police department of a Northeastern US city to collect data and generate aggregated data for different types of crimes along with spatial-temporal information related to the crime data. They applied different data mining and ensemble learning techniques on the same [14,15].
B. Crime Prediction and Hotspots
Previous research in this field has been generally tried to accomplish one of the following objectives:
Some of the research has been based on a single type of crime or have focused their research on a type of criminal such as serial offenders. Different types of machine learning and data mining algorithms have been used in criminal analysis. Both supervised and unsupervised learning methods have been used but there is clearly more research in supervised algorithms. This is obvious as most crime data available is historic data which gives us clear outputs and thus supervised algorithms have been effective in crime prediction. The models attempting to determine crime patterns require unsupervised learning models to be utilized. There is extensive research in different parts of the field but there is still room for improvement.
One paper used mobile network activity, demographic data and criminal records to identify crime hotspots in the city of London. They divided the city of London into grids and then classified each cell in the grid as either a crime hotspot or not. They used algorithms such as logistic regression, support vector machines, neural networks, decision trees and implementation of an ensemble of tree classifiers. They compared the algorithms using metrics such as accuracy, F1 score and AUC score. They also compared performance by comparing features of the combined dataset to the individual datasets. The combined dataset was obtained by using the feature ranking and feature subset selection approach [1].
Another paper focused on finding criminal hotspots in terms of both space and time. They conducted a statistical analysis on two crime datasets from Denver and Los Angeles. They then used the Apriori algorithm to find frequent hotspot patterns. The patterns were in the form of the location, day and time. Then it uses Decision Tree and Na?ve Bayes classifiers to predict potential crime types. It also combined Denver’s dataset with demographics data to determine factors for crime [2].
One paper attempted to used machine learning algorithms to identify specific criminal patterns to order to recognize a person or group which continuously commit crimes. The model attempts to recognize the modus operandi of the criminal by going through the database to recognize growing patterns in the crimes. They devised an algorithm for detecting patterns in crime by using coefficients to define an incidence of crime. They used a set of coefficients to capture the common characteristics of all patterns, patterns specific coefficients which grow to capture the M.O. of a crime and dynamism in the patterns is captured by a similarity coefficient [3].
?????Another paper attempted to create a prediction model which can accurately predict crime by using 2 approaches. The first approach defines all variables as binary and the second approach defines all variables as numeric. They used k-nearest neighbours and boosted decision trees to predict crime in the city of Vancouver. The boosted decision tree algorithm AdaBoost was used which combines several weak learners to obtain a classifier that is much stronger [4].
One paper used different classification algorithms to compare and contrast the approaches to crime prediction in order to determine the best approach for their model. They used variables such as the state, the median income, education and employment statistics along with criminal statistics to determine the type of machine learning algorithm model that can be used to predict crime [5].
Another paper used a data-driven approach based on the “broken windows” theory which states that failure to respond to low-level criminal activity in a location will lead to more serious crime. They designed a model which predicts the incidence of drug-related crime in a given month based on crimes such as theft, assault, intimidation occurring in the previous month. They determined features by using types of crime and spatial-temporal patterns for each grid on the map, set drug crime in the next month as the dependent variable and ran deep learning and random forest algorithms to predict crime hotspots [7].??
One paper uses crime theories and geographical profiling to identify the next possible location of offences in a serial crime. They use decay functions to create the geographical profile for each factor and use probability distribution functions to see which site will be the location of the next crime. Then they combine all the geographical profiles by weighing the effect functions for each factor which is adaptively adjusted using Bayesian learning [8].?
Researchers also use regression algorithms on two different datasets for a comparative study of violent crime patterns. They only observe crimes such as murder, rape, robbery and aggravated assault and trained data on the local level to test it on national statistics. They used regression algorithms such as additive regression and decision stump in order to compare violent crime patterns in two datasets, one from the Crime and Communities dataset and one from Mississippi’s crime statistics. An interesting observation in their approach was the use of additive regression, which enhances the performance of a regression-based classifier by fitting the model using the predictions of the previous iteration [9].??
One paper used compared classification algorithms on crime datasets to identify approaches that work best for crime prediction. They also determined rules for classification using the model that worked best and also predicted attributes that contributed most to the crime that occurred [10]. Another paper similarly attempted to predict the type of crime occurring in a location using the state in which the crime occurred among the features for classification. It used socio-economic and spatial data as the features and compared the Na?ve-Bayes and Decision Tree approaches to predict the type of crime occurring [11].
One paper used a grid-based prediction model to establish a range of spatial-temporal features which were used to predict crime. They attempted to use machine learning in order to reinforce traditional policing methods for improving crime prevention by designing two types of models, spatial-temporal and empirical. They combined features from both to obtain the final model which they applied in a grid of Taoyuan city on a map [12].
?One research used clustering algorithms to detect crime patterns and also used semi-supervised learning for knowledge discovery and improved accuracy for predicting the patterns. They used k-means clustering to predict crime hotspots in a North-eastern city in the US. The features used in the analysis were determined by interacting with domain experts and also by running an attribute importance algorithm. Different types of crime had different sets of attributes becoming important and future crime patterns were detected by using observations given by a detective on the small clusters formed by the model [13]. ?
One research team initially use aggregated data counts and additional features in an ensemble of data mining techniques to perform crime forecasting. Then they use classification methods to determine the best models for predicting crime hotspots along with observing identifying emerging hotspots. They also identify the approach which provides the most stable outcomes [14].
?Another paper also attempted to find crime hotspots but by using a combination of spatial data mining methods such as point mapping, kernel density estimation (KDE) and Spatial and Temporal Analysis of Crime (STAC). The method they use actually attempts to also capture the difference in patterns of classes in the dataset, providing additional variables used in the forecasting [16].?
One research team used non-linear regression models such as random forest regressors on homicide crime data along with urban metrics from a city in Brazil in order to predict crime and used the same regressors to rank the urban indicators in order of influence over crime in the city [19].
III. Methodology
The objective of this project is to ascertain crime hotspots and also to predict the type of crime occurring in the city and county of Denver by using variables such as the location (neighbourhood), date and time and the tone of the news cycle on that particular day. Thus, the approach here is to use these variables to first identify frequent patterns by using association rule mining which will determine the hotspots of criminal activity and then we use these variables in a classification model to try and predict the type of crime occurring at a particular instance of crime.
In this section, we discuss what datasets are being used in this practicum how they have been prepared, how the data has been analyzed and observed to identify crucial information and how the models have been built.
A. Datasets
Two datasets have been used in this project, namely the Denver crime dataset and the GDELT dataset. The Denver crime dataset is the main dataset that is used in the modelling while data from the GDELT dataset is integrated in order to provide more information about the news environment in Denver. ?
Denver Crime Dataset-
The Denver crime dataset includes all incidents of criminal offences reported to the police in the city and county of Denver in the past five years. The data ranges from the 2nd of January 2014 to the 15th of November 2019. The data format is based on the National Incident-Based Reporting System (NIBRS). This contains information such as first occurrence date, type of crime, geographical location of the crime and the neighbourhood in which the crime was committed. The dataset has 19 columns and 528587 instances [20]. The dataset has been taken from Kaggle, which updates the data from the Denver open data catalogue.
GDELT Dataset-
The basic idea behind GDELT is to provide a global database of society. GDELT monitors the broadcast, print and web news from nearly every corner of the world, identifies the parties involved, locations, emotions and sources, creating a free and open platform for news and events for the entire world [21].
In this project, the GDELT data helps gauge the news environment for the city of Denver. This allows us to understand what type of incidents were occurring in Denver around the time a particular instance of crime took place in the city. It also gives an idea of the mindset of the people in the city at the time.
The GDELT dataset has been obtained using a data wrangler in R as the GDELT platform provides large amounts of data but only with the date as a filter. The final GDELT file for Denver has more than 2 million rows and 58 columns [22].
B.?Data Preprocessing
The main steps in preparation of data for modelling on the two datasets were:
Data Cleaning-
To obtain the GDELT data related to Denver in the time period required, a data wrangler written in R is used which passes through GDELT files for each date, extracts rows related to Denver and exports them to a smaller set of CSV files. It then combines the filtered set of CSV files to obtain one file fulfilling the requirements of the filter in place. In the GDELT dataset, there were no Nan, missing or inconsistent values in the dates, action code, number of articles or average tone for any event.
There were a few Nan values in some of the latitude and longitude locational values of incidents in the Denver dataset. As these variables aren’t directly used the final dataset, these incidents do not need to be dropped from the dataset. There were also some incidents that took place well beyond Denver’s boundary, so they were removed from the dataset. There were no other inconsistencies in the data for the attributes to be used.
Data Reduction-
In both the datasets, we selected attributes that were of use to us and removed the rest. In the Denver dataset, we retained only three columns to use for the modelling. From the GDELT dataset, we retained three columns which were then transformed and integrated into the Denver crime dataset.
Another way we reduced the data was by using the ‘IS_CRIME’ column to remove the traffic accidents in the datasets as traffic accidents were reported as incidents in the dataset.
Data Transformation-
The data transformation on the Denver dataset was to break down the ‘FIRST_OCCURRENCE_DATE’ data into features such as year, month, day, day of the week and hour. We also minimized the ‘OFFENSE_CATEGORY_ID’ values from 14 different values to 7. Crimes such as aggravated assault, murder and arson which risk fatal harm are categorized as dangerous crimes, incidents of some form of theft such as larceny or robbery were combined into one category and vehicle-related crimes were combined into one category. The ‘HOUR’ feature and the ‘TONE’ feature was categorized into 6 intervals for the association rule mining part of the modelling.
In the GDELT dataset, we multiplied the number of articles and the average tone for each event, then we calculated the average tone for events on a particular day, which became our variable representing the news cycle in the city of Denver on that day. ??
Data Integration-
We integrated the GDELT and Denver datasets using the date as the key to obtaining the final dataset that was used for the modelling. The date attribute that was used as the key was present in the GDELT dataset as ‘SQLDATE’ so we converted the ‘FIRST_OCCURRENCE_DATE’ by using DateTime functions to obtain a variable in the form that was present in the GDELT dataset. ???? ?
C. Data Analysis
By conducting an initial statistical analysis of the datasets, we tried to understand the data and realize the features that shall be used in the modelling. Data has been visualized to better understand the data and each of them covers how criminal incidences were spread according to each aspect of a variable.
Figure 1 shows how crime is distributed in the city of Denver in terms of the different types of crimes that are occurring in the city. As we can clearly see the most frequent type of crime occurring in Denver (except for ‘all-other-crimes) are public disorder, larceny and thefts from motor vehicles. Crimes such as robbery, arson and murder are the least frequent types of crime in the city.
Figure 2 shows the hourly distribution of crime in Denver. Crime is significantly less in late night to early morning and peaks during the early evening from 4 P.M. to 6 P.M. and late night at 10 P.M.
Figure 3 shows the overall trend of crime throughout the time period of the data. There are a few peaks and troughs and the overall mean is around 185 cases per day in the city of Denver.?
Figure 4 shows neighbourhoods with the highest number of cases in the city. Neighbourhoods such as Five Points are clearly the most dangerous with well over 20,000 cases recorded in the neighbourhood, with at least 5000 more cases than the second most dangerous neighbourhood which is CBD.
领英推荐
D. Data Modeling
The final dataset was modelled in two different ways. In the first model, the ‘TONE’ and ‘HOUR’ values were converted to categorical intervals in order to execute the FP-growth algorithm which gives the most frequent patterns in the data. This helps identify hotspots in terms of location (neighbourhood), time, day and tone of the news cycle. In the second model, the ‘NEIGHBORHOOD_ID’ variable is converted to dummy variables so that classifiers such as logistic regression, random forest and decision trees can be used in order to predict the type of crime occurring in a particular incident.
FP-Growth Algorithm-
FP-growth is currently one of the fastest approaches for association rule mining. The basic logic behind the FP-growth is that it starts eliminating items from transactions that do not fulfil the minimum support requirements individually. It then starts removing the least frequent item of the remaining items from the transaction set. This process is repeated with the second least frequent item with minimum support to obtain a reduced dataset. This process is known as the recursive elimination method which gives us the final frequent itemset based on the requirements [23].
This practicum uses the FP-growth algorithm to identify hotspots based on the location, time, day and tonality in the city of Denver. We used the mlxtend library in Python to implement the algorithm [24]. This association rule mining method allowed us to come up with a list of crime hotspots along with related time and news events features. Multiple experiments were conducted with different support values in order to find the optimal choice. The final value chosen was 0.0005, which would be around 187 absolute frequencies i.e. incidents of crime occurring at that particular instance.
Classification Algorithms-
Different types of classification algorithms such as Logistic Regression, Decision Tree, Random Forest, Support Vector Machines and Neural Networks have been used in order to identify the method that works best in terms of predicting the type of crime occurring in a particular incident recorded.
The classification models are applied on two datasets, one which contains the incidents that have been categorized as ‘other-crimes’ and one in which incidents classified as ‘other-crimes’ have been removed. This has been done to observe how the metrics of the models change when it attempts to classify crimes that have been more specifically classified.
The classification models have all been constructed using the Scikit-Learn, an open-source library that provides data mining and machine learning tools for Python.
·??????Logistic Regression: Logistic regression, in its basic form uses a logistic function to classify a binary variable. It can also be used to classify multinomial variables. Here, the solver we have used for logistic regression is the ‘lbfgs’ solver because it can handle multinomial dependent variables better than other solvers [25].
·??????Decision Tree: Decision trees classifiers attempt to create a model that predicts the target value using simple decision rules that it obtains from parsing the data features. The decision tree here uses the ‘gini’ criterion, which forms the decision tree on the basis of the Gini Impurity value for each node. Gini impurity measures the probability of incorrect classification of an instance of the random variable [26].
·??????Random Forest: Random forest classifier is a meta estimator which uses sub-samples of data to fit a number of trees and estimates the value by averaging the results. We have used 150 estimators (sub-trees) in order to classify the incidents in this case [27].
·??????Support Vector Machine: Support vector machines are supervised machine learning algorithms that classify data points by forming a hyperplane that best distinguishes the data. Here, the kernel being used is ‘poly’ which creates polynomial hyperplanes for the decision boundary [28].
·??????Neural Networks: Neural network is a supervised algorithm that learns a function by passing the data through multiple layers of nodes and modifying the weights on those layers through correction by error over multiple iterations of the training data. Here we use the ‘relu’ activation and the ‘adam’ solver [29].
IV. Results and Evaluation
In this section, we evaluate and summarize key results that we obtained from the association rule mining and classification models.
A. Frequent Crime Hotspots
The first goal of this project was to find crime hotspots in the city of Denver. We have extracted the most frequent patterns which gives us a reasonable idea of the spatial and temporal conditions in which crime occurs more frequently. We obtained 43 values, 10 of which are in Table 4.
The most frequent pattern observed is that a criminal incident takes place in Five points, late in the evening on Friday on a negative newsday. Five points is clearly the most dangerous neighbourhood looking at the itemsets with Union Station, Stapleton and CBD the next most dangerous areas. An interesting observation is that after midnight on a Sunday is a frequent itemset in both Union Station and Five Points, even though the support value of the individual items compared is comparatively low.
B. Predicting Type of Crime
The second objective of the project is to predict the type of crime occurring at a particular instance. The accuracy metric of all the models is given in Table 5. This table includes accuracy metrics for both the datasets that have been used in prediction, one with the ‘other-crimes’ data and one without.
Neural networks are the most accurate model for both datasets in this scenario. And looking at the classification report, it’s observed that the neural network predicts other crimes most accurately along with strong F1-scores in a few categories. It can also predict theft, vehicle-related theft and drug and alcohol crimes reasonably well. But it is very poor in predicting crimes with a smaller number of instances. It does significantly better in predicting all types of crimes when we remove the all other crimes category and data. The classification reports of all the models are available in Appendix B.
Random forest is the most even in terms of accuracy of prediction for each category. It’s the best predictor for smaller support values such as dangerous and white-collar crimes and it has the best F1-score of all the models. It’s the best predictor across all categories.
Logistic regression is only able to predict a few categories of values such as other crimes and theft, which have large support values. It has zero accuracy for small support values such as dangerous crimes and even public disorder.
Decision trees exhibit similar prediction patterns as the random forest model but is not as accurate as the logistic regression model. But its recall, precision and F1-scores are better than the regression model?This indicates that decision trees actually work better than logistic regression in terms of predicting across all categories.
Support vector machine is the poorest performer of the lot. The only category that it is able to predict to some extent is vehicle-related crimes. It is a very poor performer in all other categories. It is only able to define a hyperplane for one category to some extent and is ineffective for this purpose.
V. Conclusion and Future Work
This project delves into the crime dataset for the city of Denver and attempts to identify hotspots and types of crime occurring in the city. The FP-growth algorithm helps to identify the location and time at which crime would be likeliest to occur in the city, which can help in police monitoring and faster response teams in the area. Using the classification algorithm, we can try to guess the type of crime occurring which allows the police and the government to take action and better deal with crime. The neural network model worked best for this purpose but if the objective is to have reasonable probability over the different categories of crime, random forest is more effective.
As the accuracy metric indicates, it’s quite clear that any of the classification models cannot actually be applied in a real-world scenario. There is clear scope for improvement in utilizing the data to make improved predictions. This can be done in the following ways:
Crime has been growing all around the world over the past few decades. As technology has advanced and more data has become available, data-driven approaches to predict crime can really help in preventing and deploying the police force more effectively. Adaptation of data-driven approaches would help the police force and make cities safer and more habitable.
VI. References
?[1]????Andrey Bogomolov, Bruno Lepri, Jacopo Staiano, Nuria Oliver, Fabio Pianesi, Alex Pentland, “Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data”, 16th International Conference on Multimodal Interaction, November 2014, Trento, Italy.
[2]????Tahani Almanie, Rsha Mirza, Elizabeth Lor,”Crime Prediction Based on Crime Types and Using Spatial and Temporal Criminal Hotspots", International Journal of Data Mining & Knowledge Management Process (IJDKP), July 2015, Boulder,USA.
[3]????Tong?Wang, Cynthia?Rudin, Daniel?Wagner, Rich?Sevieri, “Learning to Detect Patterns of Crime”, Machine Learning and Knowledge Discovery in Databases, September 2013, Cambridge, USA.
[4]????Suhong Kim, Param Joshi, Parminder Singh Kalsi, Pooya Taheri, “Crime Analysis Through Machine Learning”, 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Novmber 2018, British Columbia, Canada.?
[5]????Emmanuel Ahishakiye, Elisha Opiyo Omulo, Danison Taremwa, Ivan Niyonzima, “Crime Prediction Using Decision Tree (J48) Classification Algorithm”, International Journal of Computer and Information Technology, Volume 06 –Issue 03, May 2017, Nairobi, Kenya.
[6]????Maria R. D'Orsogna, Matja? Perc, “Statistical physics of crime: A review”, Physics of Life Reviews, March 2015, Los Angeles, USA.
[7]????Ying-Lung Lin, Tenge-Yang Chen, Liang-Chih Yu, “Using Machine Learning to Assist Crime Prevention”, 2017 6th IIAI International Congress on Advanced Applied Informatics, July 2017, Hamamatsu, Japan.
[8]????Renjie Liao, Xueyao Wang, Lun Li, Zengchang Qin, “A novel serial crime prediction model based on Bayesian learning theory”, 2010 International Conference on Machine Learning and Cybernetics, July 2010, Qingdao, China.
[9]????Lawrence McClendon, Natarajan Meghanathan, “Using Machine Learning Algorithms to Analyze Crime data”, Machine Learning and Applications: An International Journal (MLAIJ) Vol.2, No.1, March 2015, Jackson, USA.
[10]??Umair Saeed, Muhammad Sarim, Amna Usmani, Aniqa Mukhtar, Abdul Basit Shaikh, Sheikh Kashif Raffat, “Application of Machine learning Algorithms in Crime Classification and Classification Rule Mining”, Research Journal of Recent Sciences, Vol. 4(3), March 2015, Karachi, Pakistan.
[11]??Rizwan Iqbal, Masrah Azrifah Azmi Murad, Aida Mustapha, Payam Hassany Shariat Panahy, Nasim Khanahmadliravi, “An Experimental Study of Classification Algorithms for Crime Prediction”, Indian Journal of Science and Technology, March 2013, Selangor, Malaysia.
[12]??Ying-Lung Lin, Meng-Feng Yen, Liang-Chih Yu, “Grid-Based Crime Prediction Using Geographical Features”, ISPRS International Journal of Geo-Information, July 2018, Tainan City, Taiwan.
[13]??Shyam Varan Nath, “Crime Pattern Detection Using Data Mining”, 2006 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology Workshops, December 2006, Hong Kong, China.
[14]??Chung-Hsien Yu, Max W. Ward, Melissa Morabito, Wei Ding, “Crime Forecasting Using Data Mining Techniques”, 2011 IEEE 11th International Conference on Data Mining Workshops, December 2011, Vancouver, Canada.
[15]??Jacky Yu, Wei Ding, Ping Chen, Melissa Morabito, “Crime Forecasting Using Spatio-Temporal Pattern with Ensemble Learning”, Pacific-Asia Conference on Knowledge Discovery and Data Mining, May 2014, Boston, USA.
[16]??Dawei Wang, Wei Ding, Henry Lo, Tomasz Stepinski, Josue Salazar, Melissa Morabito, “Crime hotspot mapping using the crime related factors—a spatial data mining approach”, Applied Intelligence, Volume 39, December 2012, Boston, USA.
[17]???Md. Aminur Rab Ratul, “A Comparative Study on Crime in Denver City Based on Machine Learning and Data Mining”, ?Cornell University Machine Learning, Jan 2020, Ottawa, Canada.
[18]??Mrinalini Jangra, Shaveta Kalsi, “Na?ve Bayes Approach for the Crime Prediction in Data Mining” , International Journal of Computer Applications, May 2019, Jalandhar, India.
[19]??Luiz G.A. Alves, Haroldo V. Ribeiro, Francisco A. Rodrigues, “Crime prediction through urban metrics and statistical learning”, Physica A: Statistical Mechanics and its Applications, September 2018, Sao Paulo, Brazil.
[20]???‘Denver Crime Data’, Available: https://www.kaggle.com/paultimothymooney/denver-crime-data#crime.zip
[21]??‘GDELT Project’, Available: https://www.gdeltproject.org/
[22]??‘GDELT Data Wrangler’, Available: https://nbviewer.jupyter.org/github/JamesPHoughton/Published_Blog_Scripts/blob/master/GDELT%20Wrangler%20-%20Clean.ipynb
[23]??Christian Borgelt, “An Implementation of the FP-growth Algorithm”, 2005 Open Source Data Mining, Aug 2005, Chicago, USA.
[24]??‘Frequent Itemsets via the FP-Growth Algorithm’, https://rasbt.github.io/mlxtend/user_guide/frequent_patterns/fpgrowth/
[25]??‘SkLearn Logistic Regression’, https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
[26]??‘SkLearn Decision Tree’, https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html
[27]??‘SkLearn Random Forest’, https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
[28]??‘SkLearn SVM’, https://scikit-learn.org/stable/modules/svm.html
[29]??‘SkLearn Logistic Regression’, https://scikit-learn.org/stable/modules/neural_networks_supervised.html