thesis-anne/README.md

# Anne's Bachelor Thesis
State: October 2018 (in progress)

My python classes for text mining, machine learning models, …
The scripts can be called separately.

Best F1 score results were:

SVM
---
F1 score: 0.8944166649330559
best parameters set found on development set:
{'SVC__C': 0.1, 'SVC__gamma': 0.01, 'SVC__kernel': 'linear', 'perc__percentile': 50}

Naive Bayes
-----------
parameters: SelectPercentile(25), own BOW implementation, 10-fold cross validation
F1 score: min = 0.7586206896551724, max = 0.8846153846153846, average = 0.8324014738144634

The complete documentation can be found in the latex document in the thesis folder.

The csv file 'classification_labelled_corrected.csv' contains 1497 labeled news articles from Reuters.com and is used for the machine learning models.

Note: 
Please enter a valid webhose personal key before you call 'Requester.py'.
Also, please change the path to your JAVAHOME environment variable in 'NER.find_companies' method.

example:
# set paths
java_path = "C:\\Program Files (x86)\\Java\\jre1.8.0_181"
os.environ['JAVAHOME'] = java_path


## Requirements

pandas==0.20.1
nltk==3.2.5
webhoseio==0.5
numpy==1.14.0
graphviz==0.9
scikit_learn==0.19.2

## Installation under Windows

pip install XY

## Installation under UBUNTU

apt-get install XX