GitHub - marcotcr/lime-experiments: Code for all experiments. (original) (raw)

This repository contains the code to run the experiments present in this paper. The code here is frozen to what it was when we originally wrote the paper. If you're interested in using LIME, check out this repository, where we have packaged it up, improved the code quality, added visualizations and other improvements.

Running the commands below should be enough to get all of the results. You need specific versions python, sklearn, numpy, scipy. Install requirements in a virtualenv using:

pip install -r requirements.txt

If we forgot something, please email the first author.

Experiment in section 5.2:

• DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'
• ALGORITHM -> 'l1logreg', 'tree'
• EXPLAINER -> 'lime', 'parzen', 'greedy' or 'random'

  python evaluate_explanations.py --dataset DATASET --algorithm ALGORITHM --explainer EXPLAINER  

Experiment in section 5.3:

• DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'
• ALGORITHM -> 'logreg', 'random_forest', 'svm', 'tree' or 'embforest', although you would need to set up word2vec for embforest

  python data_trusting.py -d DATASET -a ALGORITHM -k 10 -u .25 -r NUM_ROUNDS  

Experiment in section 5.4:

• NUM_ROUNDS -> Desired number of rounds
• DATASET -> 'multi_polarity_books', 'multi_polarity_kitchen', 'multi_polarity_dvd', 'multi_polarity_kitchen'
• PICK -> 'submodular' or 'random' Run the following with the desired number of rounds:

  mkdir out_comparing  
  python generate_data_for_compare_classifiers.py -d DATASET -o out_comparing/ -k 10 -r NUM_ROUNDS  
  python compare_classifiers.py -d DATASET -o out_comparing/ -k 10 -n 10 -p PICK  

Religion dataset:

Available here

Multi-polarity datasets:

I got them from here