GitHub - DennisDeSwart/php-stanford-nlp-datastore: Stores NLP data from Stanford CoreNLP server. (original) (raw)

PHP Stanford NLP Datastore

Stores NLP data from Stanford CoreNLP server.

What does it do?

It analyses a text using Stanford CoreNLP server, then stores the result.

Which data gets stored?

OpenIE: these are "Subject-Relation-Object" triples. The concept is similar to "Subject-Verb-Object" triples.

http://stanfordnlp.github.io/CoreNLP/openie.html

Named-Entities: if a word is a "Named Entity", like a Location, Name or Time, it will store this data

http://stanfordnlp.github.io/CoreNLP/ner.html

Coreference: if there is a reference to a word in another sentence.

http://stanfordnlp.github.io/CoreNLP/coref.html

How does it work?

You submit a text.
The text is analyzed by the Stanford CoreNLP server
Results are stored in a SQLite file based database. The database file is called "datastore.db"

https://sqlite.org/
https://github.com/sqlitebrowser

The results are displayed on screen
There is also a search form to find data

This package depends on Stanford CoreNLP Server

http://stanfordnlp.github.io/CoreNLP/index.html#download

This package also depends on PHP-Stanford-CoreNLP-Adapter

https://github.com/DennisDeSwart/php-stanford-corenlp-adapter

Note: since this package contains a full version of the CoreNLP Adapter, you can use all of it's features with this package.

Installation

This package depends on these packages:

http://stanfordnlp.github.io/CoreNLP/index.html#download
https://github.com/DennisDeSwart/php-stanford-corenlp-adapter
https://github.com/doctrine/dbal
https://github.com/guzzle/guzzle

Install procedure using the ZIP files

Install Stanford CoreNLP Server. Check the "php-stanford-corenlp-adapter" package for an installation walkthrough
Download and unpack the files from this package.
Copy the files to your to your webserver directory. Usually "htdocs" or "var/www".
Run a Composer update to install the dependencies

Install as part of another project

Install Stanford CoreNLP Server. Check the "php-stanford-corenlp-adapter" package for an installation walkthrough
Add the following lines to your main project's "composer.json" require section:

    {
        "require": {
            "dennis-de-swart/php-stanford-nlp-datastore": "*"
        }
    }

Run a Composer update to install the dependencies

Copy these files from "/vendor/dennis-de-swart/php-stanford-nlp-datastore" to your webserver directory. Usually "htdocs" or "var/www".
- datastore.db
- bootstrap.php

Example code for your main project:

    // instantiate constants and the database
    require_once __DIR__.'/bootstrap.php';

    // startup Corenlp Adapter
    $coreNLP = new CorenlpAdapter();
    <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>c</mi><mi>o</mi><mi>r</mi><mi>e</mi><mi>N</mi><mi>L</mi><mi>P</mi><mo>−</mo><mo>&gt;</mo><mi>g</mi><mi>e</mi><mi>t</mi><mi>O</mi><mi>u</mi><mi>t</mi><mi>p</mi><mi>u</mi><mi>t</mi><mo stretchy="false">(</mo></mrow><annotation encoding="application/x-tex">coreNLP-&gt;getOutput(</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.7667em;vertical-align:-0.0833em;"></span><span class="mord mathnormal">core</span><span class="mord mathnormal" style="margin-right:0.10903em;">N</span><span class="mord mathnormal">L</span><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="mord">−</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">&gt;</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal">e</span><span class="mord mathnormal" style="margin-right:0.02778em;">tO</span><span class="mord mathnormal">u</span><span class="mord mathnormal">tp</span><span class="mord mathnormal">u</span><span class="mord mathnormal">t</span><span class="mopen">(</span></span></span></span>yourText);
    print_r($coreNLP->serverMemory); // result from CoreNLP Adapter

    // Save result to database
    <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>d</mi><mi>a</mi><mi>t</mi><mi>a</mi><mi>s</mi><mi>t</mi><mi>o</mi><mi>r</mi><mi>e</mi><mo>=</mo><mi>n</mi><mi>e</mi><mi>w</mi><mi>D</mi><mi>a</mi><mi>t</mi><mi>a</mi><mi>s</mi><mi>t</mi><mi>o</mi><mi>r</mi><mi>e</mi><mo stretchy="false">(</mo></mrow><annotation encoding="application/x-tex">datastore = new Datastore(</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal">d</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">a</span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ore</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">n</span><span class="mord mathnormal">e</span><span class="mord mathnormal" style="margin-right:0.02691em;">w</span><span class="mord mathnormal" style="margin-right:0.02778em;">D</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">a</span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ore</span><span class="mopen">(</span></span></span></span>db->conn);
    <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>d</mi><mi>a</mi><mi>t</mi><mi>a</mi><mi>s</mi><mi>t</mi><mi>o</mi><mi>r</mi><mi>e</mi><mo>−</mo><mo>&gt;</mo><mi>s</mi><mi>t</mi><mi>o</mi><mi>r</mi><mi>e</mi><mi>N</mi><mi>L</mi><mi>P</mi><mo stretchy="false">(</mo></mrow><annotation encoding="application/x-tex">datastore-&gt;storeNLP(</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.7778em;vertical-align:-0.0833em;"></span><span class="mord mathnormal">d</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">a</span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ore</span><span class="mord">−</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">&gt;</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ore</span><span class="mord mathnormal" style="margin-right:0.10903em;">N</span><span class="mord mathnormal">L</span><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="mopen">(</span></span></span></span>coreNLP);

Requirements

PHP 5.6 or higher: it also works on PHP 7
Java SE Runtime Enviroment, version 1.8
Stanford CoreNLP Server 3.7.0
Windows or Linux/Unix 64-bit OS, 8Gb or more memory recommended.
Composer for PHP

SQLite Browser

If you need a SQLite browser check here:

http://sqlitebrowser.org/

Important notes

Starting the CoreNLP server for the first time, takes some time because it will load a large amount of data.
After the first startup, the server will be much faster.
In my experience the Stanford CoreNLP server runs best with 8Gb of memory or more. Start the server with "-mx8g" instead of "-mx4g".
Also use version 3.7.0 of the server, this gives you the best and quickest results.

Example output

See

"datastore_result_a.PNG"
"datastore_result_b.PNG"
"datastore_result_search.PNG"

and "example.db", this is how a filled database looks like

Any questions?

Let me know. You can create an issue on GitHub. Any bugs will be fixed ASAP.