DependenSee: A Dependency Parse Visualisation/Visualization Tool
There aren’t many tools which allow you to visualise sentences parsed with dependency grammars. Here’s a small tool which generates a PNG of the dependency graph of a given sentence using the Stanford Parser. You can generate the image for Einey’s quote below by following these steps.
- Click here to download
DependenSee.jar. - Download the latest version of the Stanford Parser. I am using version 2.0.1 (For older versions, drop me an email)
- Extract
stanford-parser.jarandstanford-parser-2012-03-09-models.jarin the same folder asDependenSee.jar. - On the command prompt, run
java -cp DependenSee.jar;stanford-parser.jar;stanford-parser-2012-03-09-models.jar com.chaoticity.dependensee.Main "Example isn't another way to teach, it is the only way to teach." out.png
(If you are on *nix, replace the semicolon by a colon and make sure you have Arial installed. If you have an already parsed dependency output file, replace the sentence by-t input.txt.) - Open
out.pngand admire :)
I have added Part-of-Speech tags and very basic edge overlap management and might add more eye candy later (curved/coloured edges ?). You can link the library in your code as well. An example is given below. Comments and queries are welcome.
import edu.stanford.nlp.trees.*;
import edu.stanford.nlp.parser.lexparser.*;
import com.chaoticity.dependensee.*;
import java.util.Collection;
class Test {
public static void main(String []args) throws Exception {
String text = "A quick brown fox jumped over the lazy dog.";
TreebankLanguagePack tlp = new PennTreebankLanguagePack();
GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
LexicalizedParser lp = LexicalizedParser.loadModel();
Tree tree = lp.apply(text);
GrammaticalStructure gs = gsf.newGrammaticalStructure(tree);
Collection tdl = gs.typedDependenciesCCprocessed(true);
Main.writeImage(tree,tdl, "image.png",3);
}
}
Hi, your dependency parser visualiser is very cool. I was wondering if you might have any code snippets that show how one might generate an image from a Stanford Tree object when run from within Java rather than through the main() call via command line? How do you use the Node, Edge, and Graph classes?
I tried iterating over the various tagged word and dependency collections from Tree but got many errors such as
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937)
at com.chaoticity.dependensee.Node.(Node.java:41)
Many thanks!
Ah! Missed that. I just updated the jar and you can now generate an image from a Tree as well.
LexicalizedParser lp = new LexicalizedParser("englishPCFG.ser.gz");
lp.setOptionFlags(new String[]{"-maxLength", "500", "-retainTmpSubcategories"});
Tree tree = lp.apply("Hello World!");
writeImage(tree, "image.png");
Hope this helps.
Many thanks for such a quick reply – exactly what I was looking for :)
from the top of my mind, java takes a : instead of a ; as a separator on *nix so it should work if you use the command pasted below. You might have to install the Arial font though.
java-cp DependenSee.jar:stanford-parser.jar com.chaoticity.dependensee.Main "Example isn't another way to teach, it is the only way to teach." out.pngHi,
This looks wonderful; I cannot get it to work. Perhaps you can help? I have placed these 3 in a folder: DependenSee.jar/
englishPCFG.ser.gz
stanford-parser.jar
And then try running the command you posted, and get this usage error (see output below): -bash: ./stanford-parser.jar: cannot execute binary file
Thanks!
Y
l
total 7464
-rwxr-xr-x@ 1880577 Aug 20 10:31 stanford-parser.jar*
-rw-r–r–@ 1934638 Aug 20 10:31 englishPCFG.ser.gz
drwx—— 170 Nov 2 11:09 DependenSee.jar/
$ java -cp DependenSee.jar;stanford-parser.jar com.chaoticity.dependensee.Main “Example isn’t another way to teach, it is the only way to teach.” out.png
Usage: java [-options] class [args...]
(to execute a class)
or java [-options] -jar jarfile [args...]
(to execute a jar file)
where options include:
-d32 use a 32-bit data model if available
-d64 use a 64-bit data model if available (implies -server, only for x86_64)
-client to select the “client” VM
-server to select the “server” VM
-jvm is a synonym for the “client” VM [deprecated]
-hotspot is a synonym for the “client” VM [deprecated]
The default VM is client.
-cp
-classpath
A : separated list of directories, JAR archives,
and ZIP archives to search for class files.
-D=
set a system property
-verbose[:class|gc|jni]
enable verbose output
-version print product version and exit
-version:
require the specified version to run
-showversion print product version and continue
-jre-restrict-search | -jre-no-restrict-search
include/exclude user private JREs in the version search
-? -help print this help message
-X print help on non-standard options
-ea[:...|:]
-enableassertions[:...|:]
enable assertions
-da[:...|:]
-disableassertions[:...|:]
disable assertions
-esa | -enablesystemassertions
enable system assertions
-dsa | -disablesystemassertions
disable system assertions
-agentlib:[=]
load native agent library , e.g. -agentlib:hprof
see also, -agentlib:jdwp=help and -agentlib:hprof=help
-agentpath:[=]
load native agent library by full pathname
-javaagent:[=]
load Java programming language agent, see java.lang.instrument
-bash: ./stanford-parser.jar: cannot execute binary file
Thanks for such a quick reply! Works now. (apologies I’m a newby) This is fantastic! Beautiful display.
Hi, is there a way to specify options, such as that I want to use the CCPropagatedDependencies option to get propogated conjunctions rather than the basic dependencies? Thanks!!
I agree with Y. It would be interesting to allow the user to add to the command any option provided by the Stanford Parser (using as input POS tagged sentences for example). You could probably concatenate these options to the lp.setOptionFlags parameter.
I’ve added a function which takes a collection of typed dependencies as input. It won’t work on command-line though.
Hi,
This works perfectly with Stanford parser. Awesome. Great work!
Just wonder whether I can use this with some other GR representations. For example RASP GR output (Cambridge). If so what is the command line input? Is there a way to format input GRs?
Just went through your profile and came to know that you are also from Cambridge. Same here!
Have fun,
Sriwantha Attanayake
Sorry, I haven’t used RASP GRs so far. I’ll update when I need that though. Thanks.
This is really great! It works fine.
Is there a way to control the size and the resolution of the generated image. It’s now relatively small and resizing it using image editing tools makes the text unreadable.
Thanks Amjad, I’ll try to add in a scaling factor whenever I get some free time on my hands :)
“scale” added.
hello;
i want to know for using above code
LexicalizedParser lp = new LexicalizedParser(“englishPCFG.ser.gz”);
lp.setOptionFlags(new String[]{“-maxLength”, “500″, “-retainTmpSubcategories”});
Tree tree = lp.apply(“Hello World!”);
writeImage(tree, “image.png”);
i have to import which package for [ writeImage(tree,"image.png");]
i already upload librarires ( DependenSee.jar , stanford-parser.jar)
I’m adding some test code in the main post. For other things, you can perhaps try out http://nlp.stanford.edu/software/corenlp.shtml
Thanks!!
Can i integrate it with RitaWordnet(which is an online dictionary).
hello all;
can any one provide me code of
1. tokenizer
2. Sentence generation from paragraph
3. group the phrases to find the sentence from tokens.
Please reply fast!!!!!!
This is an awesome tool. Thanks!!
I linked your jar as a library, but I couldn’t find the class that contains the function writeImage() in the Main.class.
Is there are class that I can create for that?
Thanks !
TRY IT!!!!!!!!!!!!
STEPS TO RUN:
1. ADD following jars:
DependenSee.jar,stanford-parser.jar
2. Make “textfile.text” on dekstop and write sentence in it , my sentence is “Paraphrasing is a type of plagiarism.
3. Just copy & paste the code on Eclipse(IDE).
import java.io.*;
import java.util.*;
import edu.stanford.nlp.trees.*;
import edu.stanford.nlp.process.*;
import edu.stanford.nlp.objectbank.TokenizerFactory;
import edu.stanford.nlp.parser.lexparser.LexicalizedParser;
import edu.stanford.nlp.ling.Sentence;
import edu.stanford.nlp.ling.TaggedWord;
import edu.stanford.nlp.ling.HasWord;
import rita.wordnet.*;
import edu.stanford.nlp.tagger.maxent.MaxentTagger;
public class parser {
public static void main(String[] args) // start of the main method
{
// RiWordnet wordnet = new RiWordnet(null);
System.out.println(“\n\n\nSTART\n\n\n”); // print START
try // device to handle potential errors
{
// open file whose path is passed
// as the first argument of the main method:
FileInputStream fstream = new FileInputStream(“C:/Users/nidhi/Desktop/textfile.txt”);
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
// prepare Parser, Tokenizer and Tree printer:
LexicalizedParser lp = new LexicalizedParser(“englishPCFG.ser.gz”);
TokenizerFactory tf = PTBTokenizer.factory(false, new WordTokenFactory());
TreePrint tp = new TreePrint(“penn,typedDependenciesCollapsed”);
String sentence; // initialization
// for each line of the file
// retrieve it as a string called ‘sentence’:
while ((sentence = br.readLine()) != null)
{
// print sentence:
System.out.println (“\n\n\n\nORIGINAL:\n\n” + sentence);// put tokens in a list:
List tokens = tf.getTokenizer(new StringReader(sentence)).tokenize();
System.out.println(“Sentence is:” + sentence);
System.out.println(“Tokens” +tokens);
lp.parse(tokens); // parse the tokens
Tree t = lp.getBestParse(); // get the best parse tree
System.out.println(“\nPROCESSED:\n\n”); tp.printTree(t); // print tree
}
in.close(); // close input file
}
catch (Exception e) // catch error if any
{
System.err.println(“ERROR: ” + e.getMessage()); // print error message
}
System.out.println(“\n\n\nTHE END\n\n\n”); // print THE END
} // end of the main method
}
// end of the myParser class
4. See the output its amazing………..
START
Loading parser from serialized file englishPCFG.ser.gz … done [4.4 sec].
ORIGINAL:
Paraphrasing is a type of plagiarism.
Sentence is:Paraphrasing is a type of plagiarism.
Tokens[Paraphrasing, is, a, type, of, plagiarism, .]
PROCESSED:
(ROOT
(S
(NP (NN Paraphrasing))
(VP (VBZ is)
(NP
(NP (DT a) (NN type))
(PP (IN of)
(NP (NN plagiarism)))))
(. .)))
nsubj(type-4, Paraphrasing-1)
cop(type-4, is-2)
det(type-4, a-3)
prep_of(type-4, plagiarism-6)
THE END
This is the myParser example from the stanford parser site. I was talking about the DependenSee example.
Thanks though.
Neal, the updated example should work now.
hello sir;
i will try it as
Main.writeImage(tree, “out.png”);
it works!!!
thanks!!
No, doesn’t
hello sir;
i will try it as
Main.writeImage(tree, “out.png”);
it works!!!
thanks!!
Hi,
I’m using Charniak’s parser and that Stanford tool which provides the corresponding dependency structure:
[...]
Collection tDeps = gs.typedDependenciesCCprocessed(true);
Is there a method like ‘writeImage(tDeps, “img.png”)’? I cannot find one in ‘Main’.
For now, the graph nodes and the POS labels are constructed using the parse tree just to make sure that nodes without dependencies are included. If you can paste the code which generates the grammatical structure (gs) here, I’ll take a look if there is a way to get the parse tree from it. Otherwise, let me know and I’ll add a function which takes just dependencies.
my code:
TreeReader tReader = new PennTreeReader(new StringReader(parsedSentence), new LabeledScoredTreeFactory());
EnglishGrammaticalStructure egStructure = new EnglishGrammaticalStructure(tReader.readTree());
Collection collection = egStructure.typedDependenciesCCprocessed(true);
try this:
...
Tree tree = tReader.readTree();
EnglishGrammaticalStructure egStructure = new EnglishGrammaticalStructure(tree);
Collection collection = egStructure.typedDependenciesCCprocessed(true);
...
Main.writeImage(tree,collection, "image.png");
Please provide working code: There ist no ‘Main.writeImage’ in the lib :(
DerSepp, did you import com.chaoticity.dependensee.* ? The example in the Test class pasted above should work
Grade A stuff. I’m unuqsetioabnly in your debt.
Do you mean ‘com.chaoticity.dependensee.Main.writeImage(tree, collection, “image.png”);’?
My IDE does not find any method like ‘writeImage’.
yes, it’s
Main.writeImage()Awais, this is an awesome effort and I really liked this idea. But I see some problems stated so far- all about wrtiteImage method in the Main class.
We all are having same problem- we included the Dependensee.jar and stanford-parser.jar and tried you Test class. But we are getting
“The method writeImage(Tree, Collection, String, int) is undefined for the type Main” problem.
This means the in the Main class of the jar file Dependensee does not contain the method writeImage.
Can you please resolve the issue? I don’t know how it worked for many folks.
Rushdi, please try the download now. Looks like there was a problem in the previous update. It should work now.
try this:
class Test {
public static void main(String []args) throws Exception {
String text = “Slovenija gre naprej!!”;
String outImage = “image.png”;
com.chaoticity.dependensee.Main.main(new String[]{text, outImage});
}
I believe the function is protected or private. Reverse engineer to find out – I do not have time now. You can still call private functions in Java, but it in an ugly way – check Java Bible book.
Br,
Slavko
Hello,
After installing the jars and running the command, I get an error (see below).
Is it an issue with the latest parser version or is something wrong with my setup?
Thanks in advance for looking at this!
java -cp DependenSee.jar:stanford-parser.jar com.chaoticity.dependensee.Main “Example isn’t another way to teach, it is the only way to teach.” out.png
Loading parser from serialized file englishPCFG.ser.gz … done [3.5 sec].
Exception in thread “main” java.lang.NoSuchMethodError: edu.stanford.nlp.trees.Tree.taggedYield()Ledu/stanford/nlp/ling/Sentence;
at com.chaoticity.dependensee.Main.getGraph(Main.java:105)
at com.chaoticity.dependensee.Main.writeImage(Main.java:232)
at com.chaoticity.dependensee.Main.writeImage(Main.java:216)
at com.chaoticity.dependensee.Main.writeImage(Main.java:198)
at com.chaoticity.dependensee.Main.main(Main.java:50)
It should work with the latest version of the parser. Can you tell which version of the parser are you using and which OS as well?
Thanks a lot! It works.
I am getting the same error. I am using Mac OS, parser version 1.6.7.
Rita and sveter,
My bad! I never updated the file after the new API changes. The latest version of DependenSee.jar should work with v1.6.7 of parser now. Let me know if it doesn’t.
Thank you very much! It works now. I am running it on Mac OS X as well.
sorry, is this library open source?
or under which license is released?
I get this exception:
java.util.NoSuchElementException
at java.util.AbstractList$Itr.next(AbstractList.java:)
at com.chaoticity.dependensee.Main.getGraph(Main.java:)
at Foo.parseString(Foo.java:)
using this phrase “a dog is an animal that barks” with your code:
Tree tree = lexParser.apply(text);
TreebankLanguagePack tlp = new PennTreebankLanguagePack();
GrammaticalStructureFactory gsf = tlp.grammaticalStructureFactory();
GrammaticalStructure gs = gsf.newGrammaticalStructure(tree);
List tdl = gs.typedDependenciesCCprocessed();
/*
EnglishGrammaticalStructure gs = new EnglishGrammaticalStructure(tree);
Collection tdl = gs.typedDependenciesCCprocessed(true);
*/
writeImage(tree, tdl, “image.png”);
(omitting the tdl argument causes no exception)
Sorry about that. It should work now.
Hi, I already have the dependency graph for the sentence, and I’d just like to use this code to visualize the graph. How can I do that? Some DGs are from previous versions of the Parser, some are from the most current. Thanks!
hmm.. let me see what I can do.
Assuming you have one file per sentence which contains only the dependency text, download the latest version and use the -t switch. I have tested using the following command line
java -cp DependenSee.jar;stanford-parser.jar com.chaoticity.dependensee.Main -t input.txt out.png
when
input.txtcontains the following text. POS tags have not been handled yet.nsubj(way-5, Example-1)
cop(way-5, is-2)
neg(way-5, n't-3)
det(way-5, another-4)
ccomp(way-13, way-5)
aux(teach-7, to-6)
infmod(way-5, teach-7)
nsubj(way-13, it-9)
cop(way-13, is-10)
det(way-13, the-11)
amod(way-13, only-12)
aux(teach-15, to-14)
infmod(way-13, teach-15)
Has a parser been developed on the same lines for the Indian language malayalam?
None that I know off the top of my head. There’s this paper ( http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5460570 ) but you’ll have to hunt the authors down for a complete resource.
Hi,
Can I put this jar on clojars? If so, what is the version number you’re using?
Thanks!
Sure, this is 1.4.2
Brother Awais…
I want to extract the the location of a word in a sentence. During the extraction of typed dependency extraction, stanford parser provied it like “dobj(love-2, country-4)” where love is at location 2 and country is at location 4. But how can i control and display programatically these location value. Is there any function is available in Stanford-parser API.
Please Help me
given a
TypedDependencyobjecttd, you can probably get the indices bytd.dep().index() - 1andtd.gov().index() - 1. There are other methods as well. Just open up thesrcdirectory and see thetoStringmethod of the related class.Brother Awais Thanks for ur support…..
I need some more help. Basically i am working on Ontology learning from text. So the first phase of that is to parse the whole text. Now I m using Stanford Parser.
Can I Handle the dependency relation between between two word from the whol dependency graph….
For example
a sentense “I Love my coutnry”…..so the depencies are
nsubj(love-2, I-1)
poss(country-4, my-3)
dobj(love-2, country-4)
Now i want to work with the third dependency relation i.e. “dobj(love-2, country-4)”
Basically brother….I want to make some semantic patterns and apply on these dependecny relation programtically…..
can you help me in this regard
I need some snippest of code that how to extract the index as well as how get the single dependency relation from graph…
i m new in NLP…So please help me…..have u any messenger ID
try going through the ParserDemo.java file and look up the API in the javadoc and the stanford typed dependency manual.
Hi Awais,
very nice work. I was able to use it in my Java project. For another project I would like to visualize the PTB trees. However, when calling
Main.writeImage(ptbTree,”image.png”);
the tool display’s again the dependency parse tree’s instead of the PTB tree. Is there any possibility to visualize that?
Best,
Philippe
Thanks Philippe
This tool was created for dependencies only. I’ll see if I can modify it to get a general tree structure but there are some tool out there which already do so. GrammarScope is one of them (http://grammarscope.sourceforge.net/). Personally I prefer running the modified output of the pennPrint through phpSyntaxTree.
ptbTree.pennString().replaceAll("\\[.*\\]|\r\n| ", "").replaceAll("[ ]*\\(", "\\[").replaceAll("[ ]*\\)", "\\]"))
Hope this helps
forgot the link… http://www.ironcreek.net/phpsyntaxtree/?
Thanks Awais..
Hi, could you provide a version of your dependency parse visualization tool where the path to the language model is not hard-coded (i.e. englishPCFG.ser.gz)?
This would make testing your tool on other languages possible, and I guess it’s just a matter of making the path to the language model a variable.
Thanks.
I guess you can use the code given at the end of the post and change the string parameter passed to the variable
lp. Tell me if that doesn’t work.Hi, this is such a cool program and it works fine on my machine. But can you please tell how could i read the parsed structure in the form of pure text? Thank you in advance for your reply.
Lina, if you want to draw the output produced by the Stanford dependency parser, this reply might help. http://chaoticity.com/dependensee-a-dependency-parse-visualisation-tool/#comment-6830
I get an error, why? can anyone explain ?
Exception in thread “main” java.lang.NullPointerException
at com.chaoticity.dependensee.Graph.addEdge(Graph.java:50)
at com.chaoticity.dependensee.Main.getGraph(Main.java:116)
at com.chaoticity.dependensee.Main.writeImage(Main.java:253)
at com.chaoticity.dependensee.Main.writeImage(Main.java:237)
Alex, Can you paste the sentence which caused this error?
Alex, please use the updated DependenSee.jar file. The problem was due to a version update in the Stanford parser and should be fixed now. Let me know if there are any problems. Hope this helps.
Hi awais，I have a basic question about the stanford parser dependency relation.
“the mp3 is great but much more expensive” . I want to find the “shortest dependency path ” between the word “mp3″ and “expensive”. if there is a simple
implemention? thanks!
Not really sure but there seems to be a method Tree.pathNodeToNode(Tree t1, Tree t2) which might help you. If that doesn;t work, you might find https://mailman.stanford.edu/mailman/listinfo/parser-user to be helpful
Tree.pathNodeToNode method is used to find the syntactic path between two words , not used to find the dependency path between two words. Thank you very much!
Seems that it has problem to display Chinese dependency
I‘m using the xinhuaPCFG.ser.gz and using ChineseTreebankLanguagePack() instead of PennTreebankLanguagePack().
TreebankLanguagePack tlp = new ChineseTreebankLanguagePack();
I got the following exception：
java.lang.RuntimeException: Failed to invoke public edu.stanford.nlp.trees.EnglishGrammaticalStructure(edu.stanford.nlp.trees.Tree)
at edu.stanford.nlp.trees.GrammaticalStructureFactory.newGrammaticalStructure(GrammaticalStructureFactory.java:104)
at com.chaoticity.dependensee.Main.getGraph(Main.java:149)
at com.chaoticity.dependensee.Main.writeImage(Main.java:255)
at DependencySee.showCNDependency(DependencySee.java:60)
at DependencySee.main(DependencySee.java:17)
…
hi, have u any document in which it is explained how your programm work?
thank u
It seems not work with Stanford Parser 2.0.1 : no LexicalizedParser (String str) constructor.
Ah! I guess i’ll fix that soon. Keep checking back :)
The latest version should work now.
This looks great! thank you for sharing.