runtagger_parse output mismatch with input

[abhishekvaid]

I'm trying to use your module with ARK on a project. We have a dump of tweets which need to POS Tag. On extensive testing, I've found that there's an issue with your script.

Your runtagger_parse method sometimes doesn't produce same number of lists as there are strings in the input. To exemplify, if I provide a list of 100 strings to runtagger_parse, then it sometimes produces 103 to 104 tuples as well. This is clearly a problem I hope you can see.

If you need the test case, please ping me on vaid.abhi@gmail.com or email me. I'll give you the serialized version of input which you can use for testing.

Regards

Vaid, Abhishek

[ianozsvald]

Yes, please send me the smallest repeatable test case with both the inputs
you use and a grab of the output you see (so I can verify I see the same
problem). I'll then take a look.
Please also note your OS (Linux? Windows?), 32 vs 64 bit and anything else
you think is pertinent.
Thanks for the report,
Ian
On 30 May 2013 12:27, "Abhishek Vaid" notifications@github.com wrote:

I'm trying to use your module with ARK on a project. We have a dump of
tweets which need to POS Tag. On extensive testing, I've found that there's
an issue with your script.

Your runtagger_parse method sometimes doesn't produce same number of lists
as there are strings in the input. To exemplify, if I provide a list of 100
strings to runtagger_parse, then it sometimes produces 103 to 104 tuples as
well. This is clearly a problem I hope you can see.

If you need the test case, please ping me on vaid.abhi@gmail.com or email
me. I'll give you the serialized version of input which you can use for
testing.

Regards

Vaid, Abhishek

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3
.

[prateekmehta]

I want to second this issue, also what i have found is that,

1. it can not deal with empty strings as in if the tweet is '' it would not return anything ideally it should have returned an empty string, eg:

In [168]: a = ["my name is prateek","i am 10 years old","","how are you","this is my url http://prateekmehta.in"]

In [169]: a = ["my name is prateek","i am 23 years old","","how are you","this is my url http://prateekmehta.in"]

In [170]: CMUTweetTagger.runtagger_parse(a,run_tagger_cmd="java -XX:ParallelGCThreads=2 -Xmx500m -jar /home/prateek/utils/ark-tweet-nlp-0.3.2/ark-tweet-nlp-0.3.2.jar")
Out[170]:
[[('my', 'D', 0.9975),
('name', 'N', 0.9998),
('is', 'V', 0.9967),
('prateek', 'A', 0.5868)],
[('i', 'O', 0.9875),
('am', 'V', 0.9968),
('23', '$', 0.9796),
('years', 'N', 0.968),
('old', 'A', 0.9783)],
[('how', 'R', 0.9484), ('are', 'V', 0.9948), ('you', 'O', 0.9977)],
[('this', 'O', 0.9526),
('is', 'V', 0.9995),
('my', 'D', 0.9912),
('url', 'N', 0.9627),
('http://prateekmehta.in', 'U', 0.981)]]

1. If the first string is empty ie "" it returns an empty [[]]
   In [171]: a = ["","my name is prateek","i am 10 years old","","how are you","this is my url http://prateekmehta.in"]

In [172]: CMUTweetTagger.runtagger_parse(a,run_tagger_cmd="java -XX:ParallelGCThreads=2 -Xmx500m -jar /home/prateek/utils/ark-tweet-nlp-0.3.2/ark-tweet-nlp-0.3.2.jar")
Out[172]: [[]]