Minimal working example

[jvdzwaan]

I can't seem to get passim to work. I made an example data file containing:

{"id": "t1", "text": "This is a text. It contains repeated words. This is the repeated sentence and passim is supposed to find it."}
{"id": "t2", "text": "This is the repeated sentence and passim is supposed to find it."}
{"id": "t3", "text": "This text does not contain the repeated sentence. It is about nothing really."}
{"id": "t4", "text": "Another text. This is the repeated sentence and passim is supposed to find it. What a nice text"}

Then I do:

$passim data/data.json out

It runs and finished without error messages. There is a file in out/out.json, but it is empty:

$ls out/out.json/
part-r-00000-e21212ac-8adf-4adb-9f09-5eab3d2a9136.json  _SUCCESS
$ cat out/out.json/part-r-00000-e21212ac-8adf-4adb-9f09-5eab3d2a9136.json
$

I tried adding different options, including --min-match 2, --pairwise, --n 2, but the output remains empty.

Am I missing something? Could you maybe provide a minimal working example?

[mrckzgl]

I have the same problem. In addition and other than stated in the README, it now seems that the series property is required, otherwise I get an error. But even with this input:

{"id": "d1", "series": "d1", "text": "This is text1."}
{"id": "d2", "series": "d2", "text": "This is text2."}
{"id": "d3", "series": "d3", "text": "This is text."}
{"id": "d4", "series": "d4", "text": "And now for something completely different."}
{"id": "d5", "series": "d5", "text": "And now for something completely different too."}

I get an empty output as you do. @jvdzwaan Did you ever found out how to use passim?

[pleonard212]

I do have a minimal working example -- I think what you both have (and I had at first) may be "too short" to trigger an alignment match, at least without tweaking of the defaults.

{"id": "crusenstolpe", "series": "b", "text": "I detsamma öppnades dörren från öfra väningen i värdshuset och på det öppna utbygget framträdde en något länglagd, blek och mager man. Den vackra hvällda pannan öfver en stor krokig näsa oeh de stora blä ögonen, ehuru deras eld merendels skymdes af ett sömnaktigt flor, kunde ej undgä att fängsla uppmärksamheten. Håret, tillbakastruket, faddrade i naturliga lockar. Klädseln var tarflig, men snygg. Han hade lagt af sig rocken, och de hvita skjortärmarna stucko ganska bjert af mot den scharlakans-färgade västen. Han höll en luta i handen, satte sig pa en brunmalad trästol invid relingen af utspränget och knäppte några förelöpande."}
{"id": "benzon", "series": "a", "text": "Den af de tre, som starkast fängslade blicken, var en nägot långlagd, blek och mager man. Den vackra hvälfda pannan öfver en stor och krokig näsa och de stora blä ögonen, hvilkas eld dock merendels skymdes af ett sömnaktigt flor, mäste ovillkorligen fjättra uppmärk-samheten. Det tillbakastrukna håret fladdrade i naturliga lockar, utan att vanställas af stängpiska eller puder. Klädseln var nästan hvad man kallar tarflig, men snygg. Han hade lagt af sig rocken, och de bländhvita skjortärmarna stucko bjärt af mot den skarlakansroda västen. Han höll en luta i handen."}

Having said that, I'm failing at finding alignments in longer book-length texts, so I'm still figuring out what's going on there...

Edit: when in doubt, I should always validate my jsonl. A rogue backslash from OCR snuck in and created an invalid escape sequence. Now getting expected results!