Why codes file is empty.?

[ykkhan]

I am facing this error

Applying BPE to valid and test files...
Loading vocabulary from /home/UnsupervisedMT/NMT/data/mono/vocab.en.1500 ...
Read 26726 words (93 unique) from vocabulary file.
Loading codes from /home/UnsupervisedMT/NMT/data/mono/bpe_codes ...
Read 0 codes from the codes file.
Loading vocabulary from /home/UnsupervisedMT/NMT/data/para/vs11.txt ...
Read 0 words (0 unique) from text file.
Applying BPE to /home/UnsupervisedMT/NMT/data/para/vs11.txt ...
Output memory map failed : 22.

where am making error?

[glample]

This kind of issues with fastBPE typically happens when you have too many BPE codes and a too small vocabulary.
What is your vocabulary size, and how many BPE codes are you trying to compute?

[ykkhan]

Ok, I get it.
can you suggest how many code should i compute for dataset has 610 sentence in each file (train, test, valid). it is also not extract correct vocabulary. its only extract alphabets instead of words. as shown in image. it is computing just 93 vocabulary size.

for 610 sentences, I have tried different values in almost 200 to 1500 for BPE code. for each code value it is giving same issue.
Although dataset is very small, but at this time my task is to run this technique successfully. Next I will increase dataset.

[glample]

610 sentences is very small. I would simply use word level and not BPE in this case.
BPE is useful to reduce the vocabulary size and to avoid computing a softmax over hundred of thousands of elements. But in your case the vocabulary will be very small so you probably don't need BPE.

[ykkhan]

ok, thanks for replying. can you guide me little bit more. which part of data.enfr.sh I should remove, what to insert in the code to this.