Sync from S3 to local - Missing first letter of sub-directory

[YJ-AllCloud]

Hi,

When running s3-cli sync from s3 to local the first letter of the sub-directory is missing:
Copy s3://test/testfolder/ -> file://test/estfolder/
The files are copied fine.

Appreciate any assistance.
Thanks

[koblas]

Still working on reproducing. Can you send me the exact command, maybe with the -v output. Here's what I quickly did for this -- but don't see a missing letter.

$ rm -rf /tmp/t2
$ ./s3-cli sync -v s3://koblas/test_folder/ /tmp/t2/
1 files to consider - 0 bytes
Copy s3://koblas/test_folder/tf/empty.txt -> file:///tmp/t2/tf/empty.txt
Making directory dir=/tmp/t2/tf

$ rm -rf /tmp/t2
$ ./s3-cli sync -v s3://koblas/test_folder/ /tmp/t2
1 files to consider - 0 bytes
Copy s3://koblas/test_folder/tf/empty.txt -> file:///tmp/t2/tf/empty.txt
Making directory dir=/tmp/t2/tf

$ rm -rf /tmp/t2
$ ./s3-cli sync -v s3://koblas/test_folder /tmp/t2
1 files to consider - 0 bytes
Copy s3://koblas/test_folder/tf/empty.txt -> file:///tmp/t2/test_folder/tf/empty.txt
Making directory dir=/tmp/t2/test_folder/tf

$ rm -rf /tmp/t2
$ ./s3-cli sync -v s3://koblas/test_folder /tmp/t2/
1 files to consider - 0 bytes
Copy s3://koblas/test_folder/tf/empty.txt -> file:///tmp/t2/test_folder/tf/empty.txt
Making directory dir=/tmp/t2/test_folder/tf

[YJ-AllCloud]

This is the command I ran on windows:
C:\>s3-cli.exe sync -v s3://cloud-test .

Output:

Copy s3://cloud-test/test1/test1.txt -> file://./est1/test1.txt
Copy s3://cloud-test/test1/ -> file://./est1/
Copy s3://cloud-test/test2/ -> file://./est2/
0 B / 5 B (0.0%)   0 B/sec5 files to consider - 10 bytes
Copy s3://cloud-test/test2/test2.txt -> file://./est2/test2.txt
Remove file://s3-cli.exe
0 B / 10 B (0.0%)   0 B/secMaking directory dir=est1
5 B / 10 B (50.0%)   2 B/secMaking directory dir=est2
open ./est2/: is a directory
10 B / 10 B (100.0%)   4 B/sec

[koblas]

Ah! Windows, I think I know the source of the bug, going to take a few days to fix it and find a windows box to test it on.

[koblas]

The missing character issue is now fixed, though I think there may still be an issue with PC based paths and '/' vs ''

[juilin]

Hi koblas,

The problem can be reproduced with the latest codes in my ubuntu platform. Please refer the console logs below.

`
(no files at local)
$~/tests3cmd$ ls -al
total 8
drwxrwxr-x 2 csiuser csiuser 4096 Dec 1 15:54 .
drwxr-xr-x 24 csiuser csiuser 4096 Dec 1 15:52 ..

(check files at S3)
$~/tests3cmd$ ~/s3-cli/s3-cli ls -r s3://my-first-backup-bucket-1298f
2017-12-01 07:42 18 s3://my-first-backup-bucket-1298f/fakedata.zip
2017-12-01 07:42 5 s3://my-first-backup-bucket-1298f/fakedata.zip.1
2017-12-01 07:42 5 s3://my-first-backup-bucket-1298f/fakedata.zip.2
2017-12-01 07:42 5 s3://my-first-backup-bucket-1298f/fakedata.zip.3
2017-12-01 07:42 7 s3://my-first-backup-bucket-1298f/subfolder/sub1
2017-12-01 07:42 7 s3://my-first-backup-bucket-1298f/subfolder/sub2

(sync data from S3)
$~/tests3cmd$ ~/s3-cli/s3-cli sync s3://my-first-backup-bucket-1298f .
47 B / 47 B (100.0%) 6 B/sec

(List local files and found first character is gone!)
$~/tests3cmd$ ls -al
total 28
drwxrwxr-x 3 csiuser csiuser 4096 Dec 1 16:59 .
drwxr-xr-x 24 csiuser csiuser 4096 Dec 1 15:52 ..
-rw-rw-r-- 1 csiuser csiuser 18 Dec 1 16:59 akedata.zip
-rw-rw-r-- 1 csiuser csiuser 5 Dec 1 16:59 akedata.zip.1
-rw-rw-r-- 1 csiuser csiuser 5 Dec 1 16:59 akedata.zip.2
-rw-rw-r-- 1 csiuser csiuser 5 Dec 1 16:59 akedata.zip.3
drwxr-xr-x 2 csiuser csiuser 4096 Dec 1 16:59 ubfolder

(clean local files)
$~/tests3cmd$ rm -rf *

(sync subfolder to local)
$~/tests3cmd$ ~/s3-cli/s3-cli sync s3://my-first-backup-bucket-1298f/subfolder .
714 B / 14 B (100.0%) 11 B/sec

(looks good)
$~/tests3cmd$ ls -al subfolder/
total 16
drwxr-xr-x 2 csiuser csiuser 4096 Dec 1 16:59 .
drwxrwxr-x 3 csiuser csiuser 4096 Dec 1 16:59 ..
-rw-rw-r-- 1 csiuser csiuser 7 Dec 1 16:59 sub1
-rw-rw-r-- 1 csiuser csiuser 7 Dec 1 16:59 sub2

`

Thanks,
juilin

[gavv]

Still reproducing on Linux (debian).