When we run crusher on hive directory, it works like charm for first time.But for the next run, It cant move the original files. So we end up having both crushed and original files in hive directory. The problem attributes to the way crusher moves files after being crushed.As we already have original files put in destination after first run and hive creates part files starting with 00000-0 again, We end up having same name files in source directory as we have them in the destination directory(Due to first run). thus due to name conflict , Move operation fails.
When we run crusher on hive directory, it works like charm for first time.But for the next run, It cant move the original files. So we end up having both crushed and original files in hive directory. The problem attributes to the way crusher moves files after being crushed.As we already have original files put in destination after first run and hive creates part files starting with 00000-0 again, We end up having same name files in source directory as we have them in the destination directory(Due to first run). thus due to name conflict , Move operation fails.