Dmitry Rubinstein  <dimrub@google.com>  #14 Apr 19, 2017 12:43AM

Hi Caroline!

I've brought the jobs back up again. As for the alternatives to Kharon, allow me to quote the Kharon deprecation email:

If you still need to move data corp <-> prod, here are your options:
1) Use periscope or uberproxy to reach into corp
2) Google Cloud Storage for non sensitive activities
3) Expose an api via corp.googleapis.com

Please let me know how soon will you able to migrate off Kharon.

About our timeline...I've been investigating alternatives, and it looks like we will be replacing Kharon with placer.  Since I'm new to placer, I'm guessing it will take 1-2 weeks to completely migrate off Kharon (it may be a lot faster, but I'd prefer to give you a conservative estimate).  Is that ok?

Added comment to the wrong issue.

My proposal is to use borg+borgron to copy the files.
Link to document https://docs.google.com/a/google.com/document/d/1HfEXDDtHKIYU7oaJ37KNS5Wy6wCt6bbyCVx4Qra8F80/edit?usp=sharing

The test files were copied to colossus successfully with borgcron.

https://sigma.corp.google.com/global/-/#job?cell=vk&collection_name=copy_chrotomation2_reports&collection_uid=82462458816&gadget_params=1:%7C2:onePerState-false;isAscendingSort-true;sortedColumnInt-0;currentPageNumber-0;statesNotSelected-_%7C0:%7C5:isAscendingSort-false;sortedColumnInt-0;currentPageNumber-0;compact-false%7C6:isAscendingSort-false;sortedColumnInt-0;currentPageNumber-0;compact-false&is_terminated=false&run_scheduling_simulation=false&user=manojgupta&workspace=manojgupta-default


Before:
manojgupta@manojgupta:/google/data/rw/users/mo/mobiletc-prebuild/copy_to_prod$ ls -l
total 8
drwxrwx--- 0 manojgupta mobiletc-prebuild-owners 4096 Apr 25 16:11 test1
drwxrwx--- 0 manojgupta mobiletc-prebuild-owners 4096 Apr 25 16:11 test2

fileutil ls -l /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2015/12/04 11:30:26 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2015
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2016/12/01 14:00:27 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2016
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2017/04/01 14:00:27 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2017
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2017/04/27 14:01:16 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/json_files


After borgcron job completion:

manojgupta@manojgupta:/google/data/rw/users/mo/mobiletc-prebuild/copy_to_prod$ fileutil ls -l /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2015/12/04 11:30:26 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2015
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2016/12/01 14:00:27 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2016
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2017/04/01 14:00:27 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/2017
drwxr-xr-x 1 cmtice    c-compiler-chrome            0 2017/04/27 14:01:16 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/json_files
drwxrwxr-x 1 manojgupta c-compiler-chrome            0 2017/04/27 14:27:56 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/test1
drwxrwxr-x 1 manojgupta c-compiler-chrome            0 2017/04/27 14:27:56 /cns/vk-d/home/c-compiler-chrome/kharon/mobiletc-prebuild/test2

manojgupta@manojgupta:/google/data/rw/users/mo/mobiletc-prebuild/copy_to_prod$ ls -l
total 0

test1 and test2 directories were moved to colossus correctly.

Borgcron based replacement is working now.