Detailed procedures of using the program with alibaba cloud

[Boyuan-Xu]

The program have great power in constructing interface configurations, however, for those who are not so familiar with alibaba cloud the tutorial seems to be too simplified. Haven't suceeded in executing the code of optimization for rigid body traslation and beyond even with docker installed and account registered. Would you mind providing more detailed procedures of using your program with the docker image you mentioned for rigid body traslation?

[jinlhr542]

Thank you for your interests in InterOptimus. I have revised README with more details for using the Alibaba Cloud and for the docker settings. Could you please try again?

[ffshy]

Hello dear developers of InterOptimus,
I also find the usage is quite tricky, so I come here for help. I have followed the steps in README, installed InterOptimus and docker, and created a docker namespace in my Alibaba Cloud account. But when I run the tutorial code in jupyter-notebook, the program seems to get stuck in the following piece of code (in building a DockerSocker object):

      while True:
           logs = self.container.logs()
           if b"Listening at:" in logs:  # 修改为更准确的关键字
               print("Flask service is ready.")
               break
           if time.time() - start_time > timeout:
               print("Timeout waiting for Flask app to start.")
               break
           time.sleep(2)

It turns out that the logs varibale is b'exec /usr/local/bin/gunicorn: exec format error\n', which means that we are trying to run an ARM-architecture container on a Linux machine based on x86_64. I checked the architecture of the container by the following code,

docker inspect --format='{{.Architecture}}' crpi-aqvrppj8ebnguc34.cn-shenzhen.personal.cr.aliyuncs.com/jinlhr542/orb-models:0.0.1

and the result is arm64. However, I think that most of our normally used Linux clusters or workstations are based on x86 (checked by running uname -a).

So is there any way of running this container on my x86_64 linux machine? Or is there other versions of containers? I hope you could help me with the actual usage of InterOptimus.

Any help would be greatly appreciated. Thank you!

[jinlhr542]

Hello dear developers of InterOptimus, I also find the usage is quite tricky, so I come here for help. I have followed the steps in README, installed InterOptimus and docker, and created a docker namespace in my Alibaba Cloud account. But when I run the tutorial code in jupyter-notebook, the program seems to get stuck in the following piece of code (in building a DockerSocker object):

  while True:
       logs = self.container.logs()
       if b"Listening at:" in logs:  # 修改为更准确的关键字
           print("Flask service is ready.")
           break
       if time.time() - start_time > timeout:
           print("Timeout waiting for Flask app to start.")
           break
       time.sleep(2)

It turns out that the logs varibale is b'exec /usr/local/bin/gunicorn: exec format error\n', which means that we are trying to run an ARM-architecture container on a Linux machine based on x86_64. I checked the architecture of the container by the following code,

docker inspect --format='{{.Architecture}}' crpi-aqvrppj8ebnguc34.cn-shenzhen.personal.cr.aliyuncs.com/jinlhr542/orb-models:0.0.1
and the result is arm64. However, I think that most of our normally used Linux clusters or workstations are based on x86 (checked by running uname -a).

So is there any way of running this container on my x86_64 linux machine? Or is there other versions of containers? I hope you could help me with the actual usage of InterOptimus.

Any help would be greatly appreciated. Thank you!

Many thanks for your finding this issue. I have built images with amd64 platforms for orb-model. Please uninstall mlipdockers by pip uninstall mlipdockers and reinstall pip install mlipdockers and use orb-models-amd64 instead of orb-models in the notebook.

Jason

[Boyuan-Xu]

Thank you for your interests in InterOptimus. I have revised README with more details for using the Alibaba Cloud and for the docker settings. Could you please try again?

Thank you for the implements, I have successfully run the code on my windows laptop. As a traditional HPC user who do not hold any ideas/resources about instances, I turned to docker for windows instead of using docker in HPC (inaccessible to non-root users).
Here are some tips for Windows user like me:

1. install Docker for Windows (http://mirrors.aliyun.com/docker-toolbox/windows/docker-for-windows/) and use mirror accelerator (https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors). For wsl2 usage, please install docker by referring to https://help.aliyun.com/zh/ecs/use-cases/install-and-use-docker#8dca4cfa3dn0e (use [sudo service docker start] command to start Docker)
2. Modify/Create C:\ProgramData\Docker\config\daemon.json file and type:
   {"registry-mirrors": ["https://[your_code].mirror.aliyuncs.com/"]}
   I also modified the following in Docker for Windows->Settings->Docker Engine just in case:
   "registry-mirrors": ["https://[your_code].mirror.aliyuncs.com",
   "http://mirrors.aliyun.com",
   "https://mirrors.tuna.tsinghua.edu.cn"].
   For wsl2 usage, check the options in Docker for Windows->Settings->Resources->WSL INTEGRATION
3. Open CMD as administrator and run docker login --username=[your_account your_specialized_ids], then you shall run the code and conduct the optimization jobs.

[jinlhr542]

Thank you for your interests in InterOptimus. I have revised README with more details for using the Alibaba Cloud and for the docker settings. Could you please try again?

Thank you for the implements, I have successfully run the code on my windows laptop. As a traditional HPC user who do not hold any ideas/resources about instances, I turned to docker for windows instead of using docker in HPC (inaccessible to non-root users). Here are some tips for Windows user like me:

1. install Docker for Windows (http://mirrors.aliyun.com/docker-toolbox/windows/docker-for-windows/) and use mirror accelerator (https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors). For wsl2 usage, please install docker by referring to https://help.aliyun.com/zh/ecs/use-cases/install-and-use-docker#8dca4cfa3dn0e (use [sudo service docker start] command to start Docker)
2. Modify/Create C:\ProgramData\Docker\config\daemon.json file and type:
   {"registry-mirrors": ["https://[your_code].mirror.aliyuncs.com/"]}
   I also modified the following in Docker for Windows->Settings->Docker Engine just in case:
   "registry-mirrors": ["https://[your_code].mirror.aliyuncs.com",
   "http://mirrors.aliyun.com",
   "https://mirrors.tuna.tsinghua.edu.cn"].
   For wsl2 usage, check the options in Docker for Windows->Settings->Resources->WSL INTEGRATION
3. Open CMD as administrator and run docker login --username=[your_account your_specialized_ids], then you shall run the code and conduct the optimization jobs.

Many thanks for your kind suggestion!

[ffshy]

Thanks for your help! However, I still have some problems. I updated the module as you suggested. When I change to the models with amd64 suffix, it turns out that I cannot get this image. The error is like

Not Found (\"pull access denied for crpi-aqvrppj8ebnguc34.cn-shenzhen.personal.cr.aliyuncs.com/jinlhr542/orb-models-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\")"

but when using the model without amd64 suffix, this problem disappears. All other models have the similar errors. So is there a problem with these newly created images? Maybe some authority issues?

[jinlhr542]

Thanks for your help! However, I still have some problems. I updated the module as you suggested. When I change to the models with amd64 suffix, it turns out that I cannot get this image. The error is like

Not Found (\"pull access denied for crpi-aqvrppj8ebnguc34.cn-shenzhen.personal.cr.aliyuncs.com/jinlhr542/orb-models-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\")"

but when using the model without amd64 suffix, this problem disappears. All other models have the similar errors. So is there a problem with these newly created images? Maybe some authority issues?

Please try it again. I forgot to set the new image to be 'public'. Sorry!

Jason

[jinlhr542]

Thanks for your help! However, I still have some problems. I updated the module as you suggested. When I change to the models with amd64 suffix, it turns out that I cannot get this image. The error is like

Not Found (\"pull access denied for crpi-aqvrppj8ebnguc34.cn-shenzhen.personal.cr.aliyuncs.com/jinlhr542/orb-models-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\")"

but when using the model without amd64 suffix, this problem disappears. All other models have the similar errors. So is there a problem with these newly created images? Maybe some authority issues?

Please try it again. I forgot to set the new image to be 'public'. Sorry!

Jason

[ffshy]

Hi, I tried again and the above error disappeared. However, for the MLIP orb-models-amd64, the code seems to get struck in

        self.container = client.containers.run(
            image_name,
            detach=True,
            ports={'5000/tcp': self.pt},
            mem_limit=f'{mem}g',  # 限制内存为 9GB
            cpu_quota=ncore * 100000,  # 限制为 12 个 CPU 核心
            cpu_period=100000
        )

in the construction function for DockerSocker object. And after checking output line by line, I find the code actually gets struck in self.client.images.pull(image, platform=platform) in the run function:

        try:
            container = self.create(image=image, command=command,
                                    detach=detach, **kwargs)
        except ImageNotFound:
            self.client.images.pull(image, platform=platform)
            container = self.create(image=image, command=command,
                                    detach=detach, **kwargs)
            print("create after pull")

For MLIP orb-models, the try block will succeed. So what else differences do these images have? By the way, different MLIP images seem to have different problems. For sevenn, it gets stuck in the same pull function as above. But for sevenn-amd64, I got the similar error message telling me that denied: requested access to the resource is denied".

Could you please help me work through this problem? Thanks a lot for your helpful suggestions now.

[jinlhr542]

Hi, I tried again and the above error disappeared. However, for the MLIP orb-models-amd64, the code seems to get struck in

    self.container = client.containers.run(
        image_name,
        detach=True,
        ports={'5000/tcp': self.pt},
        mem_limit=f'{mem}g',  # 限制内存为 9GB
        cpu_quota=ncore * 100000,  # 限制为 12 个 CPU 核心
        cpu_period=100000
    )

in the construction function for DockerSocker object. And after checking output line by line, I find the code actually gets struck in self.client.images.pull(image, platform=platform) in the run function:

    try:
        container = self.create(image=image, command=command,
                                detach=detach, **kwargs)
    except ImageNotFound:
        self.client.images.pull(image, platform=platform)
        container = self.create(image=image, command=command,
                                detach=detach, **kwargs)
        print("create after pull")

For MLIP orb-models, the try block will succeed. So what else differences do these images have? By the way, different MLIP images seem to have different problems. For sevenn, it gets stuck in the same pull function as above. But for sevenn-amd64, I got the similar error message telling me that denied: requested access to the resource is denied".

Could you please help me work through this problem? Thanks a lot for your helpful suggestions now.

Sorry for the problems. I found it is really troublesome to make the docker usage compatible in different machines. Therefore, I removed all the docker usages and made a new commit. Please try the newly updated package and the new tutorial. Only orb-models and sevenn are implemented in the new version which are compatible in the 3.12 python env and also the two most robust mlips according to our manuscript.

Jason