MICCAI 2020 Combined Radiology and Pathology Classification Challenge Forum

Go back to competition Back to thread list Post in this thread

> Did anyone get the mail?

Hi,has anyone received the email to submit Docker?

Posted by: Sen @ Sept. 6, 2020, 12:14 a.m.

Hi, no email, tbh I don't think they were sent yet

Posted by: marvinler @ Sept. 6, 2020, 9:46 a.m.

Hi all,
Just sent the emails out. Thank you all for participating. Also for those who get the email, the date for the docker submission has been extended to Sept 7th.

Posted by: bbearce @ Sept. 6, 2020, 4:45 p.m.

Only one day to prepare the docker image? the dealine is 7th? Is the time too limit?

Posted by: tabulo @ Sept. 7, 2020, 1:16 a.m.

I have never used Docker, could you give me a few more days to prepare?

Posted by: Sen @ Sept. 7, 2020, 7:56 a.m.

The same, Making docker in one day is too difficult for me

Posted by: tabulo @ Sept. 7, 2020, 8:13 a.m.

The same, Making docker in one day is too difficult for me.
Can you provide any useful materials about building a docker? thanks

Posted by: hanchu1989 @ Sept. 7, 2020, 11:35 a.m.

Organizers provided a template in the data repository with an example of Dockerfile and files for some processing.

You will need to install docker compatible with nvidia such that containers can reach GPUs (or you can perform CPU inference I guess): https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker
Some sources to help you...:
https://docs.docker.com/get-started/
https://docs.docker.com/get-started/part2/
https://medium.com/@zaher88abd/pytorch-with-docker-b791edd67850
https://levelup.gitconnected.com/docker-containers-for-deep-learning-77e8b7e10ef1

ML

Posted by: marvinler @ Sept. 7, 2020, 12:03 p.m.

hi,how do you install Opencv-python in docker,I keep reporting mistakes 'ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory'

Posted by: Sen @ Sept. 7, 2020, 5:17 p.m.

There are some trouble in my docket, can you extend the time?one day is very difficult for newer,thanks

Posted by: tabulo @ Sept. 7, 2020, 6:24 p.m.

All,

We want to see what is in your algorithms because you all are finalist. The deadline is mainly so we have time to run the docker submissions before MICCAI. I am willing to work with you all for the sake of science and transparency with your models. Please keep in mind I need them this week or else I may run out of time to run them all assuming inference takes a full day to run for each image.

Posted by: bbearce @ Sept. 7, 2020, 10:04 p.m.

For opencv-python, I was able to get this config to work:

```
FROM ubuntu:latest

RUN apt-get update -y
RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip
RUN pip3 install opencv-python

CMD ['echo','DONE']
```

and

```
FROM nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04

RUN apt-get update -y
RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip
RUN pip3 install opencv-python

CMD ['echo','DONE']
```
and
```
FROM nvidia/cuda:10.2-runtime-ubuntu18.04

RUN apt-get update -y
RUN apt-get install -y python3-pip
RUN pip3 install --upgrade pip
RUN pip3 install opencv-python

CMD ['echo','DONE']
```

I know I skipped cuda:10.1, but I bet it works too. Does this help?

Posted by: bbearce @ Sept. 7, 2020, 11:07 p.m.

I see,but can you extend the deadline to 8th?

Posted by: tabulo @ Sept. 7, 2020, 11:26 p.m.

I have a question, shall we submit docker files or Docker images

Posted by: Sen @ Sept. 8, 2020, 3:27 a.m.

With no network to connect to, how do you PIP Install OpencV-Python succeed

Posted by: Sen @ Sept. 8, 2020, 3:31 a.m.

Sen, when you are building a new docker image, you can use network to 'PIP install' all the package you want. 'No network' means no network on runtime I guess.

Posted by: hanchu1989 @ Sept. 8, 2020, 3:38 a.m.

I try again

Posted by: Sen @ Sept. 8, 2020, 3:44 a.m.

First supply images not Dockerfiles. Second, Sen is correct. When you build the image you are doing so locally and have internet. Later when I run the container (based on the image), that container instance will have everything you already loaded in, but will not have internet at runtime. the purpose of this is so that data cannot leave the environment where I run it.

Posted by: bbearce @ Sept. 8, 2020, 2:23 p.m.

Lastly,

The deadline can be fluid, but I would highly suggest submitting one by the 8th (end of today). The issue is if I cannot get these run and validated by the 16th and an image is late (by a lot), I might miss the deadline myself and then that means you miss the deadline. So I'm willing to work with others and be flexible, but last year some of these algorithms took a day to run for inference. Add on a couple unforeseen bugs and each image is re-run twice maybe 3 times till I get it right and that could be 2-3 days...so keep this in mind.

Most importantly this is for science and the community, so I'm willing to work with others and help build these images or at least guide you. Please keep posting issues and concerns.

Posted by: bbearce @ Sept. 8, 2020, 2:27 p.m.

I have this error:

AssertionError:
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx

On my own system I have NVIDIA driver and I can use the GPU. When I build the image and try to run the inference model there, I get the above error.
In my docker file I am using this: (FROM nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04)
Do you know how I can solve this error?

Posted by: azh2 @ Sept. 8, 2020, 8:12 p.m.

Can you look up what cuda version you have installed on your local environment. Can you use that inside the Dockerfile?

Posted by: bbearce @ Sept. 8, 2020, 8:14 p.m.

Also:
```
$ nvidia-smi
```
will tell you what cuda you are running:

bb927@bryce:~$ nvidia-smi
Tue Sep 8 16:14:38 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 435.21 Driver Version: 435.21 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:06:00.0 Off | 0 |
| N/A 56C P0 60W / 149W | 11385MiB / 11441MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K80 Off | 00000000:07:00.0 Off | 0 |
| N/A 42C P0 74W / 149W | 129MiB / 11441MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 Tesla K80 Off | 00000000:85:00.0 Off | 0 |
| N/A 62C P0 62W / 149W | 129MiB / 11441MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 Tesla K80 Off | 00000000:86:00.0 Off | 0 |
| N/A 46C P0 73W / 149W | 129MiB / 11441MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 3149 C /usr/bin/python3.6 10952MiB |
| 0 8955 C /usr/bin/python3.6 419MiB |
| 1 3149 C /usr/bin/python3.6 58MiB |
| 1 8955 C /usr/bin/python3.6 58MiB |
| 2 3149 C /usr/bin/python3.6 58MiB |
| 2 8955 C /usr/bin/python3.6 58MiB |
| 3 3149 C /usr/bin/python3.6 58MiB |
| 3 8955 C /usr/bin/python3.6 58MiB |
+-----------------------------------------------------------------------------+

Posted by: bbearce @ Sept. 8, 2020, 8:15 p.m.

In the Dockerfile the "FROM" statement is the OS\drivers it will come with. If you can try to match that with your local config that will be the best bet.

Go here: https://hub.docker.com/r/nvidia/cuda/tags

They have centos\ubuntu images with different cuda versions. Also if you installed cuda from scratch on your local machine and that worked well, you can put all of those steps in the docker file and make it truly custom. You can base images on centos or ubuntu at that point and go from there. Let me know if it still does not work.

Posted by: bbearce @ Sept. 8, 2020, 8:30 p.m.

Where did you submit your Docker?docker hub?

Posted by: Sen @ Sept. 10, 2020, 2:12 p.m.
Post in this thread