Windows IoT Core 1803 update

[Hammatt]

• V2 issue

• OS version and architecture used: Windows 10 IoT Core 10.0.17134.48
  x86_64

• Version used: V2 microsoft/azureiotedge-agent:1.0.0-preview022

• V2 Docker version and containter type Windows Docker version 0.0.0-dev, build 26a2a459 (windows containers)

Description of the issue:

Last night, the IoT Core devices we've been developing on decided to update themselves to build 10.0.17134.48
Ever since this the iot edge modules have stopped starting up.
I have adjusted our in-house modules to build on microsoft/dotnet:2.0-runtime-nanoserver-1803 and they can now start up (by themselves).
However, I believe that I need somebody on microsofts end to either do something similar for the edge agent and hub modules themselves, or to tell me the secret trick to get it all working.
Thanks.

Note: modules still start up as normal on Windows 10 Pro, this only seems to be an issue for IoT Core

Console log of the issue:

Running iotedgectl start results in:
encountered an error during CreateContainer: failure in a Windows system call: The operating system of the container does not match the operating system of the host. (0xc0370101)

calling docker run on an in-house module built on microsoft/dotnet:2.0.0-runtime-nanoserver-1709 results in the same error,
running docker run on an in-house module re-built on microsoft/dotnet:2.0-runtime-nanoserver-1803 results in an error relating to the fact that it can't connect to the edge hub which is expected since the edge runtime modules will not start.

[aribeironovaes]

Hi Alexander, Have you tried to uninstall the edge runtime and installing again? (Doing an uninstall and then a setup again?) Let me know if it works, otherwise i will have to follow up with the windows iot team and on our next release (which shall be soon) we can see if see can update images) Let me know, Thanks Angelo Ribeiro

…

Sent from my phone

On May 9, 2018, at 7:05 PM, Alexander Hammatt ***@***.***> wrote: V2 issue OS version and architecture used: Windows 10 IoT Core 10.0.17134.48 x86_64 Version used: V2 microsoft/azureiotedge-agent:1.0.0-preview022 V2 Docker version and containter type Windows Docker version 0.0.0-dev, build 26a2a459 (windows containers) Description of the issue: Last night, the IoT Core devices we've been developing off decided to update themselves to build 10.0.17134.48 Ever since this the iot edge modules have stopped starting up. I have adjusted our in-house modules to build on microsoft/dotnet:2.0-runtime-nanoserver-1803 and they can now start up (by themselves). However, I believe that I need somebody on microsofts end to either do something similar for the edge agent and hub modules themselves, or to tell me the secret trick to get it all working. Thanks. Note: modules still start up as normal on Windows 10 Pro, this only seems to be an issue for IoT Core Console log of the issue: Running iotedgectl start results in: encountered an error during CreateContainer: failure in a Windows system call: The operating system of the container does not match the operating system of the host. (0xc0370101) calling docker run on an in-house module built on microsoft/dotnet:2.0.0-runtime-nanoserver-1709 results in the same error, running docker run on an in-house module re-built on microsoft/dotnet:2.0-runtime-nanoserver-1803 results in an error relating to the fact that it can't connect to the edge hub which is expected since the edge runtime modules will not start. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[Hammatt]

Hi @aribeironovaes ,

I have uninstalled the python component of the runtime with pip uninstall azure-iot-edge-runtime-ctl and re-installed with the script from Invoke-Expression (Invoke-WebRequest -useb https://aka.ms/iotedgewin) and I have ran docker system prune -a to remove everything, and re-ran the iotedgectl setup & login commands and ran into the same results.

sidenote:
in trying to do that I've encountered another issue.
When running Invoke-Expression (Invoke-WebRequest -useb https://aka.ms/iotedgewin)
to set it up, I get an error saying "Azure IoT Edge on Windows requires Windows Fall Creators Update a.k.a. RS3 (build 16299). The current Windows build is 17134. Please ensure that the current Windows build is 16299 to run Azure IoT Edge with Windows containers."

I have downloaded the script that the setup tries to run and it appears to me that it is checking for a hard equals on build 16299, I tried this on regular windows 10 too and it also failed there!
I modified the script to check for version 17134 instead and it seemed to run

Thanks!

[aribeironovaes]

Cool! So updating the iotedgectl script worked? I will let the team know and we shall fix on the next release. Thanks! Angelo Ribeiro

…

Sent from my phone

On May 9, 2018, at 7:36 PM, Alexander Hammatt ***@***.***> wrote: Hi @aribeironovaes , I have uninstalled the python component of the runtime with pip uninstall azure-iot-edge-runtime-ctl and re-installed with the script from Invoke-Expression (Invoke-WebRequest -useb https://aka.ms/iotedgewin) and I have ran docker system prune -a to remove everything, and re-ran the iotedgectl setup & login commands and ran into the same results. sidenote: in trying to do that I've encountered another issue. When running Invoke-Expression (Invoke-WebRequest -useb https://aka.ms/iotedgewin) to set it up, I get an error saying "Azure IoT Edge on Windows requires Windows Fall Creators Update a.k.a. RS3 (build 16299). The current Windows build is 17134. Please ensure that the current Windows build is 16299 to run Azure IoT Edge with Windows containers." I have downloaded the script that the setup tries to run and it appears to me that it is checking for a hard equals on build 16299, I tried this on regular windows 10 too and it also failed there! I modified the script to check for version 17134 instead and it seemed to run Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Hammatt]

@aribeironovaes
Updating the script made the script work, but the modules still don't start. Sorry if I was unclear, the script not running seems to be a separate & unrelated issue.

[aribeironovaes]

Thanks for reporting! We will work on that. Angelo Ribeiro

…

Sent from my phone

On May 9, 2018, at 7:59 PM, Alexander Hammatt ***@***.***> wrote: @aribeironovaes Updating the script made the script work, but the modules still don't start. Sorry if I was unclear, the script not running seems to be a separate & unrelated issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Hammatt]

Just a quick update on this: I'm actually seeing some issues on my Windows 10 Pro machine with 1803 on it now. Even though the modules do start up they don't seem to work properly.

I've created a new edge device in the edge hub with only one aditional module on it, microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview as per the tutorial.

I've followed the same troubleshooting steps that I outlined that I took on IoT Core.

Has anybody managed to either reproduce these issues or have it work for them on Windows 10 Pro and/or IoT Core?

Thanks

[Hammatt]

Here's an exert from the logs for the issue which I just mentioned. It just seems to print this for forever. This worked fine before 1803, therefore I've put it in this same issue - hope that's ok.

2018-05-17 15:51:47.336 +12:00 [INF] - Starting module management agent.
2018-05-17 15:51:47.604 +12:00 [INF] - Version - 1.0.0-preview022.11567621 (12a8e1bb63e619b17ca685efd470ad3f412034f4)
2018-05-17 15:51:47.604 +12:00 [INF] -
        █████╗ ███████╗██╗   ██╗██████╗ ███████╗
       ██╔══██╗╚══███╔╝██║   ██║██╔══██╗██╔════╝
       ███████║  ███╔╝ ██║   ██║██████╔╝█████╗
       ██╔══██║ ███╔╝  ██║   ██║██╔══██╗██╔══╝
       ██║  ██║███████╗╚██████╔╝██║  ██║███████╗
       ╚═╝  ╚═╝╚══════╝ ╚═════╝ ╚═╝  ╚═╝╚══════╝

 ██╗ ██████╗ ████████╗    ███████╗██████╗  ██████╗ ███████╗
 ██║██╔═══██╗╚══██╔══╝    ██╔════╝██╔══██╗██╔════╝ ██╔════╝
 ██║██║   ██║   ██║       █████╗  ██║  ██║██║  ███╗█████╗
 ██║██║   ██║   ██║       ██╔══╝  ██║  ██║██║   ██║██╔══╝
 ██║╚██████╔╝   ██║       ███████╗██████╔╝╚██████╔╝███████╗
 ╚═╝ ╚═════╝    ╚═╝       ╚══════╝╚═════╝  ╚═════╝ ╚══════╝

2018-05-17 15:51:47.754 +12:00 [INF] - Edge agent attempting to connect to IoT Hub via AMQP...
2018-05-17 15:51:48.643 +12:00 [INF] - Created persistent store at C:\Users\ContainerAdministrator\AppData\Local\Temp\edgeAgent
2018-05-17 15:51:50.285 +12:00 [INF] - Edge agent connected to IoT Hub via AMQP.
2018-05-17 15:51:52.663 +12:00 [INF] - Plan execution started for deployment 2
2018-05-17 15:51:52.691 +12:00 [INF] - Executing command: "Command Group: (
  [docker pull microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker create --name tempSensor microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker start tempSensor]
)"
2018-05-17 15:51:52.693 +12:00 [INF] - Executing command: "docker pull microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview"
2018-05-17 15:52:08.553 +12:00 [INF] - Executing command: "docker create --name tempSensor microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview"
2018-05-17 15:52:08.640 +12:00 [INF] - Executing command: "docker start tempSensor"
2018-05-17 15:56:09.075 +12:00 [ERR] - Executing command for operation ["start"] failed.
System.OperationCanceledException: The operation was canceled.
   at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Docker.DotNet.DockerClient.<MakeRequestAsync>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Docker.DotNet.ContainerOperations.<StartContainerAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.<ExecuteAsync>d__6.MoveNext() in /opt/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
2018-05-17 15:56:09.182 +12:00 [ERR] - Executing command for operation ["Command Group: (
  [docker pull microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker create --name tempSensor microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker start tempSensor]
)"] failed.
System.OperationCanceledException: The operation was canceled.
   at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Docker.DotNet.DockerClient.<MakeRequestAsync>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Docker.DotNet.ContainerOperations.<StartContainerAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.<ExecuteAsync>d__6.MoveNext() in /opt/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 66
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.<ExecuteAsync>d__5.MoveNext() in /opt/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 36
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.<ExecuteAsync>d__6.MoveNext() in /opt/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
2018-05-17 15:56:09.190 +12:00 [ERR] - Step failed in deployment 2, continuing execution. Failure when running command Command Group: (
  [docker pull microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker create --name tempSensor microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview]
  [docker start tempSensor]
)

[bibble235]

Any chance of an update or maybe a method to run Docker containers with hyperv

[bibble235]

Hi,

Please could someone update this, I have 4 machines which no longer work because of this issue. Happy to workaround it

Error I am seeing using 17134 is

[davihar]

Same issue with Windows Server 2019 preview.

Also, developing a module on Windows 10 1803 then deploy to Windows Server Core 1709 results in an error saying "operating system of the container does not match the operating system of the host". This makes sense for process isolation containers if the container isn't using 1709.

This link describes what is happening: https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility

I think developers need a way to configure the IoT Edge tooling with a target OS version so that the Docker build process creates a container that can run under process isolation on the specified OS version.

Until this is fixed, both Edge development and runtime computers need Windows Update disabled to ensure they remain 1709. Any that already updated need to be rebuilt to use 1709.

[veyalla]

@bibble235 @Hammatt Our GA version has RS4 support. Docs are updated and code is at github.com/azure/iotedge

Please give it a shot, and open an issue in the iotedge repo if needed.

[Hammatt]

Sorry, too late. already switched to Linux :)
feel free to close if this is stale.

[veyalla]

ok :)