New issue
Advanced search Search tips

Issue 921143 link

Starred by 2 users
Status: Fixed
Owner:
garrick@chromium.org
Closed: Jan 18
Cc:
jkardatzke@chromium.org
smbar...@chromium.org
hugobenichi@google.com
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

"arc: refactor arc-networkd for multi-network support" breaks Crostini networking

Project Member Reported by dverkamp@chromium.org, Jan 11 
Crostini tests are failing after https://crrev.com/c/1401481 - reverting it makes the Crostini tests pass again.

https://stainless.corp.google.com/search?view=matrix&row=model&col=build&first_date=2019-01-05&last_date=2019-01-11&test=tast.vm.CrostiniFiles&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

The issue can be reproduced by running 'tast run $DUT vm.CrostiniFiles'.

The first visible problem is that hostname resolution inside the VM fails during the container download:

2019-01-11T07:13:13.928703-08:00 WARNING VM(3)[10103]:  lxd[134]: lvl=warn msg="Failed to update instance types: Get https://images.linuxcontainers.org/meta/instance-types/.yaml: lookup images.linuxcontainers.org: no such host" t=2019-01-11T15:13:12+0000#012
2019-01-11T07:13:13.972558-08:00 ERR vm_cicerone[10123]: Failed to create LXD container: failed to get alias: Get https://storage.googleapis.com/cros-containers/73/streams/v1/index.json: lookup storage.googleapis.com: no such host
2019-01-11T07:13:13.979391-08:00 INFO tast[5929]: vm.CrostiniFiles: Error at crostini_files.go:59: Running autotestPrivate.runCrostiniInstaller failed: got exception: Error: Error installing crostini

I'm unsure what the exact root cause is, since this is a large change.
Project Member

Comment 1 by bugdroid1@chromium.org, Jan 13 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/a5847c4e839ad1f6f027383a39f3a990d1499267

commit a5847c4e839ad1f6f027383a39f3a990d1499267
Author: Stephen Barber <smbarber@chromium.org>
Date: Sun Jan 13 03:26:00 2019

Revert "Reland "arc: networkd - introduce device manager class and package unit tests.""

This reverts commit bccae5067d4839b5a46786e744f9e0cc7839ea81.

Reason for revert: collides with crostini static IP range

BUG= chromium:921143 
TEST=crostini networking works again

Bug: b:77293260
Change-Id: I6ced3c4e4ff576b449439b9dbad7a7ffd0b19d23
Reviewed-on: https://chromium-review.googlesource.com/1407389
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>

[delete] https://crrev.com/dc523d4e8781cbafb8bc2a21a8d694f7766c8c52/arc/network/device_manager.cc
[delete] https://crrev.com/dc523d4e8781cbafb8bc2a21a8d694f7766c8c52/arc/network/device_manager.h
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/device.cc
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/device.h
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/BUILD.gn
[delete] https://crrev.com/dc523d4e8781cbafb8bc2a21a8d694f7766c8c52/arc/network/device_test.cc
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/ipc.proto
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/manager.cc
[delete] https://crrev.com/dc523d4e8781cbafb8bc2a21a8d694f7766c8c52/arc/network/device_manager_test.cc
[modify] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/manager.h
Project Member

Comment 2 by bugdroid1@chromium.org, Jan 13 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/f60249150a8cc9769296859005082a565bee886e

commit f60249150a8cc9769296859005082a565bee886e
Author: Stephen Barber <smbarber@chromium.org>
Date: Sun Jan 13 03:26:00 2019

Revert "arc: Refactor arc-networkd for multi-network support."

This reverts commit 082aaf931e02fd57304c0af57d6adf917c1c0361.

Reason for revert: collides with crostini static IP range

BUG= chromium:921143 
TEST=crostini networking works again

Bug: b:77293260
Change-Id: Ia73b89c9b139e0e936482d6b9e8844a96463d887
Reviewed-on: https://chromium-review.googlesource.com/1407815
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>

[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/main.cc
[delete] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/device.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/init/arc-network-bridge.conf
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/ip_helper.h
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/shill_client.h
[delete] https://crrev.com/a5847c4e839ad1f6f027383a39f3a990d1499267/arc/network/device.h
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/BUILD.gn
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/ipc.proto
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/shill_client.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/manager.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/arc_ip_config.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/ip_helper.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/helper_process.cc
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/helper_process.h
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/manager.h
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/init/arc-network.conf
[modify] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/arc_ip_config.h
[add] https://crrev.com/f60249150a8cc9769296859005082a565bee886e/arc/network/options.h

Comment 3 by garrick@google.com, Jan 13 

Sorry about that. It's a little surprising though. Where can I find out more about crostini's networking and which IPs it's reserving?

Comment 4 by garrick@google.com, Jan 13 

I'm guessing it's 100.115.92.5?

Comment 5 by dverkamp@chromium.org, Jan 14 

There is some documentation in this comment block: https://chromium.googlesource.com/chromiumos/platform2/+/64d7e0818cfdddfcdc38a38b408033fc49be8fa9/vm_tools/concierge/subnet_pool.cc#22

I'm not sure if there is a more authoritative source than this - smbarber@ should know better.

Comment 6 by smbar...@chromium.org, Jan 14 

Right now we assume we can use all IPs in 100.115.92.0/24 except for the 100.115.92.0/30 subnet. That assumption was valid until the multinetwork work :)

That comment is unfortunately our most authoritative source at the moment. I'll put together a Markdown doc that explains these IP allocations and put up a CL to reduce the number of subnets we're using.

Comment 7 by garrick@chromium.org, Jan 14 

Thanks! Sorry I should have given this proper attention earlier. 

It looks like we'll need to coordinate address vending sooner than anticipated but for now if it's possible to start your VM pool at .24 I can reland this as-is and we can figure out a better solution. Does that work?
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 18 (5 days ago) 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/4195531fe22e4a40092a3bb370cdf4075d85a519

commit 4195531fe22e4a40092a3bb370cdf4075d85a519
Author: Stephen Barber <smbarber@chromium.org>
Date: Fri Jan 18 04:20:28 2019

vm_tools: concierge: shrink subnets for multinetwork

ARC++ multinetwork is now using a larger portion of the /24. Shrink the VM pool
accordingly.

BUG= chromium:921143 
TEST=unittests

Change-Id: I8858dc06699f301e83e00b28257a8f0247fdca04
Reviewed-on: https://chromium-review.googlesource.com/1410064
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>

[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/maitred/network-config.textproto
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/concierge/subnet_pool.h
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/concierge/subnet_test.cc
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/concierge/termina_vm_test.cc
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/concierge/subnet_pool.cc
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/concierge/subnet_pool_test.cc
[modify] https://crrev.com/4195531fe22e4a40092a3bb370cdf4075d85a519/vm_tools/maitred/service_impl_test.cc

Comment 9 by smbar...@chromium.org, Jan 18 (4 days ago)

Status: Fixed (was: Assigned)
This issue should be fixed now.
►

Sign in to add a comment