Skip to content

VR is stuck at Starting #8201

Closed as not planned
Closed as not planned
VR is stuck at Starting#8201
Labels
component:networkingcomponent:virtual-routercomponent:vpcstatus:needs-investigationstatus:needs-reproducing
Milestone
unplanned

Description

@weizhouapache
weizhouapache
opened on Nov 8, 2023

This could be same as #8055

When VR is started, there should be some logs like

2023-11-07 21:22:01,197 DEBUG [c.c.h.v.r.VmwareResource] (DirectAgent-356:ctx-dc3231a7 10.0.34.0, job-745/job-746, cmd: AggregationControlCommand) (logid:107ab608) Run command on VR: 10.0.41.155, script: vr_cfg.sh with args: -c /var/cache/cloud/VR-14e05e67-d677-4a3d-9adc-72e30b19ea3c.cfg

2023-11-07 21:22:01,827 DEBUG [c.c.h.v.r.VmwareResource] (DirectAgent-356:ctx-dc3231a7 10.0.34.0, job-745/job-746, cmd: AggregationControlCommand) (logid:107ab608) vr_cfg.sh execution result: true

However, sometimes VR is stuck at Starting, in that case, there is only first log, but no second log. There are many (~100) firewall rules /port forwarding rules with the network. The vr_cfg.sh seems to be executed without error (checked /var/log/cloud.log).

The issue seems to be caused by

cloudstack/plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Lines 1854 to 1855 in 3d7281d

result = SshHelper.sshExecute(routerIP, DefaultDomRSshPort, "root", getSystemVmKeyFile(), null, "/opt/cloud/bin/" + script + " " + args,
VRScripts.CONNECTION_TIMEOUT, VRScripts.CONNECTION_TIMEOUT, timeout);

this issue needs to be reproduced and the fix needs to be verified.

ISSUE TYPE
  • Bug Report
COMPONENT NAME
VR
CLOUDSTACK VERSION
4.17.2.0
CONFIGURATION
OS / ENVIRONMENT

vmware

SUMMARY
STEPS TO REPRODUCE
EXPECTED RESULTS
ACTUAL RESULTS

Activity

  1. DaanHoogland
    added
    component:vpc
    component:virtual-router
    component:networking
    on Nov 9, 2023
  2. DaanHoogland
    added this to the 4.18.2.0 milestone on Nov 9, 2023
  3. weizhouapache
    added
    status:needs-reproducing
    status:needs-investigation
    on Nov 9, 2023

  4. weizhouapache commented on Dec 19, 2023

    @weizhouapache
    weizhouapache
    on Dec 19, 2023
    MemberAuthor

    it looks we have faced the same issue today

    2023-12-19 19:22:28,043 DEBUG [c.c.h.v.r.VmwareResource] (DirectAgent-1001:ctx-10021003 kvm2.local, job-10021004/job-10021005, cmd: CheckSshCommand) (logid:981bae10) Run command on VR: x.x.x.x, script: netusage.sh with args: -c
2023-12-19 19:22:28,286 DEBUG [c.c.h.v.r.VmwareResource] (DirectAgent-1001:ctx-10021003 kvm2.local, job-10021004/job-10021005, cmd: CheckSshCommand) (logid:981bae10) netusage.sh execution result: true
  5. JoaoJandre
    modified the milestones: 4.18.2.0, 4.18.3 on Mar 21, 2024

  6. DaanHoogland commented on Apr 3, 2024

    @DaanHoogland
    DaanHoogland
    on Apr 3, 2024
    Contributor

    @JoaoJandre , this one was already merged; setting the milestone back to 4.18.2

  7. DaanHoogland
    modified the milestones: 4.18.3, 4.18.2.0 on Apr 3, 2024

  8. weizhouapache commented on Apr 3, 2024

    @weizhouapache
    weizhouapache
    on Apr 3, 2024
    MemberAuthor

    @DaanHoogland
    they are not same issue.
    let's close this one, and reopen it if it happens again

  9. weizhouapache
    closed this as completedon Apr 3, 2024
  10. weizhouapache
    modified the milestones: 4.18.2.0, unplanned on Apr 3, 2024
  11. weizhouapache
    closed this as not plannedon Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:networkingcomponent:virtual-routercomponent:vpcstatus:needs-investigationstatus:needs-reproducing

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions