Dec 14 20:06:34 xen-e sshd[24189]: Accepted publickey for root from 10.2.13.1 port 60535 ssh2
Dec 14 20:06:35 xen-e corosync[24218]:  [MAIN  ] Corosync Cluster Engine ('1.4.3'): started and ready to provide service.
Dec 14 20:06:35 xen-e corosync[24218]:  [MAIN  ] Corosync built-in features: nss
Dec 14 20:06:35 xen-e corosync[24218]:  [MAIN  ] Successfully configured openais services to load
Dec 14 20:06:35 xen-e corosync[24218]:  [MAIN  ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
Dec 14 20:06:35 xen-e corosync[24218]:  [TOTEM ] Initializing transport (UDP/IP Multicast).
Dec 14 20:06:35 xen-e corosync[24218]:  [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Dec 14 20:06:35 xen-e corosync[24218]:  [TOTEM ] The network interface [10.2.13.55] is now up.
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais cluster membership service B.01.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais event service B.01.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais checkpoint service B.01.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais availability management framework B.01.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais message service B.03.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais distributed locking service B.03.01
Dec 14 20:06:35 xen-e corosync[24218]:  [SERV  ] Service engine loaded: openais timer service A.01.01
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: process_ais_conf: Reading configure
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_init: Local handle: 5800670271769673730 for logging
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional logging options...
Dec 14 20:06:35 xen-e cib: [24223]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/hacluster
Dec 14 20:06:35 xen-e cib: [24223]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
Dec 14 20:06:35 xen-e cib: [24223]: WARN: retrieveCib: Cluster configuration not found: /var/lib/heartbeat/crm/cib.xml
Dec 14 20:06:35 xen-e cib: [24223]: WARN: readCibXmlFile: Primary configuration corrupt or unusable, trying backup...
Dec 14 20:06:35 xen-e cib: [24223]: WARN: readCibXmlFile: Continuing with an empty configuration.
Dec 14 20:06:35 xen-e cib: [24223]: info: validate_with_relaxng: Creating RNG parser context
Dec 14 20:06:35 xen-e cib: [24223]: info: startCib: CIB Initialization completed successfully
Dec 14 20:06:35 xen-e cib: [24223]: info: get_cluster_type: Cluster type is: 'openais'
Dec 14 20:06:35 xen-e cib: [24223]: notice: crm_cluster_connect: Connecting to cluster infrastructure: classic openais (with plugin)
Dec 14 20:06:35 xen-e cib: [24223]: info: init_ais_connection_classic: Creating connection to our Corosync plugin
Dec 14 20:06:35 xen-e stonith-ng: [24224]: info: Invoked: /usr/lib64/pacemaker/stonithd 
Dec 14 20:06:35 xen-e stonith-ng: [24224]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root
Dec 14 20:06:35 xen-e stonith-ng: [24224]: info: get_cluster_type: Cluster type is: 'openais'
Dec 14 20:06:35 xen-e stonith-ng: [24224]: notice: crm_cluster_connect: Connecting to cluster infrastructure: classic openais (with plugin)
Dec 14 20:06:35 xen-e stonith-ng: [24224]: info: init_ais_connection_classic: Creating connection to our Corosync plugin
Dec 14 20:06:35 xen-e lrmd: [24225]: info: max-children set to 4 (2 processors online)
Dec 14 20:06:35 xen-e lrmd: [24225]: info: enabling coredumps
Dec 14 20:06:35 xen-e lrmd: [24225]: info: Started.
Dec 14 20:06:35 xen-e attrd: [24226]: info: Invoked: /usr/lib64/pacemaker/attrd 
Dec 14 20:06:35 xen-e attrd: [24226]: notice: crm_cluster_connect: Connecting to cluster infrastructure: classic openais (with plugin)
Dec 14 20:06:35 xen-e pengine: [24227]: info: Invoked: /usr/lib64/pacemaker/pengine 
Dec 14 20:06:35 xen-e crmd: [24228]: info: Invoked: /usr/lib64/pacemaker/crmd 
Dec 14 20:06:35 xen-e crmd: [24228]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/hacluster
Dec 14 20:06:35 xen-e crmd: [24228]: notice: main: CRM Git Version: 77eeb099a504ceda05d648ed161ef8b1582c7daf
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found 'off' for option: debug
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Defaulting to 'off' for option: to_logfile
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found 'yes' for option: to_syslog
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found 'daemon' for option: syslog_facility
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_init: Local handle: 326700200653488131 for quorum
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: No additional configuration supplied for: quorum
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: No default for option: provider
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_init: Local handle: 6897264049088626692 for service
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: config_find_next: Processing additional service options...
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found '0' for option: ver
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Defaulting to 'pcmk' for option: clustername
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found 'yes' for option: use_logd
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: get_config_opt: Found 'yes' for option: use_mgmtd
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: pcmk_startup: CRM: Initialized
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] Logging: Initialized pcmk_startup
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: pcmk_startup: Maximum core file size is: 18446744073709551615
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: pcmk_startup: Service: 9
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: pcmk_startup: Local hostname: xen-e
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: pcmk_update_nodeid: Local node id: 923599370
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: update_member: Creating entry for node 923599370 born on 0
Dec 14 20:06:35 xen-e mgmtd: [24229]: info: Pacemaker-mgmt Hg Version: unknown
Dec 14 20:06:35 xen-e corosync[24218]:  [pcmk  ] info: update_member: 0x6a6fa0 Node 923599370 now known as xen-e (was: (null))
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node xen-e now has 1 quorum votes (was 0)
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node 923599370/xen-e is now: member
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24223 for process cib
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24224 for process stonith-ng
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24225 for process lrmd
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24226 for process attrd
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24227 for process pengine
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24228 for process crmd
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: spawn_child: Forked child 24229 for process mgmtd
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: Pacemaker Cluster Manager 1.1.7
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync extended virtual synchrony service
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync configuration service
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync cluster config database access v1.01
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync profile loading service
Dec 14 20:06:36 xen-e mgmtd: [24229]: info: Started.
Dec 14 20:06:36 xen-e corosync[24218]:  [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Dec 14 20:06:36 xen-e corosync[24218]:  [MAIN  ] Compatibility mode set to whitetank.  Using V1 and V2 of the synchronization engine.
Dec 14 20:06:36 xen-e cib: [24223]: info: init_ais_connection_classic: AIS connection established
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: init_ais_connection_classic: AIS connection established
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Recorded connection 0x6b0650 for cib/24223
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node xen-e now has process list: 00000000000000000000000000151312 (1381138)
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Sending membership update 0 to cib
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Recorded connection 0x6b4a30 for stonith-ng/24224
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Recorded connection 0x6b8d90 for attrd/24226
Dec 14 20:06:36 xen-e cib: [24223]: info: get_ais_nodeid: Server details: id=923599370 uname=xen-e cname=pcmk
Dec 14 20:06:36 xen-e cib: [24223]: info: init_ais_connection_once: Connection to 'classic openais (with plugin)': established
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_new_peer: Node xen-e now has id: 923599370
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_new_peer: Node 923599370 is now known as xen-e
Dec 14 20:06:36 xen-e cib: [24223]: info: cib_init: Starting cib mainloop
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: get_ais_nodeid: Server details: id=923599370 uname=xen-e cname=pcmk
Dec 14 20:06:36 xen-e cib: [24223]: info: ais_dispatch_message: Membership 0: quorum still lost
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: init_ais_connection_once: Connection to 'classic openais (with plugin)': established
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_update_peer: Node xen-e: id=923599370 state=member (new) addr=(null) votes=1 (new) born=0 seen=0 proc=00000000000000000000000000151312 (new)
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: crm_new_peer: Node xen-e now has id: 923599370
Dec 14 20:06:36 xen-e attrd: [24226]: notice: main: Starting mainloop...
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: crm_new_peer: Node 923599370 is now known as xen-e
Dec 14 20:06:36 xen-e stonith-ng: [24224]: notice: setup_cib: Watching for stonith topology changes
Dec 14 20:06:36 xen-e stonith-ng: [24224]: info: main: Starting stonith-ng mainloop
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Transitional membership event on ring 1289820: memb=0, new=0, lost=0
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.54) 
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.54) 
Dec 14 20:06:36 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Stable membership event on ring 1289820: memb=2, new=2, lost=0
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Creating entry for node 906822154 born on 1289820
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node 906822154/unknown is now: member
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: NEW:  .pending. 906822154
Dec 14 20:06:36 xen-e cib: [24223]: info: ais_dispatch_message: Membership 1289820: quorum still lost
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: NEW:  xen-e 923599370
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: MEMB: .pending. 906822154
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: MEMB: xen-e 923599370
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: send_member_notification: Sending membership update 1289820 to 1 children
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: 0x6a6fa0 Node 923599370 ((null)) born on: 1289820
Dec 14 20:06:36 xen-e corosync[24218]:  [TOTEM ] A processor joined or left the membership and a new membership was formed.
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_new_peer: Node <null> now has id: 906822154
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_update_peer: Node (null): id=906822154 state=member (new) addr=r(0) ip(10.2.13.54)  votes=0 born=0 seen=1289820 proc=00000000000000000000000000000000
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_update_peer: Node xen-e: id=923599370 state=member addr=r(0) ip(10.2.13.55)  (new) votes=1 born=0 seen=1289820 proc=00000000000000000000000000151312
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: 0x6aeff0 Node 906822154 (xen-d) born on: 1289820
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: 0x6aeff0 Node 906822154 now known as xen-d (was: (null))
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node xen-d now has process list: 00000000000000000000000000151312 (1381138)
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node xen-d now has 1 quorum votes (was 0)
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: send_member_notification: Sending membership update 1289820 to 1 children
Dec 14 20:06:36 xen-e cib: [24223]: notice: ais_dispatch_message: Membership 1289820: quorum acquired
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_get_peer: Node 906822154 is now known as xen-d
Dec 14 20:06:36 xen-e cib: [24223]: info: crm_update_peer: Node xen-d: id=906822154 state=member addr=r(0) ip(10.2.13.54)  votes=1 (new) born=1289820 seen=1289820 proc=00000000000000000000000000151312 (new)
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] WARN: route_ais_message: Sending message to local.crmd failed: ipc delivery failed (rc=-2)
Dec 14 20:06:36 xen-e corosync[24218]:  [CPG   ] chosen downlist: sender r(0) ip(10.2.13.54) ; members(old:1 left:0)
Dec 14 20:06:36 xen-e corosync[24218]:  [MAIN  ] Completed service synchronization, ready to provide service.
Dec 14 20:06:36 xen-e crmd: [24228]: info: do_cib_control: CIB connection established
Dec 14 20:06:36 xen-e crmd: [24228]: info: get_cluster_type: Cluster type is: 'openais'
Dec 14 20:06:36 xen-e crmd: [24228]: notice: crm_cluster_connect: Connecting to cluster infrastructure: classic openais (with plugin)
Dec 14 20:06:36 xen-e crmd: [24228]: info: init_ais_connection_classic: Creating connection to our Corosync plugin
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Recorded connection 0x6bfce0 for crmd/24228
Dec 14 20:06:36 xen-e corosync[24218]:  [pcmk  ] info: pcmk_ipc: Sending membership update 1289820 to crmd
Dec 14 20:06:36 xen-e crmd: [24228]: info: init_ais_connection_classic: AIS connection established
Dec 14 20:06:36 xen-e crmd: [24228]: info: get_ais_nodeid: Server details: id=923599370 uname=xen-e cname=pcmk
Dec 14 20:06:36 xen-e crmd: [24228]: info: init_ais_connection_once: Connection to 'classic openais (with plugin)': established
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_new_peer: Node xen-e now has id: 923599370
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_new_peer: Node 923599370 is now known as xen-e
Dec 14 20:06:36 xen-e crmd: [24228]: info: ais_status_callback: status: xen-e is now unknown
Dec 14 20:06:36 xen-e crmd: [24228]: info: do_ha_control: Connected to the cluster
Dec 14 20:06:36 xen-e crmd: [24228]: info: do_started: Delaying start, no membership data (0000000000100000)
Dec 14 20:06:36 xen-e crmd: [24228]: notice: ais_dispatch_message: Membership 1289820: quorum acquired
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_new_peer: Node xen-d now has id: 906822154
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_new_peer: Node 906822154 is now known as xen-d
Dec 14 20:06:36 xen-e crmd: [24228]: info: ais_status_callback: status: xen-d is now unknown
Dec 14 20:06:36 xen-e crmd: [24228]: info: ais_status_callback: status: xen-d is now member (was unknown)
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_update_peer: Node xen-d: id=906822154 state=member (new) addr=r(0) ip(10.2.13.54)  votes=1 born=1289820 seen=1289820 proc=00000000000000000000000000151312
Dec 14 20:06:36 xen-e crmd: [24228]: notice: crmd_peer_update: Status update: Client xen-e/crmd now has status [online] (DC=<null>)
Dec 14 20:06:36 xen-e crmd: [24228]: info: ais_status_callback: status: xen-e is now member (was unknown)
Dec 14 20:06:36 xen-e crmd: [24228]: info: crm_update_peer: Node xen-e: id=923599370 state=member (new) addr=r(0) ip(10.2.13.55)  (new) votes=1 (new) born=1289820 seen=1289820 proc=00000000000000000000000000151312 (new)
Dec 14 20:06:36 xen-e crmd: [24228]: info: do_started: Delaying start, Config not read (0000000000000040)
Dec 14 20:06:36 xen-e crmd: [24228]: notice: do_started: The local CRM is operational
Dec 14 20:06:36 xen-e crmd: [24228]: notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]
Dec 14 20:06:36 xen-e lrmd: [24225]: info: max-children already set to 4
Dec 14 20:06:37 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_election_count_vote: Election 2 (owner: xen-d) pass: vote from xen-d (Uptime)
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_readwrite: We are now in R/W mode
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_master for section 'all' (origin=local/crmd/5, version=0.0.1): ok (rc=0)
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_te_control: Registering TE UUID: 508011b9-a634-4079-959b-a66f62bb6df2
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/6, version=0.0.2): ok (rc=0)
Dec 14 20:06:57 xen-e crmd: [24228]: info: set_graph_functions: Setting custom graph functions
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="0" num_updates="2" />
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_dc_takeover: Taking over DC status for this partition
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + <cib epoch="1" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" >
Dec 14 20:06:57 xen-e crmd: [24228]: info: join_make_offer: Making join offers based on membership 1289820
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     <crm_config >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" __crm_diff_marker__="added:top" >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.7-77eeb099a504ceda05d648ed161ef8b1582c7daf" />
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       </cluster_property_set>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     </crm_config>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/9, version=0.1.1): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="1" num_updates="1" />
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + <cib epoch="2" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="xen-e" update-client="crmd" update-user="hacluster" cib-last-written="Fri Dec 14 20:06:57 2012" >
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_dc_join_offer_all: join-1: Waiting on 2 outstanding join acks
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     <crm_config >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="openais" __crm_diff_marker__="added:top" />
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       </cluster_property_set>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     </crm_config>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:06:57 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/12, version=0.2.1): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="2" num_updates="1" />
Dec 14 20:06:57 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + <cib epoch="3" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="xen-e" update-client="crmd" update-user="hacluster" cib-last-written="Fri Dec 14 20:06:57 2012" >
Dec 14 20:06:57 xen-e crmd: [24228]: info: update_dc: Set DC to xen-e (3.0.6)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:06:57 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     <crm_config >
Dec 14 20:06:57 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-expected-quorum-votes" name="expected-quorum-votes" value="2" __crm_diff_marker__="added:top" />
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       </cluster_property_set>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     </crm_config>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:06:57 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/16, version=0.3.1): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/20, version=0.3.2): ok (rc=0)
Dec 14 20:06:57 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_dc_join_finalize: join-1: Syncing the CIB from xen-e to the rest of the cluster
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/22, version=0.3.3): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/23, version=0.3.3): ok (rc=0)
Dec 14 20:06:57 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-e']/transient_attributes
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="3" num_updates="3" />
Dec 14 20:06:57 xen-e attrd: [24226]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Dec 14 20:06:57 xen-e crmd: [24228]: info: update_attrd: Connecting to attrd...
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + <cib epoch="4" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="xen-e" update-client="crmd" update-user="hacluster" cib-last-written="Fri Dec 14 20:06:57 2012" >
Dec 14 20:06:57 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Dec 14 20:06:57 xen-e pengine: [24227]: ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_dc_join_ack: join-1: Updating node state to member for xen-e
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:06:57 xen-e attrd: [24226]: notice: attrd_perform_update: Sent update 4: probe_complete=true
Dec 14 20:06:57 xen-e pengine: [24227]: ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
Dec 14 20:06:57 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-e']/lrm
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     <nodes >
Dec 14 20:06:57 xen-e pengine: [24227]: ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_dc_join_ack: join-1: Updating node state to member for xen-d
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       <node id="xen-d" uname="xen-d" type="normal" __crm_diff_marker__="added:top" />
Dec 14 20:06:57 xen-e pengine: [24227]: notice: stage6: Delaying fencing operations until there are resources to manage
Dec 14 20:06:57 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/lrm
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     </nodes>
Dec 14 20:06:57 xen-e pengine: [24227]: notice: process_pe_message: Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-43.bz2
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:06:57 xen-e pengine: [24227]: notice: process_pe_message: Configuration ERRORs found during PE processing.  Please run "crm_verify -L" to identify issues.
Dec 14 20:06:57 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:162 - Triggered transition abort (complete=1) : Peer Cancelled
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/24, version=0.4.1): ok (rc=0)
Dec 14 20:06:57 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 0 (ref=pe_calc-dc-1355512017-9) derived from /var/lib/pengine/pe-input-43.bz2
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="4" num_updates="1" />
Dec 14 20:06:57 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 3: probe_complete probe_complete on xen-e (local) - no waiting
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + <cib epoch="5" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="xen-e" update-client="crmd" update-user="hacluster" cib-last-written="Fri Dec 14 20:06:57 2012" >
Dec 14 20:06:57 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 2: probe_complete probe_complete on xen-d - no waiting
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:06:57 xen-e crmd: [24228]: notice: run_graph: ==== Transition 0 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-43.bz2): Complete
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     <nodes >
Dec 14 20:06:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +       <node id="xen-e" uname="xen-e" type="normal" __crm_diff_marker__="added:top" />
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +     </nodes>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/25, version=0.5.1): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-e']/transient_attributes (origin=local/crmd/26, version=0.5.2): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-e']/lrm (origin=local/crmd/27, version=0.5.3): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/transient_attributes (origin=xen-d/crmd/8, version=0.5.5): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/lrm (origin=local/crmd/29, version=0.5.6): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/31, version=0.5.8): ok (rc=0)
Dec 14 20:06:57 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/33, version=0.5.10): ok (rc=0)
Dec 14 20:07:16 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:16 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_replace_notify: Replaced: 0.5.14 -> 0.6.1 from xen-d
Dec 14 20:07:19 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Dec 14 20:07:19 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:126 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.6.1) : Non-status change
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: - <cib admin_epoch="0" epoch="5" num_updates="14" />
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: + <cib epoch="6" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="xen-e" update-client="crmd" update-user="hacluster" cib-last-written="Fri Dec 14 20:06:57 2012" have-quorum="1" dc-uuid="xen-e" >
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +   <configuration >
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     <crm_config >
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_dc_takeover: Taking over DC status for this partition
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="true" __crm_diff_marker__="added:top" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore" __crm_diff_marker__="added:top" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-startup-fencing" name="startup-fencing" value="false" __crm_diff_marker__="added:top" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       </cluster_property_set>
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     </crm_config>
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     <resources >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       <primitive class="ocf" id="d1" provider="heartbeat" type="Dummy" __crm_diff_marker__="added:top" >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <operations >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +           <op id="d1-monitor-5s" interval="5s" name="monitor" timeout="30s" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         </operations>
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       </primitive>
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       <primitive class="stonith" id="s-libvirt" type="external/libvirt" __crm_diff_marker__="added:top" >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <instance_attributes id="s-libvirt-instance_attributes" >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +           <nvpair id="s-libvirt-instance_attributes-hostlist" name="hostlist" value="xen-d xen-e" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +           <nvpair id="s-libvirt-instance_attributes-hypervisor_uri" name="hypervisor_uri" value="xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +           <nvpair id="s-libvirt-instance_attributes-reset_method" name="reset_method" value="reboot" />
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_dc_join_offer_all: join-2: Waiting on 2 outstanding join acks
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         </instance_attributes>
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         <operations >
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +           <op id="s-libvirt-monitor-5s" interval="5s" name="monitor" timeout="60s" />
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +         </operations>
Dec 14 20:07:19 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       </primitive>
Dec 14 20:07:19 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     </resources>
Dec 14 20:07:19 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     <constraints >
Dec 14 20:07:19 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       <rsc_location id="d1-pref" node="xen-d" rsc="d1" score="100" __crm_diff_marker__="added:top" />
Dec 14 20:07:19 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289820: quorum retained
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +       <rsc_location id="s-pref" node="xen-e" rsc="s-libvirt" score="100" __crm_diff_marker__="added:top" />
Dec 14 20:07:19 xen-e attrd: [24226]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Dec 14 20:07:19 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +     </constraints>
Dec 14 20:07:19 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Dec 14 20:07:19 xen-e crmd: [24228]: info: update_dc: Set DC to xen-e (3.0.6)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: +   </configuration>
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib:diff: + </cib>
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_dc_join_finalize: join-2: Syncing the CIB from xen-e to the rest of the cluster
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_replace for section 'all' (origin=xen-d/cibadmin/2, version=0.6.1): ok (rc=0)
Dec 14 20:07:19 xen-e lrmd: [24225]: info: rsc:d1 probe[2] (pid 24243)
Dec 14 20:07:19 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_dc_join_ack: join-2: Updating node state to member for xen-e
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/35, version=0.6.2): ok (rc=0)
Dec 14 20:07:19 xen-e lrmd: [24225]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
Dec 14 20:07:19 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:19 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-e']/lrm
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_master for section 'all' (origin=local/crmd/39, version=0.6.4): ok (rc=0)
Dec 14 20:07:19 xen-e lrmd: [24225]: info: rsc:s-libvirt probe[3] (pid 24244)
Dec 14 20:07:19 xen-e pengine: [24227]: notice: LogActions: Start   d1	(xen-d)
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_dc_join_ack: join-2: Updating node state to member for xen-d
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync_one for section 'all' (origin=xen-d/xen-d/(null), version=0.6.4): ok (rc=0)
Dec 14 20:07:19 xen-e pengine: [24227]: notice: LogActions: Start   s-libvirt	(xen-e)
Dec 14 20:07:19 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/lrm
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/40, version=0.6.5): ok (rc=0)
Dec 14 20:07:19 xen-e pengine: [24227]: notice: process_pe_message: Transition 1: PEngine Input stored in: /var/lib/pengine/pe-input-44.bz2
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/42, version=0.6.7): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:162 - Triggered transition abort (complete=1) : Peer Cancelled
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/44, version=0.6.8): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/47, version=0.6.11): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 1 (ref=pe_calc-dc-1355512039-19) derived from /var/lib/pengine/pe-input-44.bz2
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/49, version=0.6.13): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 7: monitor d1_monitor_0 on xen-e (local)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/51, version=0.6.14): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 4: monitor d1_monitor_0 on xen-d
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/53, version=0.6.14): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 8: monitor s-libvirt_monitor_0 on xen-e (local)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/54, version=0.6.15): ok (rc=0)
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 5: monitor s-libvirt_monitor_0 on xen-d
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/55, version=0.6.16): ok (rc=0)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-e']/lrm (origin=local/crmd/56, version=0.6.17): ok (rc=0)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/lrm (origin=local/crmd/58, version=0.6.19): ok (rc=0)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/60, version=0.6.21): ok (rc=0)
Dec 14 20:07:19 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/62, version=0.6.24): ok (rc=0)
Dec 14 20:07:19 xen-e stonith-ng: [24224]: notice: stonith_device_action: Device s-libvirt not found
Dec 14 20:07:19 xen-e lrmd: [24225]: info: operation monitor[3] on s-libvirt for client 24228: pid 24244 exited with return code 7
Dec 14 20:07:19 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-12
Dec 14 20:07:19 xen-e lrmd: [24225]: info: operation monitor[2] on d1 for client 24228: pid 24243 exited with return code 7
Dec 14 20:07:19 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_monitor_0 (call=3, rc=7, cib-update=64, confirmed=true) not running
Dec 14 20:07:19 xen-e crmd: [24228]: info: process_lrm_event: LRM operation d1_monitor_0 (call=2, rc=7, cib-update=65, confirmed=true) not running
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 6: probe_complete probe_complete on xen-e (local) - no waiting
Dec 14 20:07:19 xen-e lrmd: [24225]: info: rsc:s-libvirt start[4] (pid 24264)
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 3: probe_complete probe_complete on xen-d - no waiting
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 9: start d1_start_0 on xen-d
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 11: start s-libvirt_start_0 on xen-e (local)
Dec 14 20:07:19 xen-e stonith-ng: [24224]: info: stonith_device_register: Added 's-libvirt' to the device list (1 active devices)
Dec 14 20:07:19 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_device_register from lrmd: rc=0
Dec 14 20:07:19 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 10: monitor d1_monitor_5000 on xen-d
Dec 14 20:07:19 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:20 xen-e external/libvirt(s-libvirt)[24271]: [24288]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:21 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:21 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:21 xen-e lrmd: [24225]: info: operation start[4] on s-libvirt for client 24228: pid 24264 exited with return code 0
Dec 14 20:07:21 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_start_0 (call=4, rc=0, cib-update=66, confirmed=true) ok
Dec 14 20:07:21 xen-e lrmd: [24225]: info: rsc:s-libvirt monitor[5] (pid 24289)
Dec 14 20:07:21 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 12: monitor s-libvirt_monitor_5000 on xen-e (local)
Dec 14 20:07:21 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:21 xen-e crmd: [24228]: info: handle_request: Current ping state: S_TRANSITION_ENGINE
Dec 14 20:07:22 xen-e crmd: [24228]: info: handle_request: Current ping state: S_TRANSITION_ENGINE
Dec 14 20:07:22 xen-e external/libvirt(s-libvirt)[24296]: [24313]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:23 xen-e crmd: [24228]: info: handle_request: Current ping state: S_TRANSITION_ENGINE
Dec 14 20:07:23 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:23 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:23 xen-e lrmd: [24225]: info: operation monitor[5] on s-libvirt for client 24228: pid 24289 exited with return code 0
Dec 14 20:07:23 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_monitor_5000 (call=5, rc=0, cib-update=67, confirmed=false) ok
Dec 14 20:07:23 xen-e crmd: [24228]: notice: run_graph: ==== Transition 1 (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-44.bz2): Complete
Dec 14 20:07:23 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:07:24 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:28 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:29 xen-e crmd: [24228]: info: process_graph_event: Action d1_monitor_5000 arrived after a completed transition
Dec 14 20:07:29 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:29 xen-e crmd: [24228]: info: abort_transition_graph: process_graph_event:481 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=d1_last_failure_0, magic=0:7;10:1:0:508011b9-a634-4079-959b-a66f62bb6df2, cib=0.6.36) : Inactive graph
Dec 14 20:07:29 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:29 xen-e crmd: [24228]: WARN: update_failcount: Updating failcount for d1 on xen-d after failed monitor: rc=7 (update=value++, time=1355512049)
Dec 14 20:07:29 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op d1_last_failure_0 on xen-d: not running (7)
Dec 14 20:07:29 xen-e crmd: [24228]: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 14 20:07:29 xen-e pengine: [24227]: notice: LogActions: Recover d1	(Started xen-d)
Dec 14 20:07:29 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:29 xen-e pengine: [24227]: notice: process_pe_message: Transition 2: PEngine Input stored in: /var/lib/pengine/pe-input-45.bz2
Dec 14 20:07:29 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 2 (ref=pe_calc-dc-1355512049-30) derived from /var/lib/pengine/pe-input-45.bz2
Dec 14 20:07:29 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 3: stop d1_stop_0 on xen-d
Dec 14 20:07:29 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair, id=status-xen-d-fail-count-d1, name=fail-count-d1, value=1, magic=NA, cib=0.6.37) : Transient attribute: update
Dec 14 20:07:29 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair, id=status-xen-d-last-failure-d1, name=last-failure-d1, value=1355512049, magic=NA, cib=0.6.38) : Transient attribute: update
Dec 14 20:07:29 xen-e crmd: [24228]: notice: run_graph: ==== Transition 2 (Complete=1, Pending=0, Fired=0, Skipped=3, Incomplete=0, Source=/var/lib/pengine/pe-input-45.bz2): Stopped
Dec 14 20:07:29 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:07:29 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:29 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:29 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:29 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 3 (ref=pe_calc-dc-1355512049-32) derived from /var/lib/pengine/pe-input-46.bz2
Dec 14 20:07:29 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op d1_last_failure_0 on xen-d: not running (7)
Dec 14 20:07:29 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 6: start d1_start_0 on xen-d
Dec 14 20:07:29 xen-e pengine: [24227]: notice: common_apply_stickiness: d1 can fail 999999 more times on xen-d before being forced off
Dec 14 20:07:29 xen-e pengine: [24227]: notice: LogActions: Start   d1	(xen-d)
Dec 14 20:07:29 xen-e pengine: [24227]: notice: process_pe_message: Transition 3: PEngine Input stored in: /var/lib/pengine/pe-input-46.bz2
Dec 14 20:07:29 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 7: monitor d1_monitor_5000 on xen-d
Dec 14 20:07:29 xen-e crmd: [24228]: notice: run_graph: ==== Transition 3 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-46.bz2): Complete
Dec 14 20:07:29 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:07:29 xen-e external/libvirt(s-libvirt)[24321]: [24338]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:30 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:30 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:30 xen-e sshd[24339]: Accepted publickey for root from 10.2.13.1 port 60543 ssh2
Dec 14 20:07:30 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:30 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:35 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:36 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:36 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:36 xen-e external/libvirt(s-libvirt)[24371]: [24387]: ERROR: Failed to get status for xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen
Dec 14 20:07:36 xen-e external/libvirt(s-libvirt)[24371]: [24393]: ERROR: setlocale: No such file or directory
Dec 14 20:07:36 xen-e external/libvirt(s-libvirt)[24371]: [24393]: error: Cannot recv data: Warning: Identity file /root/.ssh/xen not accessible: No such file or directory.
Dec 14 20:07:36 xen-e external/libvirt(s-libvirt)[24371]: [24393]: Permission denied (publickey,keyboard-interactive). : Connection reset by peer
Dec 14 20:07:36 xen-e external/libvirt(s-libvirt)[24371]: [24393]: error: failed to connect to the hypervisor
Dec 14 20:07:36 xen-e sshd[24380]: Accepted publickey for root from 10.2.13.1 port 60545 ssh2
Dec 14 20:07:37 xen-e stonith-ng: [24224]: notice: log_operation: Operation 'monitor' [24365] for device 's-libvirt' returned: -2
Dec 14 20:07:37 xen-e stonith: [24366]: WARN: external_status: 'libvirt status' failed with rc 1
Dec 14 20:07:37 xen-e stonith-ng: [24224]: ERROR: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:37 xen-e stonith: [24366]: ERROR: external/libvirt device not accessible.
Dec 14 20:07:37 xen-e stonith-ng: [24224]: ERROR: log_operation: s-libvirt: failed:  1
Dec 14 20:07:37 xen-e lrm-stonith: [24364]: WARN: map_ra_retvalue: Mapped the invalid return code -2.
Dec 14 20:07:37 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_monitor_5000 (call=5, rc=1, cib-update=70, confirmed=false) unknown error
Dec 14 20:07:37 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-s-libvirt (1)
Dec 14 20:07:37 xen-e crmd: [24228]: info: process_graph_event: Detected action s-libvirt_monitor_5000 from a different transition: 1 vs. 3
Dec 14 20:07:37 xen-e attrd: [24226]: notice: attrd_perform_update: Sent update 19: fail-count-s-libvirt=1
Dec 14 20:07:37 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:37 xen-e crmd: [24228]: info: abort_transition_graph: process_graph_event:476 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=s-libvirt_last_failure_0, magic=0:1;12:1:0:508011b9-a634-4079-959b-a66f62bb6df2, cib=0.6.42) : Old event
Dec 14 20:07:37 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-s-libvirt (1355512057)
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:37 xen-e crmd: [24228]: WARN: update_failcount: Updating failcount for s-libvirt on xen-e after failed monitor: rc=1 (update=value++, time=1355512057)
Dec 14 20:07:37 xen-e attrd: [24226]: notice: attrd_perform_update: Sent update 22: last-failure-s-libvirt=1355512057
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op s-libvirt_last_failure_0 on xen-e: unknown error (1)
Dec 14 20:07:37 xen-e crmd: [24228]: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 14 20:07:37 xen-e lrmd: [24225]: info: cancel_op: operation monitor[5] on s-libvirt for client 24228, its parameters: CRM_meta_name=[monitor] crm_feature_set=[3.0.6] CRM_meta_timeout=[60000] CRM_meta_interval=[5000] hostlist=[xen-d xen-e] hypervisor_uri=[xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen] reset_method=[reboot]  cancelled
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op d1_last_failure_0 on xen-d: not running (7)
Dec 14 20:07:37 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:176 - Triggered transition abort (complete=1, tag=nvpair, id=status-xen-e-fail-count-s-libvirt, name=fail-count-s-libvirt, value=1, magic=NA, cib=0.6.43) : Transient attribute: update
Dec 14 20:07:37 xen-e lrmd: [24225]: info: rsc:s-libvirt stop[6] (pid 24417)
Dec 14 20:07:37 xen-e pengine: [24227]: notice: common_apply_stickiness: d1 can fail 999999 more times on xen-d before being forced off
Dec 14 20:07:37 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:176 - Triggered transition abort (complete=1, tag=nvpair, id=status-xen-e-last-failure-s-libvirt, name=last-failure-s-libvirt, value=1355512057, magic=NA, cib=0.6.44) : Transient attribute: update
Dec 14 20:07:37 xen-e pengine: [24227]: notice: LogActions: Recover s-libvirt	(Started xen-e)
Dec 14 20:07:37 xen-e crmd: [24228]: info: handle_response: pe_calc calculation pe_calc-dc-1355512057-35 is obsolete
Dec 14 20:07:37 xen-e pengine: [24227]: notice: process_pe_message: Transition 4: PEngine Input stored in: /var/lib/pengine/pe-input-47.bz2
Dec 14 20:07:37 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:37 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:37 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 5 (ref=pe_calc-dc-1355512057-36) derived from /var/lib/pengine/pe-input-48.bz2
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:37 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 2: stop s-libvirt_stop_0 on xen-e (local)
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op s-libvirt_last_failure_0 on xen-e: unknown error (1)
Dec 14 20:07:37 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_monitor_5000 (call=5, status=1, cib-update=0, confirmed=true) Cancelled
Dec 14 20:07:37 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op d1_last_failure_0 on xen-d: not running (7)
Dec 14 20:07:37 xen-e pengine: [24227]: notice: common_apply_stickiness: d1 can fail 999999 more times on xen-d before being forced off
Dec 14 20:07:37 xen-e pengine: [24227]: notice: common_apply_stickiness: s-libvirt can fail 999999 more times on xen-e before being forced off
Dec 14 20:07:37 xen-e pengine: [24227]: notice: LogActions: Recover s-libvirt	(Started xen-e)
Dec 14 20:07:37 xen-e pengine: [24227]: notice: process_pe_message: Transition 5: PEngine Input stored in: /var/lib/pengine/pe-input-48.bz2
Dec 14 20:07:37 xen-e stonith-ng: [24224]: info: stonith_device_remove: Removed 's-libvirt' from the device list (0 active devices)
Dec 14 20:07:37 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_device_remove from lrmd: rc=0
Dec 14 20:07:37 xen-e lrmd: [24225]: info: operation stop[6] on s-libvirt for client 24228: pid 24417 exited with return code 0
Dec 14 20:07:37 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_stop_0 (call=6, rc=0, cib-update=74, confirmed=true) ok
Dec 14 20:07:37 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 10: start s-libvirt_start_0 on xen-e (local)
Dec 14 20:07:37 xen-e lrmd: [24225]: info: rsc:s-libvirt start[7] (pid 24418)
Dec 14 20:07:37 xen-e stonith-ng: [24224]: info: stonith_device_register: Added 's-libvirt' to the device list (1 active devices)
Dec 14 20:07:37 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_device_register from lrmd: rc=0
Dec 14 20:07:37 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:39 xen-e external/libvirt(s-libvirt)[24425]: [24442]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:40 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:40 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:40 xen-e lrmd: [24225]: info: operation start[7] on s-libvirt for client 24228: pid 24418 exited with return code 0
Dec 14 20:07:40 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_start_0 (call=7, rc=0, cib-update=75, confirmed=true) ok
Dec 14 20:07:40 xen-e lrmd: [24225]: info: rsc:s-libvirt monitor[8] (pid 24456)
Dec 14 20:07:40 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 1: monitor s-libvirt_monitor_5000 on xen-e (local)
Dec 14 20:07:40 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:41 xen-e external/libvirt(s-libvirt)[24463]: [24480]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:42 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:42 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:42 xen-e lrmd: [24225]: info: operation monitor[8] on s-libvirt for client 24228: pid 24456 exited with return code 0
Dec 14 20:07:42 xen-e crmd: [24228]: info: process_lrm_event: LRM operation s-libvirt_monitor_5000 (call=8, rc=0, cib-update=76, confirmed=false) ok
Dec 14 20:07:42 xen-e crmd: [24228]: notice: run_graph: ==== Transition 5 (Complete=3, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-48.bz2): Complete
Dec 14 20:07:42 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:07:42 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:42 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:07:47 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:48 xen-e corosync[24218]:  [TOTEM ] A processor failed, forming new configuration.
Dec 14 20:07:48 xen-e external/libvirt(s-libvirt)[24488]: [24505]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:49 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:49 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.54) 
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Transitional membership event on ring 1289824: memb=1, new=0, lost=1
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: memb: xen-e 923599370
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: lost: xen-d 906822154
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:07:54 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Stable membership event on ring 1289824: memb=1, new=0, lost=0
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: MEMB: xen-e 923599370
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: ais_mark_unseen_peer_dead: Node xen-d was not seen in the previous transition
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node 906822154/xen-d is now: lost
Dec 14 20:07:54 xen-e corosync[24218]:  [pcmk  ] info: send_member_notification: Sending membership update 1289824 to 2 children
Dec 14 20:07:54 xen-e corosync[24218]:  [TOTEM ] A processor joined or left the membership and a new membership was formed.
Dec 14 20:07:54 xen-e corosync[24218]:  [CPG   ] chosen downlist: sender r(0) ip(10.2.13.55) ; members(old:2 left:1)
Dec 14 20:07:54 xen-e corosync[24218]:  [MAIN  ] Completed service synchronization, ready to provide service.
Dec 14 20:07:54 xen-e cib: [24223]: notice: ais_dispatch_message: Membership 1289824: quorum lost
Dec 14 20:07:54 xen-e crmd: [24228]: notice: ais_dispatch_message: Membership 1289824: quorum lost
Dec 14 20:07:54 xen-e cib: [24223]: info: crm_update_peer: Node xen-d: id=906822154 state=lost (new) addr=r(0) ip(10.2.13.54)  votes=1 born=1289820 seen=1289820 proc=00000000000000000000000000151312
Dec 14 20:07:54 xen-e crmd: [24228]: info: ais_status_callback: status: xen-d is now lost (was member)
Dec 14 20:07:54 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/77, version=0.6.48): ok (rc=0)
Dec 14 20:07:54 xen-e crmd: [24228]: info: crm_update_peer: Node xen-d: id=906822154 state=lost (new) addr=r(0) ip(10.2.13.54)  votes=1 born=1289820 seen=1289820 proc=00000000000000000000000000151312
Dec 14 20:07:54 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/79, version=0.6.50): ok (rc=0)
Dec 14 20:07:54 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:07:54 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/81, version=0.6.51): ok (rc=0)
Dec 14 20:07:54 xen-e crmd: [24228]: WARN: match_down_event: No match for shutdown action on xen-d
Dec 14 20:07:54 xen-e crmd: [24228]: info: te_update_diff: Stonith/shutdown of xen-d not matched
Dec 14 20:07:54 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:234 - Triggered transition abort (complete=1, tag=node_state, id=xen-d, magic=NA, cib=0.6.49) : Node failure
Dec 14 20:07:54 xen-e crmd: [24228]: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Dec 14 20:07:54 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: pe_fence_node: Node xen-d will be fenced because it is un-expectedly down
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: determine_online_status: Node xen-d is unclean
Dec 14 20:07:54 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op s-libvirt_last_failure_0 on xen-e: unknown error (1)
Dec 14 20:07:54 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 6 (ref=pe_calc-dc-1355512074-41) derived from /var/lib/pengine/pe-warn-272.bz2
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op d1_last_failure_0 on xen-d: not running (7)
Dec 14 20:07:54 xen-e pengine: [24227]: notice: common_apply_stickiness: d1 can fail 999999 more times on xen-d before being forced off
Dec 14 20:07:54 xen-e crmd: [24228]: notice: te_fence_node: Executing reboot fencing operation (12) on xen-d (timeout=60000)
Dec 14 20:07:54 xen-e pengine: [24227]: notice: common_apply_stickiness: s-libvirt can fail 999999 more times on xen-e before being forced off
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: custom_action: Action d1_stop_0 on xen-d is unrunnable (offline)
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: initiate_remote_stonith_op: Initiating remote operation reboot for xen-d: 2444cfc7-97ba-40f2-b553-c6a764928d54
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: custom_action: Marking node xen-d unclean
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: stage6: Scheduling Node xen-d for STONITH
Dec 14 20:07:54 xen-e pengine: [24227]: notice: LogActions: Move    d1	(Started xen-d -> xen-e)
Dec 14 20:07:54 xen-e pengine: [24227]: WARN: process_pe_message: Transition 6: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/pengine/pe-warn-272.bz2
Dec 14 20:07:54 xen-e pengine: [24227]: notice: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: can_fence_host_with_device: Refreshing port list for s-libvirt
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: can_fence_host_with_device: s-libvirt can fence xen-d: dynamic-list
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: call_remote_stonith: Requesting that xen-e perform op reboot xen-d
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: can_fence_host_with_device: s-libvirt can fence xen-d: dynamic-list
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: stonith_fence: Found 1 matching devices for 'xen-d'
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_fence from xen-e: rc=-1
Dec 14 20:07:54 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:07:55 xen-e external/libvirt(s-libvirt)[24525]: [24540]: notice: Domain xen-d was rebooted
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/lrm (origin=local/crmd/84, version=0.6.53): ok (rc=0)
Dec 14 20:07:56 xen-e stonith-ng: [24224]: notice: log_operation: Operation 'reboot' [24519] (call 0 from c0c111a5-d332-48f7-9375-739b91e04f0e) for host 'xen-d' with device 's-libvirt' returned: 0
Dec 14 20:07:56 xen-e attrd: [24226]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Dec 14 20:07:56 xen-e crmd: [24228]: info: tengine_stonith_callback: StonithOp <st-reply st_origin="stonith_construct_async_reply" t="stonith-ng" st_op="reboot" st_remote_op="2444cfc7-97ba-40f2-b553-c6a764928d54" st_clientid="c0c111a5-d332-48f7-9375-739b91e04f0e" st_target="xen-d" st_device_action="st_fence" st_callid="0" st_callopt="0" st_rc="0" st_output="Performing: stonith -t external/libvirt -T reset xen-d success: xen-d 0 " src="xen-e" seq="4" state="2" />
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/transient_attributes (origin=local/crmd/85, version=0.6.54): ok (rc=0)
Dec 14 20:07:56 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -T reset xen-d
Dec 14 20:07:56 xen-e lrmd: [24225]: info: rsc:d1 start[9] (pid 24568)
Dec 14 20:07:56 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-s-libvirt (1355512057)
Dec 14 20:07:56 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:07:56 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/lrm
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/88, version=0.6.55): ok (rc=0)
Dec 14 20:07:56 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success: xen-d 0
Dec 14 20:07:56 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Dec 14 20:07:56 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:07:56 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/transient_attributes
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/89, version=0.6.56): ok (rc=0)
Dec 14 20:07:56 xen-e stonith-ng: [24224]: notice: remote_op_done: Operation reboot of xen-d by xen-e for xen-e[c0c111a5-d332-48f7-9375-739b91e04f0e]: OK
Dec 14 20:07:56 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-s-libvirt (1)
Dec 14 20:07:56 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op s-libvirt_last_failure_0 on xen-e: unknown error (1)
Dec 14 20:07:56 xen-e crmd: [24228]: notice: crmd_peer_update: Status update: Client xen-d/crmd now has status [offline] (DC=true)
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-e']/lrm (origin=local/crmd/90, version=0.6.57): ok (rc=0)
Dec 14 20:07:56 xen-e pengine: [24227]: notice: common_apply_stickiness: s-libvirt can fail 999999 more times on xen-e before being forced off
Dec 14 20:07:56 xen-e crmd: [24228]: notice: tengine_stonith_notify: Peer xen-d was terminated (reboot) by xen-e for xen-e: OK (ref=2444cfc7-97ba-40f2-b553-c6a764928d54)
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/92, version=0.6.59): ok (rc=0)
Dec 14 20:07:56 xen-e pengine: [24227]: notice: LogActions: Start   d1	(xen-e)
Dec 14 20:07:56 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:07:56 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/94, version=0.6.61): ok (rc=0)
Dec 14 20:07:56 xen-e pengine: [24227]: notice: process_pe_message: Transition 7: PEngine Input stored in: /var/lib/pengine/pe-input-49.bz2
Dec 14 20:07:56 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:169 - Triggered transition abort (complete=0) : Peer Halt
Dec 14 20:07:56 xen-e crmd: [24228]: notice: run_graph: ==== Transition 6 (Complete=2, Pending=0, Fired=0, Skipped=5, Incomplete=0, Source=/var/lib/pengine/pe-warn-272.bz2): Stopped
Dec 14 20:07:56 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:169 - Triggered transition abort (complete=1) : Peer Halt
Dec 14 20:07:56 xen-e crmd: [24228]: info: join_make_offer: Making join offers based on membership 1289824
Dec 14 20:07:56 xen-e lrmd: [24225]: info: operation start[9] on d1 for client 24228: pid 24568 exited with return code 0
Dec 14 20:07:56 xen-e crmd: [24228]: info: do_dc_join_offer_all: join-3: Waiting on 1 outstanding join acks
Dec 14 20:07:56 xen-e crmd: [24228]: info: cib_fencing_updated: Fencing update 83 for xen-d: complete
Dec 14 20:07:56 xen-e crmd: [24228]: info: update_dc: Set DC to xen-e (3.0.6)
Dec 14 20:07:56 xen-e crmd: [24228]: notice: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:07:56 xen-e crmd: [24228]: info: do_dc_join_finalize: join-3: Syncing the CIB from xen-e to the rest of the cluster
Dec 14 20:07:56 xen-e crmd: [24228]: info: do_dc_join_ack: join-3: Updating node state to member for xen-e
Dec 14 20:07:56 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-e']/lrm
Dec 14 20:07:56 xen-e crmd: [24228]: notice: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:07:56 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:162 - Triggered transition abort (complete=1) : Peer Cancelled
Dec 14 20:07:56 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:07:56 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 7 (ref=pe_calc-dc-1355512076-46) derived from /var/lib/pengine/pe-input-49.bz2
Dec 14 20:07:56 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 5: start d1_start_0 on xen-e (local)
Dec 14 20:07:56 xen-e crmd: [24228]: info: process_lrm_event: LRM operation d1_start_0 (call=9, rc=0, cib-update=96, confirmed=true) ok
Dec 14 20:07:57 xen-e lrmd: [24225]: info: rsc:d1 monitor[10] (pid 24577)
Dec 14 20:07:57 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 6: monitor d1_monitor_5000 on xen-e (local)
Dec 14 20:07:57 xen-e lrmd: [24225]: info: operation monitor[10] on d1 for client 24228: pid 24577 exited with return code 0
Dec 14 20:07:57 xen-e crmd: [24228]: info: process_lrm_event: LRM operation d1_monitor_5000 (call=10, rc=0, cib-update=97, confirmed=false) ok
Dec 14 20:07:57 xen-e crmd: [24228]: notice: run_graph: ==== Transition 7 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-49.bz2): Complete
Dec 14 20:07:57 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:07:57 xen-e external/libvirt(s-libvirt)[24555]: [24588]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:07:58 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:07:58 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:03 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:07 xen-e external/libvirt(s-libvirt)[24599]: [24616]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:08 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:08:08 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:13 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:15 xen-e external/libvirt(s-libvirt)[24630]: [24647]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:16 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:08:16 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:21 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:22 xen-e external/libvirt(s-libvirt)[24658]: [24678]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:23 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:08:23 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:28 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:29 xen-e external/libvirt(s-libvirt)[24689]: [24706]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:30 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:08:30 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:35 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:36 xen-e external/libvirt(s-libvirt)[24717]: [24734]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:37 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: Performing: stonith -t external/libvirt -S
Dec 14 20:08:37 xen-e stonith-ng: [24224]: info: log_operation: s-libvirt: success:  0
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Transitional membership event on ring 1289828: memb=1, new=0, lost=0
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: memb: xen-e 923599370
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] CLM CONFIGURATION CHANGE
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] New Configuration:
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.54) 
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.55) 
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] Members Left:
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] Members Joined:
Dec 14 20:08:40 xen-e cib: [24223]: notice: ais_dispatch_message: Membership 1289828: quorum acquired
Dec 14 20:08:40 xen-e corosync[24218]:  [CLM   ] 	r(0) ip(10.2.13.54) 
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] notice: pcmk_peer_update: Stable membership event on ring 1289828: memb=2, new=1, lost=0
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: update_member: Node 906822154/xen-d is now: member
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: NEW:  xen-d 906822154
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: MEMB: xen-d 906822154
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: pcmk_peer_update: MEMB: xen-e 923599370
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: send_member_notification: Sending membership update 1289828 to 2 children
Dec 14 20:08:40 xen-e corosync[24218]:  [TOTEM ] A processor joined or left the membership and a new membership was formed.
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: update_member: 0x6aeff0 Node 906822154 (xen-d) born on: 1289828
Dec 14 20:08:40 xen-e corosync[24218]:  [pcmk  ] info: send_member_notification: Sending membership update 1289828 to 2 children
Dec 14 20:08:40 xen-e crmd: [24228]: notice: ais_dispatch_message: Membership 1289828: quorum acquired
Dec 14 20:08:40 xen-e cib: [24223]: info: crm_update_peer: Node xen-d: id=906822154 state=member (new) addr=r(0) ip(10.2.13.54)  votes=1 born=1289820 seen=1289828 proc=00000000000000000000000000151312
Dec 14 20:08:40 xen-e crmd: [24228]: notice: crmd_peer_update: Status update: Client xen-d/crmd now has status [online] (DC=true)
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/lrm (origin=local/crmd/99, version=0.6.68): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: info: ais_status_callback: status: xen-d is now member (was lost)
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/transient_attributes (origin=local/crmd/100, version=0.6.69): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: WARN: match_down_event: No match for shutdown action on xen-d
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/101, version=0.6.70): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/lrm
Dec 14 20:08:40 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/transient_attributes
Dec 14 20:08:40 xen-e crmd: [24228]: info: crm_update_peer: Node xen-d: id=906822154 state=member (new) addr=r(0) ip(10.2.13.54)  votes=1 born=1289820 seen=1289828 proc=00000000000000000000000000151312 (new)
Dec 14 20:08:40 xen-e cib: [24223]: info: ais_dispatch_message: Membership 1289828: quorum retained
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/103, version=0.6.72): ok (rc=0)
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/105, version=0.6.73): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:08:40 xen-e crmd: [24228]: info: ais_dispatch_message: Membership 1289828: quorum retained
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/106, version=0.6.74): ok (rc=0)
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync_one for section 'all' (origin=xen-d/xen-d/(null), version=0.6.75): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: info: crmd_ais_dispatch: Setting expected votes to 2
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/109, version=0.6.76): ok (rc=0)
Dec 14 20:08:40 xen-e crmd: [24228]: notice: do_state_transition: State transition S_IDLE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=crmd_peer_update ]
Dec 14 20:08:40 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:169 - Triggered transition abort (complete=1) : Peer Halt
Dec 14 20:08:40 xen-e crmd: [24228]: info: join_make_offer: Making join offers based on membership 1289828
Dec 14 20:08:40 xen-e crmd: [24228]: info: do_dc_join_offer_all: join-4: Waiting on 2 outstanding join acks
Dec 14 20:08:40 xen-e crmd: [24228]: info: update_dc: Set DC to xen-e (3.0.6)
Dec 14 20:08:40 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync_one for section 'all' (origin=xen-d/xen-d/(null), version=0.6.76): ok (rc=0)
Dec 14 20:08:40 xen-e corosync[24218]:  [CPG   ] chosen downlist: sender r(0) ip(10.2.13.54) ; members(old:1 left:0)
Dec 14 20:08:40 xen-e corosync[24218]:  [MAIN  ] Completed service synchronization, ready to provide service.
Dec 14 20:08:41 xen-e crmd: [24228]: info: do_dc_join_offer_all: A new node joined the cluster
Dec 14 20:08:41 xen-e crmd: [24228]: info: do_dc_join_offer_all: join-5: Waiting on 2 outstanding join acks
Dec 14 20:08:41 xen-e crmd: [24228]: info: update_dc: Set DC to xen-e (3.0.6)
Dec 14 20:08:42 xen-e stonith-ng: [24224]: info: stonith_command: Processed st_execute from lrmd: rc=-1
Dec 14 20:08:42 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/112, version=0.6.76): ok (rc=0)
Dec 14 20:08:42 xen-e crmd: [24228]: notice: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:08:42 xen-e crmd: [24228]: info: do_dc_join_finalize: join-5: Syncing the CIB from xen-e to the rest of the cluster
Dec 14 20:08:42 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/113, version=0.6.77): ok (rc=0)
Dec 14 20:08:42 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/114, version=0.6.78): ok (rc=0)
Dec 14 20:08:42 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/transient_attributes (origin=xen-d/crmd/6, version=0.6.79): ok (rc=0)
Dec 14 20:08:43 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-d']/lrm (origin=local/crmd/115, version=0.6.80): ok (rc=0)
Dec 14 20:08:43 xen-e crmd: [24228]: info: do_dc_join_ack: join-5: Updating node state to member for xen-d
Dec 14 20:08:43 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-d']/lrm
Dec 14 20:08:43 xen-e crmd: [24228]: info: do_dc_join_ack: join-5: Updating node state to member for xen-e
Dec 14 20:08:43 xen-e crmd: [24228]: info: erase_status_tag: Deleting xpath: //node_state[@uname='xen-e']/lrm
Dec 14 20:08:43 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='xen-e']/lrm (origin=local/crmd/117, version=0.6.82): ok (rc=0)
Dec 14 20:08:43 xen-e attrd: [24226]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Dec 14 20:08:43 xen-e crmd: [24228]: notice: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Dec 14 20:08:43 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-s-libvirt (1355512057)
Dec 14 20:08:43 xen-e crmd: [24228]: info: abort_transition_graph: do_te_invoke:162 - Triggered transition abort (complete=1) : Peer Cancelled
Dec 14 20:08:43 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/119, version=0.6.84): ok (rc=0)
Dec 14 20:08:43 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:320 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=s-libvirt_last_0, magic=0:0;10:5:0:508011b9-a634-4079-959b-a66f62bb6df2, cib=0.6.82) : Resource op removal
Dec 14 20:08:43 xen-e crmd: [24228]: info: abort_transition_graph: te_update_diff:276 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.6.83) : LRM Refresh
Dec 14 20:08:43 xen-e cib: [24223]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/121, version=0.6.86): ok (rc=0)
Dec 14 20:08:43 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Dec 14 20:08:43 xen-e pengine: [24227]: notice: unpack_config: On loss of CCM Quorum: Ignore
Dec 14 20:08:43 xen-e crmd: [24228]: notice: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Dec 14 20:08:43 xen-e attrd: [24226]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-s-libvirt (1)
Dec 14 20:08:43 xen-e pengine: [24227]: WARN: unpack_nodes: Blind faith: not fencing unseen nodes
Dec 14 20:08:43 xen-e crmd: [24228]: info: do_te_invoke: Processing graph 8 (ref=pe_calc-dc-1355512123-60) derived from /var/lib/pengine/pe-input-50.bz2
Dec 14 20:08:43 xen-e pengine: [24227]: WARN: unpack_rsc_op: Processing failed op s-libvirt_last_failure_0 on xen-e: unknown error (1)
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 6: monitor d1_monitor_0 on xen-d
Dec 14 20:08:43 xen-e pengine: [24227]: notice: common_apply_stickiness: s-libvirt can fail 999999 more times on xen-e before being forced off
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 7: monitor s-libvirt_monitor_0 on xen-d
Dec 14 20:08:43 xen-e pengine: [24227]: notice: LogActions: Move    d1	(Started xen-e -> xen-d)
Dec 14 20:08:43 xen-e pengine: [24227]: notice: process_pe_message: Transition 8: PEngine Input stored in: /var/lib/pengine/pe-input-50.bz2
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 5: probe_complete probe_complete on xen-d - no waiting
Dec 14 20:08:43 xen-e lrmd: [24225]: info: cancel_op: operation monitor[10] on d1 for client 24228, its parameters: CRM_meta_name=[monitor] crm_feature_set=[3.0.6] CRM_meta_timeout=[30000] CRM_meta_interval=[5000]  cancelled
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 9: stop d1_stop_0 on xen-e (local)
Dec 14 20:08:43 xen-e lrmd: [24225]: info: rsc:d1 stop[11] (pid 24774)
Dec 14 20:08:43 xen-e crmd: [24228]: info: process_lrm_event: LRM operation d1_monitor_5000 (call=10, status=1, cib-update=0, confirmed=true) Cancelled
Dec 14 20:08:43 xen-e lrmd: [24225]: info: operation stop[11] on d1 for client 24228: pid 24774 exited with return code 0
Dec 14 20:08:43 xen-e crmd: [24228]: info: process_lrm_event: LRM operation d1_stop_0 (call=11, rc=0, cib-update=125, confirmed=true) ok
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 10: start d1_start_0 on xen-d
Dec 14 20:08:43 xen-e crmd: [24228]: info: te_rsc_command: Initiating action 11: monitor d1_monitor_5000 on xen-d
Dec 14 20:08:43 xen-e crmd: [24228]: notice: run_graph: ==== Transition 8 (Complete=8, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-50.bz2): Complete
Dec 14 20:08:43 xen-e crmd: [24228]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 14 20:08:43 xen-e external/libvirt(s-libvirt)[24748]: [24786]: notice: xen+ssh://hex-12.suse.de/?keyfile=/root/.ssh/xen: Running hypervisor: Xen 4.1.0
Dec 14 20:08:44 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
Dec 14 20:08:44 xen-e crmd: [24228]: info: handle_request: Current ping state: S_IDLE
